[ { "path": ".devcontainer/devcontainer.json", "content": "{\n \"name\": \"Tianshou\",\n \"dockerFile\": \"../Dockerfile\",\n \"workspaceFolder\": \"/workspaces/tianshou\",\n \"runArgs\": [\"--shm-size=1g\"],\n \"customizations\": {\n \"vscode\": {\n \"settings\": {\n \"terminal.integrated.shell.linux\": \"/bin/bash\",\n \"python.pythonPath\": \"/usr/local/bin/python\"\n },\n \"extensions\": [\n \"ms-python.python\",\n \"ms-toolsai.jupyter\",\n \"ms-python.vscode-pylance\"\n ]\n }\n },\n \"forwardPorts\": [],\n \"postCreateCommand\": \"poetry install --with dev\",\n \"remoteUser\": \"root\"\n }" }, { "path": ".dockerignore", "content": "data\nlogs\ntest/log\ndocs/jupyter_execute\ndocs/.jupyter_cache\n.lsp\n.clj-kondo\ndocs/_build\ncoverage*\n__pycache__\n*.egg-info\n*.egg\n.*cache\ndist" }, { "path": ".github/ISSUE_TEMPLATE.md", "content": "- [ ] I have marked all applicable categories:\n + [ ] exception-raising bug\n + [ ] RL algorithm bug\n + [ ] documentation request (i.e. \"X is missing from the documentation.\")\n + [ ] new feature request\n + [ ] design request (i.e. \"X should be changed to Y.\")\n- [ ] I have visited the [source website](https://github.com/thu-ml/tianshou/)\n- [ ] I have searched through the [issue tracker](https://github.com/thu-ml/tianshou/issues) for duplicates\n- [ ] I have mentioned version numbers, operating system and environment, where applicable:\n ```python\n import tianshou, gymnasium as gym, torch, numpy, sys\n print(tianshou.__version__, gym.__version__, torch.__version__, numpy.__version__, sys.version, sys.platform)\n ```\n" }, { "path": ".github/PULL_REQUEST_TEMPLATE.md", "content": "- [ ] I have added the correct label(s) to this Pull Request or linked the relevant issue(s)\n- [ ] I have provided a description of the changes in this Pull Request\n- [ ] I have added documentation for my changes and have listed relevant changes in CHANGELOG.md\n- [ ] If applicable, I have added tests to cover my changes.\n- [ ] If applicable, I have made sure that the determinism tests run through, meaning that my changes haven't influenced any aspect of training. See info in the contributing documentation.\n- [ ] I have reformatted the code using `poe format` \n- [ ] I have checked style and types with `poe lint` and `poe type-check`\n- [ ] (Optional) I ran tests locally with `poe test` \n(or a subset of them with `poe test-reduced`) ,and they pass\n- [ ] (Optional) I have tested that documentation builds correctly with `poe doc-build`" }, { "path": ".github/workflows/extra_sys.yml", "content": "name: Windows/MacOS\n\non:\n pull_request:\n branches:\n - master\n push:\n branches:\n - master\n workflow_dispatch:\n inputs:\n debug_enabled:\n type: boolean\n description: 'Run the build with tmate debugging enabled (https://github.com/marketplace/actions/debugging-with-tmate)'\n required: false\n default: false\njobs:\n cpu-extra:\n runs-on: ${{ matrix.os }}\n if: \"!contains(github.event.head_commit.message, 'ci skip')\"\n strategy:\n matrix:\n os: [macos-latest, windows-latest]\n python-version: [3.11]\n steps:\n - name: Setup tmate session\n uses: mxschmitt/action-tmate@v3\n if: ${{ github.event_name == 'workflow_dispatch' && inputs.debug_enabled }}\n - name: Cancel previous run\n uses: styfle/cancel-workflow-action@0.11.0\n with:\n access_token: ${{ github.token }}\n - uses: actions/checkout@v3\n - name: Set up Python ${{ matrix.python-version }}\n uses: actions/setup-python@v4\n with:\n python-version: ${{ matrix.python-version }}\n # use poetry and cache installed packages, see https://github.com/marketplace/actions/python-poetry-action\n - name: Install poetry\n uses: abatilo/actions-poetry@v2\n - name: Setup a local virtual environment (if no poetry.toml file)\n run: |\n poetry config virtualenvs.create true --local\n poetry config virtualenvs.in-project true --local\n - uses: actions/cache@v3\n name: Define a cache for the virtual environment based on the dependencies lock file\n with:\n path: ./.venv\n key: venv-${{ hashFiles('poetry.lock') }}\n - name: Install the project dependencies\n # ugly as hell, but well...\n # see https://github.com/python-poetry/poetry/issues/7611\n run: poetry install --with dev || poetry install --with dev || poetry install --with dev\n - name: wandb login\n run: poetry run wandb login e2366d661b89f2bee877c40bee15502d67b7abef\n - name: Test with pytest\n run: poetry run poe test-reduced\n" }, { "path": ".github/workflows/gputest.yml", "content": "name: Ubuntu GPU\n\non:\n pull_request:\n branches:\n - master\n push:\n branches:\n - master\n workflow_dispatch:\n inputs:\n debug_enabled:\n type: boolean\n description: 'Run the build with tmate debugging enabled (https://github.com/marketplace/actions/debugging-with-tmate)'\n required: false\n default: false\n\njobs:\n gpu:\n runs-on: [self-hosted, Linux, X64]\n if: \"!contains(github.event.head_commit.message, 'ci skip')\"\n steps:\n - name: Setup tmate session\n uses: mxschmitt/action-tmate@v3\n if: ${{ github.event_name == 'workflow_dispatch' && inputs.debug_enabled }}\n - name: Cancel previous run\n uses: styfle/cancel-workflow-action@0.11.0\n with:\n access_token: ${{ github.token }}\n - uses: actions/checkout@v3\n - name: Set up Python 3.11\n uses: actions/setup-python@v4\n with:\n python-version: \"3.11\"\n # use poetry and cache installed packages, see https://github.com/marketplace/actions/python-poetry-action\n - name: Install poetry\n uses: abatilo/actions-poetry@v2\n - name: Setup a local virtual environment (if no poetry.toml file)\n run: |\n poetry config virtualenvs.create true --local\n poetry config virtualenvs.in-project true --local\n - uses: actions/cache@v3\n name: Define a cache for the virtual environment based on the dependencies lock file\n with:\n path: ./.venv\n key: venv-${{ hashFiles('poetry.lock') }}\n - name: Install the project dependencies\n run: |\n poetry install --with dev --extras \"envpool\"\n - name: wandb login\n run: |\n poetry run wandb login e2366d661b89f2bee877c40bee15502d67b7abef\n - name: Test with pytest\n run: |\n poetry run poe test\n" }, { "path": ".github/workflows/lint_and_docs.yml", "content": "name: Check Formatting/Typing and Build Docs\n\non:\n pull_request:\n branches:\n - master\n push:\n branches:\n - master\n workflow_dispatch:\n inputs:\n debug_enabled:\n type: boolean\n description: 'Run the build with tmate debugging enabled (https://github.com/marketplace/actions/debugging-with-tmate)'\n required: false\n default: false\n\njobs:\n check:\n runs-on: ubuntu-latest\n steps:\n - name: Setup tmate session\n uses: mxschmitt/action-tmate@v3\n if: ${{ github.event_name == 'workflow_dispatch' && inputs.debug_enabled }}\n - name: Cancel previous run\n uses: styfle/cancel-workflow-action@0.11.0\n with:\n access_token: ${{ github.token }}\n - uses: actions/checkout@v3\n - name: Set up Python 3.11\n uses: actions/setup-python@v4\n with:\n python-version: 3.11\n # use poetry and cache installed packages, see https://github.com/marketplace/actions/python-poetry-action\n - name: Install poetry\n uses: abatilo/actions-poetry@v2\n - name: Setup a local virtual environment (if no poetry.toml file)\n run: |\n poetry config virtualenvs.create true --local\n poetry config virtualenvs.in-project true --local\n - uses: actions/cache@v3\n name: Define a cache for the virtual environment based on the dependencies lock file\n with:\n path: ./.venv\n key: venv-${{ hashFiles('poetry.lock') }}\n - name: Install the project dependencies\n run: |\n poetry install --with dev --extras \"eval\"\n - name: Check formatting\n run: poetry run poe lint\n - name: Check typing\n run: poetry run poe type-check\n - name: Build docs\n run: MYSTNB_DEBUG=1 poetry run poe doc-build\n - name: Show errors (if any)\n if: failure()\n run: find docs/_build/reports -name \"*.err.log\" -exec echo \"--- {} ---\" \\; -exec cat {} \\;\n" }, { "path": ".github/workflows/publish.yaml", "content": "name: Upload Python Package\n\non:\n release:\n types: [created]\n\njobs:\n deploy:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v2\n - name: Set up Python\n uses: actions/setup-python@v1\n with:\n python-version: 3.11\n # use poetry and cache installed packages, see https://github.com/marketplace/actions/python-poetry-action\n - name: Install poetry\n uses: abatilo/actions-poetry@v2\n - name: Setup a local virtual environment (if no poetry.toml file)\n run: |\n poetry config virtualenvs.create true --local\n poetry config virtualenvs.in-project true --local\n - name: Build and publish\n env:\n POETRY_PYPI_TOKEN_PYPI: ${{ secrets.PYPI_TOKEN }}\n run: |\n if [ -z \"${POETRY_PYPI_TOKEN_PYPI}\" ]; then echo \"Set the PYPI_TOKEN variable in your repository secrets\"; exit 1; fi \n poetry publish --build\n" }, { "path": ".github/workflows/pytest.yml", "content": "name: Ubuntu\n\non:\n pull_request:\n branches:\n - master\n push:\n branches:\n - master\n workflow_dispatch:\n inputs:\n debug_enabled:\n type: boolean\n description: 'Run the build with tmate debugging enabled (https://github.com/marketplace/actions/debugging-with-tmate)'\n required: false\n default: false\n\n\n# This job runs the test suite in two environments:\n# - py_pinned: uses Python 3.11 with the existing poetry.lock file (our stable, pinned dev environment)\n# - py_latest: latest Python version we want to support, without the lock file to furthermore install the newest dependency versions\n#\n# This ensures compatibility with both our controlled dev setup and the latest upstream packages,\n# helping catch issues introduced by dependency updates or newer Python versions.\njobs:\n cpu:\n runs-on: ubuntu-latest\n if: \"!contains(github.event.head_commit.message, 'ci skip')\"\n strategy:\n matrix:\n include:\n - env_name: py_pinned\n python-version: \"3.11\"\n use_lock: true\n - env_name: py_latest\n python-version: \"3.13\"\n use_lock: false\n\n steps:\n - name: Setup tmate session\n uses: mxschmitt/action-tmate@v3\n if: ${{ github.event_name == 'workflow_dispatch' && inputs.debug_enabled }}\n\n - name: Cancel previous run\n uses: styfle/cancel-workflow-action@0.11.0\n with:\n access_token: ${{ github.token }}\n\n - uses: actions/checkout@v3\n\n - name: Set up Python ${{ matrix.python-version }}\n uses: actions/setup-python@v4\n with:\n python-version: ${{ matrix.python-version }}\n\n - name: Install poetry\n uses: abatilo/actions-poetry@v2\n\n - name: Setup a local virtual environment (if no poetry.toml file)\n run: |\n poetry config virtualenvs.create true --local\n poetry config virtualenvs.in-project true --local\n\n - name: Remove poetry.lock for latest dependency test\n if: ${{ !matrix.use_lock }}\n run: rm -f poetry.lock\n\n - name: Define a cache for the virtual environment based on the dependencies lock file\n if: matrix.use_lock\n uses: actions/cache@v3\n with:\n path: ./.venv\n key: venv-${{ matrix.env_name }}-${{ hashFiles('poetry.lock') }}\n restore-keys: |\n venv-${{ matrix.env_name }}-\n\n - name: Install the project dependencies\n run: |\n if [ \"${{ matrix.env_name }}\" = \"py_latest\" ]; then\n poetry install --with dev --extras \"eval\"\n else\n poetry install --with dev --extras \"envpool eval\"\n fi\n\n - name: List installed packages\n run: |\n poetry run pip list\n\n - name: wandb login\n run: |\n poetry run wandb login e2366d661b89f2bee877c40bee15502d67b7abef\n\n - name: Test with pytest\n run: |\n if [ \"${{ matrix.env_name }}\" = \"py_pinned\" ]; then\n poetry run poe test\n else\n poetry run poe test-nocov\n fi\n - name: Upload coverage to Codecov\n if: matrix.env_name == 'py_pinned'\n uses: codecov/codecov-action@v1\n with:\n token: ${{ secrets.CODECOV }}\n file: ./coverage.xml\n flags: ${{ matrix.env_name }}\n name: codecov-${{ matrix.env_name }}\n fail_ci_if_error: false\n" }, { "path": ".gitignore", "content": "# Byte-compiled / optimized / DLL files\n__pycache__/\n*.py[cod]\n*$py.class\n\n# C extensions\n*.so\n\n# .idea folder\n.idea/\n\n# Distribution / packaging\n.Python\nbuild/\ndevelop-eggs/\ndist/\ndownloads/\neggs/\n.eggs/\nlib/\nlib64/\nparts/\nsdist/\nvar/\nwheels/\npip-wheel-metadata/\nshare/python-wheels/\n*.egg-info/\n.installed.cfg\n*.egg\nMANIFEST\n\n# PyInstaller\n# Usually these files are written by a python script from a template\n# before PyInstaller builds the exe, so as to inject date/other infos into it.\n*.manifest\n*.spec\n\n# Installer logs\npip-log.txt\npip-delete-this-directory.txt\n\n# Unit test / coverage reports\nhtmlcov/\n.tox/\n.nox/\n.coverage\n.coverage.*\n.cache\nnosetests.xml\ncoverage.xml\n*.cover\n*.py,cover\n.hypothesis/\n.pytest_cache/\ncover/\n\n# Translations\n*.mo\n*.pot\n\n# Django stuff:\n*.log\nlocal_settings.py\ndb.sqlite3\ndb.sqlite3-journal\n\n# Flask stuff:\ninstance/\n.webassets-cache\n\n# Scrapy stuff:\n.scrapy\n\n# Sphinx documentation\ndocs/_build/\n\n# PyBuilder\ntarget/\n\n# Jupyter Notebook\n.ipynb_checkpoints\n\n# IPython\nprofile_default/\nipython_config.py\n\n# pyenv\n# For a library or package, you might want to ignore these files since the code is\n# intended to run in multiple environments; otherwise, check them in:\n# .python-version\n\n# pipenv\n# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.\n# However, in case of collaboration, if having platform-specific dependencies or dependencies\n# having no cross-platform support, pipenv may install dependencies that don't work, or not\n# install all needed dependencies.\n#Pipfile.lock\n\n# PEP 582; used by e.g. github.com/David-OConnor/pyflow\n__pypackages__/\n\n# Celery stuff\ncelerybeat-schedule\ncelerybeat.pid\n\n# SageMath parsed files\n*.sage.py\n\n# Environments\n.env\n.venv\nvenv/\n/ENV/\nenv.bak/\nvenv.bak/\n\n# Spyder project settings\n.spyderproject\n.spyproject\n\n# Rope project settings\n.ropeproject\n\n# mkdocs documentation\n/site\n\n# mypy\n.mypy_cache/\n.dmypy.json\ndmypy.json\n\n# Pyre type checker\n.pyre/\n\n# pytype static type analyzer\n.pytype/\n\n# customize\nlog/\nMUJOCO_LOG.TXT\n*.pth\n.vscode/\n.DS_Store\n*.zip\n*.pstats\n*.swp\n*.pkl\n*.hdf5\nwandb/\nvideos/\n\n# might be needed for IDE plugins that can't read ruff config\n.flake8\n\ndocs/notebooks/_build/\ndocs/conf.py\n\n# temporary scripts (for ad-hoc testing), temp folder\n/temp\n/temp*.py\n\n# Serena\n/.serena\n\n# determinism test snapshots\n/test/resources/determinism/\n" }, { "path": ".pre-commit-config.yaml", "content": "default_install_hook_types: [commit-msg, pre-commit]\ndefault_stages: [commit, manual]\nfail_fast: false\nrepos:\n - repo: https://github.com/pre-commit/pre-commit-hooks\n rev: v4.4.0\n hooks:\n - id: check-added-large-files\n - repo: local\n hooks:\n - id: ruff\n name: ruff\n entry: poetry run ruff\n require_serial: true\n language: system\n types: [python]\n - id: ruff-nb\n name: ruff-nb\n entry: poetry run nbqa ruff .\n require_serial: true\n language: system\n pass_filenames: false\n types: [python]\n - id: black\n name: black\n entry: poetry run black\n require_serial: true\n language: system\n types: [python]\n - id: poetry-check\n name: poetry check\n entry: poetry check\n language: system\n files: pyproject.toml\n pass_filenames: false\n - id: poetry-lock-check\n name: poetry lock check\n entry: poetry check\n args: [--lock]\n language: system\n pass_filenames: false\n - id: mypy\n name: mypy\n entry: poetry run mypy tianshou examples test\n # filenames should not be passed as they would collide with the config in pyproject.toml\n pass_filenames: false\n files: '^tianshou(/[^/]*)*/[^/]*\\.py$'\n language: system\n - id: mypy-nb\n name: mypy-nb\n entry: poetry run nbqa mypy\n language: system\n" }, { "path": ".readthedocs.yaml", "content": "# .readthedocs.yaml\n# Read the Docs configuration file\n# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details\n\n# Required\nversion: 2\n\n# Set the version of Python and other tools you might need\nbuild:\n os: ubuntu-22.04\n tools:\n python: \"3.11\"\n commands:\n - mkdir -p $READTHEDOCS_OUTPUT/html\n - curl -sSL https://install.python-poetry.org | python -\n# - ~/.local/bin/poetry config virtualenvs.create false\n - ~/.local/bin/poetry install --with dev -E eval\n## Same as poe tasks, but unfortunately poe doesn't work with poetry not creating virtualenvs\n - ~/.local/bin/poetry run python docs/autogen_rst.py\n - ~/.local/bin/poetry run which jupyter-book\n - ~/.local/bin/poetry run python docs/create_toc.py\n - ~/.local/bin/poetry run jupyter-book config sphinx docs/\n - ~/.local/bin/poetry run sphinx-build -W -b html docs $READTHEDOCS_OUTPUT/html\n" }, { "path": "CHANGELOG.md", "content": "# Release 2.0.0 (2025-12-01)\n\nThis major release of Tianshou is a big step towards cleaner design and improved usability.\n\nGiven the large extent of the changes, it was not possible to maintain compatibility with the previous version.\n * Persisted agents that were created with earlier versions cannot be loaded in v2.\n * Source code from v1 can, however, be migrated to v2 with minimal effort. \n See migration information below. For concrete examples, you may use git to diff individual \n example scripts with the corresponding ones in `v1.2.0`.\n\nThis release is brought to you by [Applied AI Institute gGmbH](https://www.appliedai-institute.de). \n\nDevelopers:\n * Dr. Dominik Jain (@opcode81)\n * Michael Panchenko (@MischaPanch) \n\n## Runtime Environment Compatibility\n\nTianshou v2 is now compatible with \n * Python 3.12 and Python 3.13 #1274\n * newer versions of gymnasium (v1+) and numpy (v2+)\n\nOur main test environment remains Python 3.11-based for the time being (see `poetry.lock`).\n\n## Trainer Abstraction\n\n* The trainer logic and configuration is now properly separated between the three cases of on-policy, off-policy\n and offline learning: The base class is no longer a \"God\" class (formerly `BaseTrainer`) which does it all; logic and functionality has moved\n to the respective subclasses (`OnPolicyTrainer`, `OffPolicyTrainer` and `OfflineTrainer`, with `OnlineTrainer`\n being introduced as a base class for the two former specialisations).\n\n* The trainers now use configuration objects with central documentation (which has been greatly improved to enhance\n clarity and usability in general); every type of trainer now has a dedicated configuration class which provides\n precisely the options that are applicable.\n\n* The interface has been streamlined with improved naming of functions/parameters and limiting the public interface to purely\n the methods and attributes a user should reasonably access.\n\n* Further changes potentially affecting usage:\n * We dropped the iterator semantics: Method `__next__` has been replaced by `execute_epoch`. #913\n * We no longer report outdated statistics (e.g. on rewards/returns when a training step does not collect any full\n episodes)\n * See also \"Issues resolved\" below (as issue resolution can result in usage changes) \n * The default value for `test_in_train` was changed from True to False (updating all usage sites to explicitly\n set the parameter), because False is the more natural default, which does not make assumptions about\n returns/score values computed for the data from a collection step being at all meaningful for early stopping\n * The management of epsilon-greedy exploration for discrete Q-learning algorithms has been simplified:\n * All respective Policy implementations (e.g. `DQNPolicy`, `C51Policy`, etc.) now accept two parameters\n `eps_training` and `eps_inference`, which allows the training and test collection cases to be sufficiently\n differentiated and makes the use of callback functions (`train_fn`, `test_fn`) unnecessary if only\n constants are to be set.\n * The setter method `set_eps` has been replaced with `set_eps_training` and `set_eps_inference` accordingly.\n \n* Further internal changes unlikely to affect usage:\n * Module `trainer.utils` was removed and the functions therein where moved to class `Trainer`\n * The two places that collected and evaluated test episodes (`_test_in_train` and `_reset`) in addition to \n `_test_step` were unified to use `_test_step` (with some minor parametrisation) and now log the results \n of the test step accordingly.\n\n* Issues resolved:\n * Methods `run` and `reset`: Parameter `reset_prior_to_run` of `run` was never respected if it was set to `False`,\n because the implementation of `__iter__` (now removed) would call `reset` regardless - and calling `reset`\n is indeed necessary, because it initializes the training. The parameter was removed and replaced by\n `reset_collectors` (such that `run` now replicates the parameters of `reset`).\n * Inconsistent configuration options now raise exceptions rather than silently ignoring the issue in the \n hope that default behaviour will achieve what the user intended.\n One condition where `test_in_train` was silently set to `False` was removed and replaced by a warning.\n * The stop criterion `stop_fn` did not consider scores as computed by `compute_score_fn` but instead always used\n mean returns (i.e. it was assumed that the default implementation of `compute_score_fn` applies).\n This is an inconsistency which has been resolved.\n * The `gradient_step` counter was flawed (as it made assumptions about the underlying algorithms, which were \n not valid). It has been replaced with an update step counter.\n Members of `InfoStats` and parameters of `Logger` (and subclasses) were changed accordingly.\n\n* Migration information at a glance:\n * Training parameters are now passed via instances of configuration objects instead of directly as keyword arguments:\n `OnPolicyTrainerParams`, `OffPolicyTrainerParams`, `OfflineTrainerParams`.\n * Changed parameter default: Default for `test_in_train` was changed from True to False.\n * Changed parameter names to improve clarity:\n * `max_epoch` (`num_epochs` in high-level API) -> `max_epochs`\n * `step_per_epoch` -> `epoch_num_steps`\n * `episode_per_test` (`num_test_episodes` in high-level API) -> `test_step_num_episodes`\n * `step_per_collect` -> `collection_step_num_env_steps`\n * `episode_per_collect` -> collection_step_num_episodes`\n * `update_per_step` -> `update_step_num_gradient_steps_per_sample`\n * `repeat_per_collect` -> `update_step_num_repetitions`\n * Trainer classes have been renamed:\n * `OnpolicyTrainer` -> `OnPolicyTrainer`\n * `OffpolicyTrainer` -> `OffPolicyTrainer`\n * Method `run`: The parameter `reset_prior_to_run` was removed and replaced by `reset_collectors` (see above).\n * Methods `run` and `reset`: The parameter `reset_buffer` was renamed to `reset_collector_buffers` for clarity\n * Trainers are no longer iterators; manual usage (not using `run`) should simply call `reset` followed by\n calls of `execute_epoch`.\n\n## Algorithms and Policies\n\n* We now conceptually differentiate between the learning algorithm and the policy being optimised:\n\n * The abstraction `BasePolicy` is thus replaced by `Algorithm` and `Policy`, and the package was renamed \n from `tianshou.policy` to `tianshou.algorithm`.\n\n * Migration information: The instantiation of a policy is replaced by the instantiation of an `Algorithm`,\n which is passed a `Policy`. In most cases, the former policy class name `Policy` is replaced by algorithm\n class ``; exceptions are noted below.\n \n * `ImitationPolicy` -> `OffPolicyImitationLearning`, `OfflineImitationLearning` \n * `PGPolicy` -> `Reinforce` \n * `MultiAgentPolicyManager` -> `MultiAgentOnPolicyAlgorithm`, `MultiAgentOffPolicyAlgorithm` \n * `MARLRandomPolicy` -> `MARLRandomDiscreteMaskedOffPolicyAlgorithm`\n\n For the respective subtype of `Policy` to use, see the respective algorithm class' constructor.\n \n* Interface changes/improvements:\n * Core methods have been renamed (and removed from the public interface; #898):\n * `process_fn` -> `_preprocess_batch`\n * `post_process_fn` -> `_postprocess_batch`\n * `learn` -> `_update_with_batch`\n * The updating interface has been cleaned up (#949):\n * Functions `update` and `_update_with_batch` (formerly `learn`) no longer have `*args` and `**kwargs`.\n * Instead, the interfaces for the offline, off-policy and on-policy cases are properly differentiated.\n * New method `run_training`: The `Algorithm` abstraction can now directly initiate the learning process via this method.\n * `Algorithms` no longer require `torch.optim.Optimizer` instances and instead require `OptimizerFactory` \n instances, which create the actual optimizers internally. #959\n The new `OptimizerFactory` abstraction simultaneously handles the creation of learning rate schedulers\n for the optimizers created (via method `with_lr_scheduler_factory` and accompanying factory abstraction \n `LRSchedulerFactory`).\n The parameter `lr_scheduler` has thus been removed from all algorithm constructors.\n * The flag `updating` has been removed (no internal usage, general usefulness questionable).\n * Removed `max_action_num`, instead read it off from `action_space`\n * Parameter changes:\n * `actor_step_size` -> `trust_region_size` in NP\n * `discount_factor` -> `gamma` (was already used internally almost everywhere) \n * `reward_normalization` -> `return_standardization` or `return_scaling` (more precise naming) or removed (was actually unsupported by Q-learning algorithms)\n * `return_standardization` in `Reinforce` and `DiscreteCRR` (as it applies standardization of returns)\n * `return_scaling` in actor-critic on-policy algorithms (A2C, PPO, GAIL, NPG, TRPO)\n * removed from Q-learning algorithms, where it was actually unsupported (DQN, C561, etc.)\n * `clip_grad` -> `max_grad_norm` (for consistency)\n * `clip_loss_grad` -> `huber_loss_delta` (allowing to control not only the use of the Huber loss but also its essential parameter)\n * `estimation_step` -> `n_step_return_horizon` (more precise naming)\n \n* Internal design improvements:\n\n * Introduced an abstraction for the alpha parameter (coefficient of the entropy term) \n in `SAC`, `DiscreteSAC` and other algorithms.\n * Class hierarchy:\n * Abstract base class `Alpha` base class with value property and update method\n * `FixedAlpha` for constant entropy coefficients\n * `AutoAlpha` for automatic entropy tuning (replaces the old tuple-based representation)\n * The (auto-)updating logic is now completely encapsulated, reducing the complexity of the algorithms.\n * Implementations for continuous and discrete cases now share the same abstraction,\n making the codebase more consistent while preserving the original functionality.\n \n * Introduced a policy base class `ContinuousPolicyWithExplorationNoise` which encapsulates noise generation \n for continuous action spaces (e.g. relevant to `DDPG`, `SAC` and `REDQ`).\n \n * Multi-agent RL methods are now differentiated by the type of the sub-algorithms being employed\n (`MultiAgentOnPolicyAlgorithm`, `MultiAgentOffPolicyAlgorithm`), which renders all interfaces clean.\n Helper class `MARLDispatcher` has been factored out to manage the dispatching of data to the respective agents.\n \n * Algorithms now internally use a wrapper (`Algorithm.Optimizer`) around the optimizers; creation is handled\n by method `_create_optimizer`. \n * This facilitates backpropagation steps with gradient clipping. \n * The optimizers of an Algorithm instance are now centrally tracked, such that we can ensure that the \n optimizers' states are handled alongside the model parameters when calling `state_dict` or `load_state_dict` \n on the `Algorithm` instance.\n Special handling of the restoration of optimizers' state dicts was thus removed from examples and tests.\n \n * Lagged networks (target networks) are now conveniently handled via the new algorithm mixins \n `LaggedNetworkPolyakUpdateAlgorithmMixin` and `LaggedNetworkFullUpdateAlgorithmMixin`. \n Using these mixins, \n \n * a lagged network can simply be added by calling `_add_lagged_network`\n * the torch method `train` must no longer be overridden to ensure that the target networks\n are never set to train mode/remain in eval mode (which was prone to errors),\n * a method which updates all target networks with their source networks is automatically\n provided and does not need to be implemented specifically for every algorithm \n (`_update_lagged_network_weights`). \n\n All classes which make use of lagged networks were updated to use these mixins, simplifying\n the implementations and reducing the potential for implementation errors.\n (In the BCQ implementation, the VAE network was not correctly handled, but due to the way \n in which examples were structured, it did not result in an error.)\n\n* Fixed issues in the class hierarchy (particularly critical violations of the Liskov substitution principle): \n * Introduced base classes (to retain factorization without abusive inheritance):\n * `ActorCriticOnPolicyAlgorithm`\n * `ActorCriticOffPolicyAlgorithm`\n * `ActorDualCriticsOffPolicyAlgorithm` (extends `ActorCriticOffPolicyAlgorithm`)\n * `QLearningOffPolicyAlgorithm`\n * `A2C`: Inherit from `ActorCriticOnPolicyAlgorithm` instead of `Reinforce`\n * `BDQN`:\n * Inherit from `QLearningOffPolicyAlgorithm` instead of `DQN`\n * Remove parameter `clip_loss_grad` (unused; only passed on to former base class)\n * Remove parameter `estimation_step`, for which only one option was valid \n * `C51`:\n * Inherit from `QLearningOffPolicyAlgorithm` instead of `DQN`\n * Remove parameters `clip_loss_grad` and `is_double` (unused; only passed on to former base class)\n * `CQL`:\n * Inherit directly from `OfflineAlgorithm` instead of `SAC` (off-policy).\n * Remove parameter `estimation_step` (now `n_step_return_horizon`), which was not actually used (it was only passed it on to its\n superclass).\n * `DiscreteBCQ`: \n * Inherit directly from `OfflineAlgorithm` instead of `DQN`\n * Remove unused parameters `clip_loss_grad` and `is_double`, which were only passed on to\n former the base class but actually unused. \n * `DiscreteCQL`: Remove unused parameters `clip_loss_grad` and `is_double`, which were only passed on to\n base class `QRDQN` (and unused by it).\n * `DiscreteCRR`: Inherit directly from `OfflineAlgorithm` instead of `Reinforce` (on-policy)\n * `FQF`: Remove unused parameters `clip_loss_grad` and `is_double`, which were only passed on to\n base class `QRDQN` (and unused by it).\n * `IQN`: Remove unused parameters `clip_loss_grad` and `is_double`, which were only passed on to \n base class `QRDQN` (and unused by it).\n * `NPG`: Inherit from `ActorCriticOnPolicyAlgorithm` instead of `A2C`\n * `QRDQN`: \n * Inherit from `QLearningOffPolicyAlgorithm` instead of `DQN`\n * Remove parameters `clip_loss_grad` and `is_double` (unused; only passed on to former base class) \n * `REDQ`: Inherit from `ActorCriticOffPolicyAlgorithm` instead of `DDPG`\n * `SAC`: Inherit from `ActorDualCriticsOffPolicyAlgorithm` instead of `DDPG`\n * `TD3`: Inherit from `ActorDualCriticsOffPolicyAlgorithm` instead of `DDPG`\n\n## High-Level API\n\n* Detailed optimizer configuration (analogous to the procedural API) is now possible:\n * All optimizers can be configured in the respective algorithm-specific `Params` object by using\n `OptimizerFactoryFactory` instances as parameter values (e.g. `optim`, `actor_optim`, `critic_optim`, etc.).\n * Learning rate schedulers remain separate parameters and now use `LRSchedulerFactoryFactory` \n instances. The respective parameter names now use the suffix `lr_scheduler` instead of `lr_scheduler_factory`\n (as the precise nature need not be reflected in the name; brevity is preferable).\n \n* `SamplingConfig` is replaced by `TrainingConfig` and subclasses differentiating off-policy and on-policy cases \n appropriately (`OnPolicyTrainingConfig`, `OffPolicyTrainingConfig`).\n * The `test_in_train` parameter is now exposed (default False).\n * Inapplicable arguments can no longer be set in the respective subclass (e.g. `OffPolicyTrainingConfig` does not\n contain parameter `repeat_per_collect`). \n * All parameter names have been aligned with the new names used by `TrainerParams` (see above).\n\n* Add option to customize the factory for the collector (`ExperimentBuilder.with_collector_factory`),\n adding the abstraction `CollectorFactory`. #1256\n\n## Peripheral Changes\n\n* The `Actor` classes have been renamed for clarity (#1091):\n * `BaseActor` -> `Actor` \n * `continuous.ActorProb` -> `ContinuousActorProbabilistic`\n * `coninuous.Actor` -> `ContinuousActorDeterministic`\n * `discrete.Actor` -> `DiscreteActor`\n* The `Critic` classes have been renamed for clarity (#1091):\n * `continuous.Critic` -> `ContinuousCritic`\n * `discrete.Critic` -> `DiscreteCritic`\n* Moved Atari helper modules `atari_network` and `atari_wrapper` to the library under `tianshou.env.atari`.\n* Fix issues pertaining to the torch device assignment of network components (#810):\n * Remove 'device' member (and the corresponding constructor argument) from the following classes:\n `BranchingNet`, `C51Net`, `ContinuousActorDeterministic`, `ContinuousActorProbabilistic`, `ContinuousCritic`, \n `DiscreteActor`, `DiscreteCritic`, `DQNet`, `FullQuantileFunction`, `ImplicitQuantileNetwork`, \n `IntrinsicCuriosityModule`, `MLPActor`, `MLP`, `Perturbation`, `QRDQNet`, `Rainbow`, `Recurrent`, \n `RecurrentActorProb`, `RecurrentCritic`, `VAE`\n * (Peripheral change:) Require the use of keyword arguments for the constructors of all of these classes \n* Clean up handling of modules that define attribute `output_dim`, introducing the explicit base class \n `ModuleWithVectorOutput`\n * Interfaces where one could specify either a module with `output_dim` or additionally provide the output \n dimension as an argument were changed to use `ModuleWithVectorOutput`.\n * The high-level API class `IntermediateModule` can now provide a `ModuleWithVectorOutput` instance \n (via adaptation if necessary).\n* The class hierarchy of supporting `nn.Module` implementations was cleaned up (#1091):\n * With the fundamental base classes `ActionReprNet` and `ActionReprNetWithVectorOutput`, we etablished a \n well-defined interface for the most commonly used `forward` interface in Tianshou's algorithms & policies. #948\n * Some network classes were renamed:\n * `ScaledObsInputModule` -> `ScaledObsInputActionReprNet` \n * `Rainbow` -> `RainbowNet` \n* All modules containing base classes were renamed from `base` to a more descriptive name, rendering\n file names unique.\n\n# Release 1.2.0 (2025-06-23)\n\nThis is the final release in the 1.x series before Tianshou v2.0.0.\nIt resolves performance regressions introduced in v1.1.0 and resolves several issues, \npartly by backporting improvements from the upcoming v2.0.0 release.\n\nThis release is brought to you by [Applied AI Institute gGmbH](https://www.appliedai-institute.de).\n\nCore developers:\n* Dr. Dominik Jain (@opcode81)\n* Michael Panchenko (@MischaPanch)\n\n## Changes/Improvements\n\n- `trainer`:\n - Custom scoring now supported for selecting the best model. #1202\n- `highlevel`:\n - `DiscreteSACExperimentBuilder`: Expose method `with_actor_factory_default` #1248 #1250\n - `ActorFactoryDefault`: Fix parameters for hidden sizes and activation not being \n passed on in the discrete case (affects `with_actor_factory_default` method of experiment builders)\n - `ExperimentConfig`: Do not inherit from other classes, as this breaks automatic handling by\n `jsonargparse` when the class is used to define interfaces (as in high-level API examples)\n - `AutoAlphaFactoryDefault`: Differentiate discrete and continuous action spaces\n and allow coefficient to be modified, adding an informative docstring\n (previous implementation was reasonable only for continuous action spaces)\n - Adjust usage in `atari_sac_hl` example accordingly.\n - `NPGAgentFactory`, `TRPOAgentFactory`: Fix optimizer instantiation including the actor parameters\n (which was misleadingly suggested in the docstring in the respective policy classes; docstrings were fixed),\n as the actor parameters are intended to be handled via natural gradients internally\n- `data`:\n - `ReplayBuffer`: Fix collection of empty episodes being disallowed \n - Collection was slow due to `isinstance` checks on Protocols and due to Buffer integrity validation. This was solved\n by no longer performing `isinstance` on Protocols and by making the integrity validation disabled by default.\n- Tests:\n - We have introduced extensive **determinism tests** which allow to validate whether\n training processes deterministically compute the same results across different development branches.\n This is an important step towards ensuring reproducibility and consistency, which will be \n instrumental in supporting Tianshou developers in their work, especially in the context of\n algorithm development and evaluation. \n \n## Breaking Changes\n\n- `trainer`:\n - `BaseTrainer.run` and `__iter__`: Resetting was never optional prior to running the trainer,\n yet the recently introduced parameter `reset_prior_to_run` of `run` suggested that it _was_ optional.\n Yet the parameter was ultimately not respected, because `__iter__` would always call `reset(reset_collectors=True, reset_buffer=False)`\n regardless. The parameter was removed; instead, the parameters of `run` now mirror the parameters of `reset`,\n and the implicit `reset` call in `__iter__` was removed. \n This aligns with upcoming changes in Tianshou v2.0.0. \n * NOTE: If you have been using a trainer without calling `run` but by directly iterating over it, you\n will need to call `reset` on the trainer explicitly before iterating over the trainer.\n * Using a trainer as an iterator is considered deprecated and support for this will be removed in Tianshou v2.0.0.\n- `data`:\n - `InfoStats` has a new non-optional field `best_score` which is used\n for selecting the best model. #1202\n- `highlevel`:\n - Change the way in which seeding is handled: The mechanism introduced in v1.1.0 \n was completely revised:\n - The `training_seed` and `test_seed` attributes were removed from `SamplingConfig`.\n Instead, the seeds are derived from the seed defined in `ExperimentConfig`.\n - Seed attributes of `EnvFactory` classes were removed. \n Instead, seeds are passed to methods of `EnvFactory`.\n\n# Release 1.1.0 (2024-08-10)\n\n**NOTE**: This release introduced (potentially severe) performance regressions in data collection, please switch to a newer release for better performance.\n\n## Highlights\n\n### Evaluation Package\n\nThis release introduces a new package `evaluation` that integrates best\npractices for running experiments (seeding test and train environmets) and for\nevaluating them using the [rliable](https://github.com/google-research/rliable)\nlibrary. This should be especially useful for algorithm developers for comparing\nperformances and creating meaningful visualizations. **This functionality is\ncurrently in alpha state** and will be further improved in the next releases.\nYou will need to install tianshou with the extra `eval` to use it.\n\nThe creation of multiple experiments with varying random seeds has been greatly\nfacilitated. Moreover, the `ExpLauncher` interface has been introduced and\nimplemented with several backends to support the execution of multiple\nexperiments in parallel.\n\nAn example for this using the high-level interfaces can be found\n[here](examples/mujoco/mujoco_ppo_hl_multi.py), examples that use low-level\ninterfaces will follow soon.\n\n### Improvements in Batch\n\nApart from that, several important\nextensions have been added to internal data structures, most notably to `Batch`.\nBatches now implement `__eq__` and can be meaningfully compared. Applying\noperations in a nested fashion has been significantly simplified, and checking\nfor NaNs and dropping them is now possible.\n\nOne more notable change is that torch `Distribution` objects are now sliced when\nslicing a batch. Previously, when a Batch with say 10 actions and a dist\ncorresponding to them was sliced to `[:3]`, the `dist` in the result would still\ncorrespond to all 10 actions. Now, the dist is also \"sliced\" to be the\ndistribution of the first 3 actions.\n\nA detailed list of changes can be found below.\n\n## Changes/Improvements\n\n- `evaluation`: New package for repeating the same experiment with multiple\n seeds and aggregating the results. #1074 #1141 #1183\n- `data`:\n - `Batch`:\n - Add methods `to_dict` and `to_list_of_dicts`. #1063 #1098\n - Add methods `to_numpy_` and `to_torch_`. #1098, #1117\n - Add `__eq__` (semantic equality check). #1098\n - `keys()` deprecated in favor of `get_keys()` (needed to make iteration\n consistent with naming) #1105.\n - Major: new methods for applying functions to values, to check for NaNs\n and drop them, and to set values. #1181\n - Slicing a batch with a torch distribution now also slices the\n distribution. #1181\n - `data.collector`:\n - `Collector`:\n - Introduced `BaseCollector` as a base class for all collectors.\n #1123\n - Add method `close` #1063\n - Method `reset` is now more granular (new flags controlling\n behavior). #1063\n - `CollectStats`: Add convenience\n constructor `with_autogenerated_stats`. #1063\n- `trainer`:\n - Trainers can now control whether collectors should be reset prior to\n training. #1063\n- `policy`:\n - introduced attribute `in_training_step` that is controlled by the trainer.\n #1123\n - policy automatically set to `eval` mode when collecting and to `train`\n mode when updating. #1123\n - Extended interface of `compute_action` to also support array-like inputs\n #1169\n- `highlevel`:\n - `SamplingConfig`:\n - Add support for `batch_size=None`. #1077\n - Add `training_seed` for explicit seeding of training and test\n environments, the `test_seed` is inferred from `training_seed`. #1074\n - `experiment`:\n - `Experiment` now has a `name` attribute, which can be set\n using `ExperimentBuilder.with_name` and\n which determines the default run name and therefore the persistence\n subdirectory.\n It can still be overridden in `Experiment.run()`, the new parameter\n name being `run_name` rather than\n `experiment_name` (although the latter will still be interpreted\n correctly). #1074 #1131\n - Add class `ExperimentCollection` for the convenient execution of\n multiple experiment runs #1131\n - The `World` object, containing all low-level objects needed for\n experimentation,\n can now be extracted from an `Experiment` instance. This enables\n customizing\n the experiment prior to its execution, bridging the low and high-level\n interfaces. #1187\n - `ExperimentBuilder`:\n - Add method `build_seeded_collection` for the sound creation of\n multiple\n experiments with varying random seeds #1131\n - Add method `copy` to facilitate the creation of multiple\n experiments from a single builder #1131\n - `env`:\n - Added new `VectorEnvType` called `SUBPROC_SHARED_MEM_AUTO` and used in\n for Atari and Mujoco venv creation. #1141\n- `utils`:\n - `logger`:\n - Loggers can now restore the logged data into python by using the\n new `restore_logged_data` method. #1074\n - Wandb logger extended #1183\n - `net.continuous.Critic`:\n - Add flag `apply_preprocess_net_to_obs_only` to allow the\n preprocessing network to be applied to the observations only (without\n the actions concatenated), which is essential for the case where we\n want\n to reuse the actor's preprocessing network #1128\n - `torch_utils` (new module)\n - Added context managers `torch_train_mode`\n and `policy_within_training_step` #1123\n - `print`\n - `DataclassPPrintMixin` now supports outputting a string, not just\n printing the pretty repr. #1141\n\n## Fixes\n\n- `highlevel`:\n - `CriticFactoryReuseActor`: Enable the Critic\n flag `apply_preprocess_net_to_obs_only` for continuous critics,\n fixing the case where we want to reuse an actor's preprocessing network\n for the critic (affects usages\n of the experiment builder method `with_critic_factory_use_actor` with\n continuous environments) #1128\n - Policy parameter `action_scaling` value `\"default\"` was not correctly\n transformed to a Boolean value for\n algorithms SAC, DDPG, TD3 and REDQ. The value `\"default\"` being truthy\n caused action scaling to be enabled\n even for discrete action spaces. #1191\n- `atari_network.DQN`:\n - Fix constructor input validation #1128\n - Fix `output_dim` not being set if `features_only`=True\n and `output_dim_added_layer` is not None #1128\n- `PPOPolicy`:\n - Fix `max_batchsize` not being used in `logp_old` computation\n inside `process_fn` #1168\n- Fix `Batch.__eq__` to allow comparing Batches with scalar array values #1185\n\n## Internal Improvements\n\n- `Collector`s rely less on state, the few stateful things are stored explicitly\n instead of through a `.data` attribute. #1063\n- Introduced a first iteration of a naming convention for vars in `Collector`s.\n #1063\n- Generally improved readability of Collector code and associated tests (still\n quite some way to go). #1063\n- Improved typing for `exploration_noise` and within Collector. #1063\n- Better variable names related to model outputs (logits, dist input etc.).\n #1032\n- Improved typing for actors and critics, using Tianshou classes\n like `Actor`, `ActorProb`, etc.,\n instead of just `nn.Module`. #1032\n- Added interfaces for most `Actor` and `Critic` classes to enforce the presence\n of `forward` methods. #1032\n- Simplified `PGPolicy` forward by unifying the `dist_fn` interface (see\n associated breaking change). #1032\n- Use `.mode` of distribution instead of relying on knowledge of the\n distribution type. #1032\n- Exception no longer raised on `len` of empty `Batch`. #1084\n- tests and examples are covered by `mypy`. #1077\n- `Actor` is more used, stricter typing by making it generic. #1077\n- Use explicit multiprocessing context for creating `Pipe` in `subproc.py`.\n #1102\n\n## Breaking Changes\n\n- `data`:\n - `Collector`:\n - Removed `.data` attribute. #1063\n - Collectors no longer reset the environment on initialization.\n Instead, the user might have to call `reset` expicitly or\n pass `reset_before_collect=True` . #1063\n - Removed `no_grad` argument from `collect` method (was unused in\n tianshou). #1123\n - `Batch`:\n - Fixed `iter(Batch(...)` which now behaves the same way\n as `Batch(...).__iter__()`.\n Can be considered a bugfix. #1063\n - The methods `to_numpy` and `to_torch` in are not in-place anymore\n (use `to_numpy_` or `to_torch_` instead). #1098, #1117\n - The method `Batch.is_empty` has been removed. Instead, the user can\n simply check for emptiness of Batch by using `len` on dicts. #1144\n - Stricter `cat_`, only concatenation of batches with the same structure\n is allowed. #1181\n - `to_torch` and `to_numpy` are no longer static methods.\n So `Batch.to_numpy(batch)` should be replaced by `batch.to_numpy()`.\n #1200\n- `utils`:\n - `logger`:\n - `BaseLogger.prepare_dict_for_logging` is now abstract. #1074\n - Removed deprecated and unused `BasicLogger` (only affects users who\n subclassed it). #1074\n - `utils.net`:\n - `Recurrent` now receives and returns\n a `RecurrentStateBatch` instead of a dict. #1077\n - Modules with code that was copied from sensAI have been replaced by\n imports from new dependency sensAI-utils:\n - `tianshou.utils.logging` is replaced with `sensai.util.logging`\n - `tianshou.utils.string` is replaced with `sensai.util.string`\n - `tianshou.utils.pickle` is replaced with `sensai.util.pickle`\n- `env`:\n - All VectorEnvs now return a numpy array of info-dicts on reset instead of\n a list. #1063\n- `policy`:\n - Changed interface of `dist_fn` in `PGPolicy` and all subclasses to take a\n single argument in both\n continuous and discrete cases. #1032\n- `AtariEnvFactory` constructor (in examples, so not really breaking) now\n requires explicit train and test seeds. #1074\n- `EnvFactoryRegistered` now requires an explicit `test_seed` in the\n constructor. #1074\n- `highlevel`:\n - `params`: The parameter `dist_fn` has been removed from the parameter\n objects (`PGParams`, `A2CParams`, `PPOParams`, `NPGParams`, `TRPOParams`).\n The correct distribution is now determined automatically based on the\n actor factory being used, avoiding the possibility of\n misspecification. Persisted configurations/policies continue to work as\n expected, but code must not specify the `dist_fn` parameter.\n #1194 #1195\n - `env`:\n - `EnvFactoryRegistered`: parameter `seed` has been replaced by the pair\n of parameters `training_seed` and `test_seed`\n Persisted instances will continue to work correctly.\n Subclasses such as `AtariEnvFactory` are also affected requires\n explicit train and test seeds. #1074\n - `VectorEnvType`: `SUBPROC_SHARED_MEM` has been replaced\n by `SUBPROC_SHARED_MEM_DEFAULT`. It is recommended to\n use `SUBPROC_SHARED_MEM_AUTO` instead. However, persisted configs will\n continue working. #1141\n\n## Tests\n\n- Fixed env seeding it `test_sac_with_il.py` so that the test doesn't fail\n randomly. #1081\n- Improved CI triggers and added telemetry (if requested by user) #1177\n- Improved environment used in tests.\n- Improved tests bach equality to check with scalar values #1185\n\n## Dependencies\n\n- [DeepDiff](https://github.com/seperman/deepdiff) added to help with diffs of\n batches in tests. #1098\n- Bumped black, idna, pillow\n- New extra \"eval\"\n- Bumped numba to >=60.0.0, permitting installation on python 3.12 # 1177\n- New dependency sensai-utils\n\n\n# Release 1.0.0 (2024-03-20)\n\nThis release focuses on updating and improving Tianshou internals (in particular, code quality) while creating relatively few breaking changes (apart from things like the python and dependencies' versions).\n\nWe view it as a significant step for transforming Tianshou into the go-to place both for RL researchers, as well as for RL practitioners working on industry projects. \n \nThis is the first release after the [appliedAI Institute](https://www.appliedai-institute.de/en/) (the [TransferLab](https://transferlab.ai/) division) has decided to further develop Tianshou and provide long-term support. \n\n## Breaking Changes\n- dropped support of python<3.11\n- dropped support of gym, from now on only Gymnasium envs are supported\n- removed functions like `offpolicy_trainer` in favor of `OffpolicyTrainer(...).run()` (this affects all example scripts)\n- several breaking changes related to removing `**kwargs` from signatures, renamings of internal attributes (like `critic1` -> `critic`)\n- Outputs of training methods are now dataclasses instead of dicts\n\n## Functionality Extensions\n### Major\n- High level interfaces for experiments, demonstrated by the new example scripts with names ending in `_hl.py`\n### Minor\n- Method to compute action directly from a policy's observation, can be used for unrolling\n- Support for custom keys in ReplayBuffer\n- Support for CalQL as part of CQL\n- Support for explicit setting of multiprocessing context for SubprocEnvWorker\n- `critic2` no longer has to be explicitly constructed and passed if it is supposed to be the same network as `critic` (formerly `critic1`)\n\n## Internal Improvements\n### Build and Docs\n- Completely changed the build pipeline. Tianshou now uses poetry, black, ruff, poethepoet, nbqa and other niceties.\n- Notebook tutorials are now part of the repository (previously they were in a drive). They were fixed and are executed during the build as integration tests, in addition to serving as documentation. Parts of the content have been improved.\n- Documentation is now built with jupyter book. JavaScript code has been slightly improved, JS dependencies are included as part of the repository.\n- Many improvements in docstrings\n### Typing\n- Adding `BatchPrototypes` to cover the fields needed and returned by methods relying on batches in a backwards compatible way\n- Removing `**kwargs` from policies' constructors\n- Overall, much stricter and more correct typing. Removing `kwargs` and replacing dicts by dataclasses in several places.\n- Making use of `Generic` to express different kinds of stats that can be returned by `learn` and `update`\n- Improved typing in `tests` and `examples`, close to passing mypy\n### General\n- Reduced duplication, improved readability and simplified code in several places\n- Use `dist.mode` instead of inferring `loc` or `argmax` from the `dist_fn` input\n\n## Contributions\n### The OG creators\n- @Trinkle23897 participated in almost all aspects of the coordination and reviewed most of the merged PRs\n- @nuance1979 participated in several discussions\n### From appliedAI\nThe team working on this release of Tianshou consisted of @opcode81 @MischaPanch @maxhuettenrauch @carlocagnetta @bordeauxred\n### External contributions\n- @BFAnas participated in several discussions and contributed the CalQL implementation, extending the pre-processing logic.\n- @dantp-ai fixed many mypy issues and improved the tests\n- @arnaujc91 improved the logic of computing deterministic actions\n- Many other contributors, among them many new ones participated in this release. The Tianshou team is very grateful for your contributions!\n\n\n# Older Releases\n\nSee [releases on GitHub](https://github.com/thu-ml/tianshou/releases)" }, { "path": "CONTRIBUTING.md", "content": "# Contributing to Tianshou\n\nPlease refer to the ['Developer Guide' on tianshou.org](https://tianshou.org/en/latest/04_developer_guide/developer_guide.html).\n" }, { "path": "Dockerfile", "content": "# Use the official Python image for the base image.\nFROM --platform=linux/amd64 python:3.11-slim\n\n# Set environment variables to make Python print directly to the terminal and avoid .pyc files.\nENV PYTHONUNBUFFERED=1\nENV PYTHONDONTWRITEBYTECODE=1\n\n# Install system dependencies required for the project.\nRUN apt-get update && apt-get install -y --no-install-recommends \\\n curl \\\n build-essential \\\n git \\\n wget \\\n unzip \\\n libvips-dev \\\n gnupg2 \\\n && rm -rf /var/lib/apt/lists/*\n\n\n# Install pipx.\nRUN python3 -m pip install --no-cache-dir pipx \\\n && pipx ensurepath\n\n# Add poetry to the path\nENV PATH=\"${PATH}:/root/.local/bin\"\n\n# Install the latest version of Poetry using pipx.\nRUN pipx install poetry\n\n# Set the working directory. IMPORTANT: can't be changed as needs to be in sync to the dir where the project is cloned\n# to in the codespace\nWORKDIR /workspaces/tianshou\n\n# Copy the pyproject.toml and poetry.lock files (if available) into the image.\nCOPY pyproject.toml poetry.lock* README.md /workspaces/tianshou/\n\nRUN poetry config virtualenvs.create false\nRUN poetry install --no-root --with dev\n\n# The entrypoint will perform an editable install, it is expected that the code is mounted in the container then\n# If you don't want to mount the code, you should override the entrypoint\nENTRYPOINT [\"/bin/bash\", \"-c\", \"poetry install --with dev && poetry run jupyter trust notebooks/*.ipynb docs/02_notebooks/*.ipynb && $0 $@\"]" }, { "path": "LICENSE", "content": "MIT License\n\nCopyright (c) 2022 Tianshou contributors\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n" }, { "path": "MANIFEST.in", "content": "include LICENSE\n" }, { "path": "README.md", "content": "
\n \n
\n\n---\n\n[![PyPI](https://img.shields.io/pypi/v/tianshou)](https://pypi.org/project/tianshou/) [![Conda](https://img.shields.io/conda/vn/conda-forge/tianshou)](https://github.com/conda-forge/tianshou-feedstock) [![Read the Docs](https://readthedocs.org/projects/tianshou/badge/?version=master)](https://tianshou.org/en/master/) [![Pytest](https://github.com/thu-ml/tianshou/actions/workflows/pytest.yml/badge.svg)](https://github.com/thu-ml/tianshou/actions) [![codecov](https://img.shields.io/codecov/c/gh/thu-ml/tianshou)](https://codecov.io/gh/thu-ml/tianshou) [![GitHub issues](https://img.shields.io/github/issues/thu-ml/tianshou)](https://github.com/thu-ml/tianshou/issues) [![GitHub stars](https://img.shields.io/github/stars/thu-ml/tianshou)](https://github.com/thu-ml/tianshou/stargazers) [![GitHub forks](https://img.shields.io/github/forks/thu-ml/tianshou)](https://github.com/thu-ml/tianshou/network) [![GitHub license](https://img.shields.io/github/license/thu-ml/tianshou)](https://github.com/thu-ml/tianshou/blob/master/LICENSE)\n\n> [!NOTE]\n> **Tianshou version 2 is here!** \n> \n> We have released the new major version of Tianshou on PyPI. \n> Version 2 is a complete overhaul of the software design of the procedural API, in which\n> * we establish a clear separation between learning algorithms and policies (via the separate abstractions `Algorithm` and `Policy`).\n> * we provide more well-defined, more usable interfaces with extensive documentation of all algorithm and trainer parameters,\n> renaming some parameters to make their names more consistent and intuitive.\n> * the class hierarchy is fully revised, establishing a clear separation between on-policy, off-policy and offline algorithms\n> at the type level and ensuring that all inheritance relationships are meaningful.\n> \n> Because of the extent of the changes, this version is not backwards compatible with previous versions of Tianshou.\n> For migration information, please see the [change log](CHANGELOG.md). \n\n**Tianshou** ([天授](https://baike.baidu.com/item/%E5%A4%A9%E6%8E%88)) is a reinforcement learning (RL) library based on pure PyTorch and [Gymnasium](http://github.com/Farama-Foundation/Gymnasium). Tianshou's main features at a glance are:\n\n1. Modular low-level interfaces for algorithm developers (RL researchers) that are both flexible, hackable and type-safe.\n1. Convenient high-level interfaces for applications of RL (training an implemented algorithm on a custom environment).\n1. Large scope: online (on- and off-policy) and offline RL, experimental support for multi-agent RL (MARL), experimental support for model-based RL, and more\n\nUnlike other reinforcement learning libraries, which may have complex codebases,\nunfriendly high-level APIs, or are not optimized for speed, Tianshou provides a high-performance, modularized framework\nand user-friendly interfaces for building deep reinforcement learning agents. One more aspect that sets Tianshou apart is its\ngenerality: it supports online and offline RL, multi-agent RL, and model-based algorithms.\n\nTianshou aims at enabling concise implementations, both for researchers and practitioners, without sacrificing flexibility.\n\nSupported algorithms include:\n\n- [Deep Q-Network (DQN)](https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf)\n- [Double DQN](https://arxiv.org/pdf/1509.06461.pdf)\n- [Dueling DQN](https://arxiv.org/pdf/1511.06581.pdf)\n- [Branching DQN](https://arxiv.org/pdf/1711.08946.pdf)\n- [Categorical DQN (C51)](https://arxiv.org/pdf/1707.06887.pdf)\n- [Rainbow DQN (Rainbow)](https://arxiv.org/pdf/1710.02298.pdf)\n- [Quantile Regression DQN (QRDQN)](https://arxiv.org/pdf/1710.10044.pdf)\n- [Implicit Quantile Network (IQN)](https://arxiv.org/pdf/1806.06923.pdf)\n- [Fully-parameterized Quantile Function (FQF)](https://arxiv.org/pdf/1911.02140.pdf)\n- [Policy Gradient (PG)](https://papers.nips.cc/paper/1713-policy-gradient-methods-for-reinforcement-learning-with-function-approximation.pdf)\n- [Natural Policy Gradient (NPG)](https://proceedings.neurips.cc/paper/2001/file/4b86abe48d358ecf194c56c69108433e-Paper.pdf)\n- [Advantage Actor-Critic (A2C)](https://openai.com/blog/baselines-acktr-a2c/)\n- [Trust Region Policy Optimization (TRPO)](https://arxiv.org/pdf/1502.05477.pdf)\n- [Proximal Policy Optimization (PPO)](https://arxiv.org/pdf/1707.06347.pdf)\n- [Deep Deterministic Policy Gradient (DDPG)](https://arxiv.org/pdf/1509.02971.pdf)\n- [Twin Delayed DDPG (TD3)](https://arxiv.org/pdf/1802.09477.pdf)\n- [Soft Actor-Critic (SAC)](https://arxiv.org/pdf/1812.05905.pdf)\n- [Randomized Ensembled Double Q-Learning (REDQ)](https://arxiv.org/pdf/2101.05982.pdf)\n- [Discrete Soft Actor-Critic (SAC-Discrete)](https://arxiv.org/pdf/1910.07207.pdf)\n- [Vanilla Imitation Learning](https://en.wikipedia.org/wiki/Apprenticeship_learning)\n- [Batch-Constrained deep Q-Learning (BCQ)](https://arxiv.org/pdf/1812.02900.pdf)\n- [Conservative Q-Learning (CQL)](https://arxiv.org/pdf/2006.04779.pdf)\n- [Twin Delayed DDPG with Behavior Cloning (TD3+BC)](https://arxiv.org/pdf/2106.06860.pdf)\n- [Discrete Batch-Constrained deep Q-Learning (BCQ-Discrete)](https://arxiv.org/pdf/1910.01708.pdf)\n- [Discrete Conservative Q-Learning (CQL-Discrete)](https://arxiv.org/pdf/2006.04779.pdf)\n- [Discrete Critic Regularized Regression (CRR-Discrete)](https://arxiv.org/pdf/2006.15134.pdf)\n- [Generative Adversarial Imitation Learning (GAIL)](https://arxiv.org/pdf/1606.03476.pdf)\n- [Prioritized Experience Replay (PER)](https://arxiv.org/pdf/1511.05952.pdf)\n- [Generalized Advantage Estimator (GAE)](https://arxiv.org/pdf/1506.02438.pdf)\n- [Posterior Sampling Reinforcement Learning (PSRL)](https://www.ece.uvic.ca/~bctill/papers/learning/Strens_2000.pdf)\n- [Intrinsic Curiosity Module (ICM)](https://arxiv.org/pdf/1705.05363.pdf)\n- [Hindsight Experience Replay (HER)](https://arxiv.org/pdf/1707.01495.pdf)\n\nOther noteworthy features:\n\n- Elegant framework with dual APIs:\n - Tianshou's high-level API maximizes ease of use for application development while still retaining a high degree\n of flexibility.\n - The fundamental procedural API provides a maximum of flexibility for algorithm development without being\n overly verbose.\n- State-of-the-art results in [MuJoCo benchmarks](https://github.com/thu-ml/tianshou/tree/master/examples/mujoco) for REINFORCE/A2C/TRPO/PPO/DDPG/TD3/SAC algorithms\n- Support for vectorized environments (synchronous or asynchronous) for all algorithms (see [usage](https://tianshou.readthedocs.io/en/master/01_tutorials/07_cheatsheet.html#parallel-sampling))\n- Support for super-fast vectorized environments based on [EnvPool](https://github.com/sail-sg/envpool/) for all algorithms (see [usage](https://tianshou.readthedocs.io/en/master/01_tutorials/07_cheatsheet.html#envpool-integration))\n- Support for recurrent state representations in actor networks and critic networks (RNN-style training for POMDPs) (see [usage](https://tianshou.readthedocs.io/en/master/01_tutorials/07_cheatsheet.html#rnn-style-training))\n- Support any type of environment state/action (e.g. a dict, a self-defined class, ...) [Usage](https://tianshou.readthedocs.io/en/master/01_tutorials/07_cheatsheet.html#user-defined-environment-and-different-state-representation)\n- Support for customized training processes (see [usage](https://tianshou.readthedocs.io/en/master/01_tutorials/07_cheatsheet.html#customize-training-process))\n- Support n-step returns estimation and prioritized experience replay for all Q-learning based algorithms; GAE, nstep and PER are highly optimized thanks to numba's just-in-time compilation and vectorized numpy operations\n- Support for multi-agent RL (see [usage](https://tianshou.readthedocs.io/en/master/01_tutorials/07_cheatsheet.html#multi-agent-reinforcement-learning))\n- Support for logging based on both [TensorBoard](https://www.tensorflow.org/tensorboard) and [W&B](https://wandb.ai/)\n- Support for multi-GPU training (see [usage](https://tianshou.readthedocs.io/en/master/01_tutorials/07_cheatsheet.html#multi-gpu))\n- Comprehensive documentation, PEP8 code-style checking, type checking and thorough [tests](https://github.com/thu-ml/tianshou/actions)\n\nIn Chinese, Tianshou means divinely ordained, being derived to the gift of being born.\nTianshou is a reinforcement learning platform, and the nature of RL is not learn from humans.\nSo taking \"Tianshou\" means that there is no teacher to learn from, but rather to learn by oneself through constant interaction with the environment.\n\n“天授”意指上天所授，引申为与生具有的天赋。天授是强化学习平台，而强化学习算法并不是向人类学习的，所以取“天授”意思是没有老师来教，而是自己通过跟环境不断交互来进行学习。\n\n## Installation\n\nTianshou is currently hosted on [PyPI](https://pypi.org/project/tianshou/) and [conda-forge](https://github.com/conda-forge/tianshou-feedstock). It requires Python >= 3.11.\n\nFor installing the most recent version of Tianshou, the best way is clone the repository and install it with [poetry](https://python-poetry.org/)\n(which you need to install on your system first)\n\n```bash\ngit clone git@github.com:thu-ml/tianshou.git\ncd tianshou\npoetry install\n```\n\nYou can also install the dev requirements by adding `--with dev` or the extras\nfor say mujoco and acceleration by [envpool](https://github.com/sail-sg/envpool)\nby adding `--extras \"mujoco envpool\"`\n\nIf you wish to install multiple extras, ensure that you include them in a single command. Sequential calls to `poetry install --extras xxx` will overwrite prior installations, leaving only the last specified extras installed.\nOr you may install all the following extras by adding `--all-extras`.\n\nAvailable extras are:\n\n- `atari` (for Atari environments)\n- `box2d` (for Box2D environments)\n- `classic_control` (for classic control (discrete) environments)\n- `mujoco` (for MuJoCo environments)\n- `mujoco-py` (for legacy mujoco-py environments[^1])\n- `pybullet` (for pybullet environments)\n- `robotics` (for gymnasium-robotics environments)\n- `vizdoom` (for ViZDoom environments)\n- `envpool` (for [envpool](https://github.com/sail-sg/envpool/) integration)\n- `argparse` (in order to be able to run the high level API examples)\n\n[^1]:\n `mujoco-py` is a legacy package and is not recommended for new projects.\n It is only included for compatibility with older projects.\n Also note that there may be compatibility issues with macOS newer than\n Monterey.\n\nOtherwise, you can install the latest release from PyPI (currently\nfar behind the master) with the following command:\n\n```bash\n$ pip install tianshou\n```\n\nIf you are using Anaconda or Miniconda, you can install Tianshou from conda-forge:\n\n```bash\n$ conda install tianshou -c conda-forge\n```\n\nAlternatively to the poetry install, you can also install the latest source version through GitHub:\n\n```bash\n$ pip install git+https://github.com/thu-ml/tianshou.git@master --upgrade\n```\n\nFinally, you may check the installation via your Python console as follows:\n\n```python\nimport tianshou\nprint(tianshou.__version__)\n```\n\nIf no errors are reported, you have successfully installed Tianshou.\n\n## Documentation\n\nFind example scripts in the [test/]( https://github.com/thu-ml/tianshou/blob/master/test) and [examples/](https://github.com/thu-ml/tianshou/blob/master/examples) folders.\n\nTutorials and API documentation are hosted on [tianshou.readthedocs.io](https://tianshou.readthedocs.io/).\n\n## Why Tianshou?\n\n### Comprehensive Functionality\n\n### High Software Engineering Standards\n\n| RL Platform | Documentation | Code Coverage | Type Hints | Last Update |\n| ------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------ | ----------------------------------------------------------------------------------------------------------------- |\n| [Stable-Baselines3](https://github.com/DLR-RM/stable-baselines3) | [![Documentation Status](https://readthedocs.org/projects/stable-baselines/badge/?version=master)](https://stable-baselines3.readthedocs.io/en/master/?badge=master) | [![coverage report](https://gitlab.com/araffin/stable-baselines3/badges/master/coverage.svg)](https://gitlab.com/araffin/stable-baselines3/-/commits/master) | :heavy_check_mark: | ![GitHub last commit](https://img.shields.io/github/last-commit/DLR-RM/stable-baselines3?label=last%20update) |\n| [Ray/RLlib](https://github.com/ray-project/ray/tree/master/rllib/) | [![](https://readthedocs.org/projects/ray/badge/?version=master)](http://docs.ray.io/en/master/rllib.html) | :heavy_minus_sign:(1) | :heavy_check_mark: | ![GitHub last commit](https://img.shields.io/github/last-commit/ray-project/ray?label=last%20update) |\n| [SpinningUp](https://github.com/openai/spinningup) | [![](https://img.shields.io/readthedocs/spinningup)](https://spinningup.openai.com/) | :x: | :x: | ![GitHub last commit](https://img.shields.io/github/last-commit/openai/spinningup?label=last%20update) |\n| [Dopamine](https://github.com/google/dopamine) | [![](https://img.shields.io/badge/docs-passing-green)](https://github.com/google/dopamine/tree/master/docs) | :x: | :x: | ![GitHub last commit](https://img.shields.io/github/last-commit/google/dopamine?label=last%20update) |\n| [ACME](https://github.com/deepmind/acme) | [![](https://img.shields.io/badge/docs-passing-green)](https://github.com/deepmind/acme/blob/master/docs/index.md) | :heavy_minus_sign:(1) | :heavy_check_mark: | ![GitHub last commit](https://img.shields.io/github/last-commit/deepmind/acme?label=last%20update) |\n| [Sample Factory](https://github.com/alex-petrenko/sample-factory) | [:heavy_minus_sign:](https://arxiv.org/abs/2006.11751) | [![codecov](https://codecov.io/gh/alex-petrenko/sample-factory/branch/master/graph/badge.svg)](https://codecov.io/gh/alex-petrenko/sample-factory) | :x: | ![GitHub last commit](https://img.shields.io/github/last-commit/alex-petrenko/sample-factory?label=last%20update) |\n| | | | | |\n| [Tianshou](https://github.com/thu-ml/tianshou) | [![Read the Docs](https://img.shields.io/readthedocs/tianshou)](https://tianshou.readthedocs.io/en/master) | [![codecov](https://img.shields.io/codecov/c/gh/thu-ml/tianshou)](https://codecov.io/gh/thu-ml/tianshou) | :heavy_check_mark: | ![GitHub last commit](https://img.shields.io/github/last-commit/thu-ml/tianshou?label=last%20update) |\n\n(1): it has continuous integration but the coverage rate is not available\n\n### Reproducible, High-Quality Results\n\nTianshou is rigorously tested. In contrast to other RL platforms, **our tests include the full agent training procedure for all of the implemented algorithms**. Our tests would fail once if any of the agents failed to achieve a consistent level of performance on limited epochs.\nOur tests thus ensure reproducibility.\nCheck out the [GitHub Actions](https://github.com/thu-ml/tianshou/actions) page for more detail.\n\nAtari and MuJoCo benchmark results can be found in the [examples/atari/](examples/atari/) and [examples/mujoco/](examples/mujoco/) folders respectively. **Our MuJoCo results reach or exceed the level of performance of most existing benchmarks.**\n\n### Algorithm Abstraction\n\nReinforcement learning algorithms are build on abstractions for\n\n- on-policy algorithms (`OnPolicyAlgorithm`),\n- off-policy algorithms (`OffPolicyAlgorithm`), and\n- offline algorithms (`OfflineAlgorithm`),\n\nall of which clearly separate the core algorithm from the training process and the respective environment interactions.\n\nIn each case, the implementation of an algorithm necessarily involves only the implementation of methods for\n\n- pre-processing a batch of data, augmenting it with necessary information/sufficient statistics for learning (`_preprocess_batch`),\n- updating model parameters based on an augmented batch of data (`_update_with_batch`).\n\nThe implementation of these methods suffices for a new algorithm to be applicable within Tianshou,\nmaking experimentation with new approaches particularly straightforward.\n\n## Quick Start\n\nTianshou provides two API levels:\n\n- the high-level interface, which provides ease of use for end users seeking to run deep reinforcement learning applications\n- the procedural interface, which provides a maximum of control, especially for very advanced users and developers of reinforcement learning algorithms.\n\nIn the following, let us consider an example application using the _CartPole_ gymnasium environment.\nWe shall apply the deep Q-network (DQN) learning algorithm using both APIs.\n\n### High-Level API\n\nIn the high-level API, the basis for an RL experiment is an `ExperimentBuilder`\nwith which we can build the experiment we then seek to run.\nSince we want to use DQN, we use the specialization `DQNExperimentBuilder`.\n\nThe high-level API provides largely declarative semantics, i.e. the code is\nalmost exclusively concerned with configuration that controls what to do\n(rather than how to do it).\n\n```python\nfrom tianshou.highlevel.config import OffPolicyTrainingConfig\nfrom tianshou.highlevel.env import (\n EnvFactoryRegistered,\n VectorEnvType,\n)\nfrom tianshou.highlevel.experiment import DQNExperimentBuilder, ExperimentConfig\nfrom tianshou.highlevel.params.algorithm_params import DQNParams\nfrom tianshou.highlevel.trainer import (\n EpochStopCallbackRewardThreshold,\n)\n\nexperiment = (\n DQNExperimentBuilder(\n EnvFactoryRegistered(\n task=\"CartPole-v1\",\n venv_type=VectorEnvType.DUMMY,\n training_seed=0,\n test_seed=10,\n ),\n ExperimentConfig(\n persistence_enabled=False,\n watch=True,\n watch_render=1 / 35,\n watch_num_episodes=100,\n ),\n OffPolicyTrainingConfig(\n max_epochs=10,\n epoch_num_steps=10000,\n batch_size=64,\n num_training_envs=10,\n num_test_envs=100,\n buffer_size=20000,\n collection_step_num_env_steps=10,\n update_step_num_gradient_steps_per_sample=1 / 10,\n ),\n )\n .with_dqn_params(\n DQNParams(\n lr=1e-3,\n gamma=0.9,\n n_step_return_horizon=3,\n target_update_freq=320,\n eps_training=0.3,\n eps_inference=0.0,\n ),\n )\n .with_model_factory_default(hidden_sizes=(64, 64))\n .with_epoch_stop_callback(EpochStopCallbackRewardThreshold(195))\n .build()\n)\nexperiment.run()\n```\n\nThe experiment builder takes three arguments:\n\n- the environment factory for the creation of environments. In this case,\n we use an existing factory implementation for gymnasium environments.\n- the experiment configuration, which controls persistence and the overall\n experiment flow. In this case, we have configured that we want to observe\n the agent's behavior after it is trained (`watch=True`) for a number of\n episodes (`watch_num_episodes=100`). We have disabled persistence, because\n we do not want to save training logs, the agent or its configuration for\n future use.\n- the training configuration, which controls fundamental training parameters,\n such as the total number of epochs we run the experiment for (`num_epochs=10`) \n and the number of environment steps each epoch shall consist of\n (`epoch_num_steps=10000`).\n Every epoch consists of a series of data collection (rollout) steps and\n training steps.\n The parameter `collection_step_num_env_steps` controls the amount of data that is\n collected in each collection step and after each collection step, we\n perform a training step, applying a gradient-based update based on a sample\n of data (`batch_size=64`) taken from the buffer of data that has been\n collected. For further details, see the documentation of configuration class.\n\nWe then proceed to configure some of the parameters of the DQN algorithm itself:\nFor instance, we control the epsilon parameter for exploration.\nWe want to use random exploration during rollouts for training (`eps_training`),\nbut we don't when evaluating the agent's performance in the test environments\n(`eps_inference`).\nFurthermore, we configure model parameters of the network for the Q function,\nparametrising the number of hidden layers of the default MLP factory.\n\nFind the script in [examples/discrete/discrete_dqn_hl.py](examples/discrete/discrete_dqn_hl.py).\nHere's a run (with the training time cut short):\n\n

\n \n

\n\nFind many further applications of the high-level API in the `examples/` folder;\nlook for scripts ending with `_hl.py`.\nNote that most of these examples require the extra `argparse`\n(install it by adding `--extras argparse` when invoking poetry).\n\n### Procedural API\n\nLet us now consider an analogous example in the procedural API.\nFind the full script in [examples/discrete/discrete_dqn.py](https://github.com/thu-ml/tianshou/blob/master/examples/discrete/discrete_dqn.py).\n\nFirst, import the relevant packages:\n\n```python\nimport gymnasium as gym\nimport tianshou as ts\nfrom tianshou.algorithm.modelfree.dqn import DiscreteQLearningPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import CollectStats\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.space_info import SpaceInfo\nfrom torch.utils.tensorboard import SummaryWriter\n```\n\nDefine hyper-parameters:\n\n```python\ntask = 'CartPole-v1'\nlr, epoch, batch_size = 1e-3, 10, 64\nnum_training_envs, num_test_envs = 10, 100\ngamma, n_step, target_freq = 0.9, 3, 320\nbuffer_size = 20000\neps_train, eps_test = 0.1, 0.05\nepoch_num_steps, collection_step_num_env_steps = 10000, 10\n```\n\nInitialize the logger:\n\n```python\nlogger = ts.utils.TensorboardLogger(SummaryWriter('log/dqn'))\n```\n\nCreate the environments:\n\n```python\n# You can also try SubprocVectorEnv, which will use parallelization\ntraining_envs = ts.env.DummyVectorEnv([lambda: gym.make(task) for _ in range(num_training_envs)])\ntest_envs = ts.env.DummyVectorEnv([lambda: gym.make(task) for _ in range(num_test_envs)])\n```\n\nCreate the network, policy, and algorithm:\n\n```python\n# Create the network\n# Note: You can easily define other networks.\n# See https://tianshou.readthedocs.io/en/master/01_tutorials/00_dqn.html#build-the-network\nenv = gym.make(task, render_mode=\"human\")\nassert isinstance(env.action_space, gym.spaces.Discrete)\nspace_info = SpaceInfo.from_env(env)\nstate_shape = space_info.observation_info.obs_shape\naction_shape = space_info.action_info.action_shape\nnet = Net(state_shape=state_shape, action_shape=action_shape, hidden_sizes=[128, 128, 128])\noptim = AdamOptimizerFactory(lr=lr)\n\n# Create the policy\npolicy = DiscreteQLearningPolicy(\n model=net,\n action_space=env.action_space,\n eps_training=eps_train,\n eps_inference=eps_test\n)\n\n# Create the algorithm with the policy and optimizer factory\nalgorithm = DQN(\n policy=policy,\n optim=AdamOptimizerFactory(lr=lr),\n gamma=gamma,\n n_step_return_horizon=n_step,\n target_update_freq=target_freq\n)\n```\n\nSet up the collectors:\n\n```python\ntraining_collector = ts.data.Collector[CollectStats](\n algorithm,\n training_envs,\n ts.data.VectorReplayBuffer(buffer_size, num_training_envs),\n exploration_noise=True,\n)\ntest_collector = ts.data.Collector[CollectStats](\n algorithm,\n test_envs,\n exploration_noise=True,\n) \n```\n\nLet's train the model using the algorithm:\n\n```python\nresult = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n collection_step_num_env_steps=collection_step_num_env_steps,\n test_step_num_episodes=num_test_envs,\n batch_size=batch_size,\n update_step_num_gradient_steps_per_sample=1 / collection_step_num_env_steps,\n stop_fn=lambda mean_rewards: mean_rewards >= env.spec.reward_threshold,\n logger=logger,\n test_in_training=True,\n )\n)\nprint(f\"Finished training in {result.timing.total_time} seconds\")\n```\n\nThis is how you could manually save/load the trained policy (it's exactly the same as loading a `torch.nn.module`):\n\n```python\ntorch.save(policy.state_dict(), 'dqn.pth')\npolicy.load_state_dict(torch.load('dqn.pth'))\n```\n\nNow let's watch the agent with 35 FPS:\n\n```python\ncollector = ts.data.Collector(policy, env, exploration_noise=True)\ncollector.collect(n_episode=1, render=1 / 35)\n```\n\nInspect the data saved in TensorBoard:\n\n```bash\n$ tensorboard --logdir log/dqn\n```\n\nPlease read the [documentation](https://tianshou.readthedocs.io) for advanced usage.\n\n## Contributing\n\nTianshou is still under development.\nFurther algorithms and features are continuously being added, and we always welcome contributions to help make Tianshou better.\nIf you would like to contribute, please check out [this link](https://tianshou.org/en/master/04_contributing/04_contributing.html).\n\n## Citing Tianshou\n\nIf you find Tianshou useful, please cite it in your publications.\n\n```latex\n@article{tianshou,\n author = {Jiayi Weng and Huayu Chen and Dong Yan and Kaichao You and Alexis Duburcq and Minghao Zhang and Yi Su and Hang Su and Jun Zhu},\n title = {Tianshou: A Highly Modularized Deep Reinforcement Learning Library},\n journal = {Journal of Machine Learning Research},\n year = {2022},\n volume = {23},\n number = {267},\n pages = {1--6},\n url = {http://jmlr.org/papers/v23/21-1127.html}\n}\n```\n\n## Acknowledgments\n\nTianshou is supported by [appliedAI Institute for Europe](https://www.appliedai-institute.de/en/),\nwho is committed to providing long-term support and development.\n\nTianshou was previously a reinforcement learning platform based on TensorFlow. You can check out the branch [`priv`](https://github.com/thu-ml/tianshou/tree/priv) for more detail. Many thanks to [Haosheng Zou](https://github.com/HaoshengZou)'s pioneering work for Tianshou before version 0.1.1.\n\nWe would like to thank [TSAIL](http://ml.cs.tsinghua.edu.cn/) and [Institute for Artificial Intelligence, Tsinghua University](http://ml.cs.tsinghua.edu.cn/thuai/) for providing such an excellent AI research platform.\n" }, { "path": "benchmark/run_benchmark.py", "content": "\"\"\"Benchmark orchestration script for evaluating Tianshou's algorithm implementations.\n\nThis module provides automated benchmarking capabilities for reinforcement learning algorithms\nacross different environments (Atari, MuJoCo). It manages parallel experiment execution using\ntmux sessions, handles experiment lifecycle, and aggregates results.\n\nKey features:\n- Discovers and runs multiple RL algorithm scripts in parallel\n- Manages concurrency limits to prevent resource exhaustion\n- Each script runs in its own isolated tmux session for easy monitoring\n- Supports multiple tasks/environments per benchmark run\n- Aggregates rliable evaluation results into a unified format\n- Configurable experiment parameters (epochs, environments, parallel workers)\n- Filtering capabilities to run subsets of algorithms or tasks\n\nThe script is designed to be run from the command line,\nallowing easy customization of benchmark parameters without code modification.\n\nExample usage:\n python run_benchmark.py --benchmark_type mujoco --num_experiments 5 --max_concurrent_sessions 4\n\"\"\"\n\nimport json\nimport subprocess\nimport sys\nimport time\nfrom pathlib import Path\nfrom typing import Literal\n\nfrom sensai.util import logging\nfrom sensai.util.logging import datetime_tag\n\nTMUX_SESSION_PREFIX = \"tianshou\"\n\n# Sleep durations in seconds\nTMUX_SESSION_START_DELAY = 2\nSESSION_CHECK_INTERVAL = 5\nCOMPLETION_CHECK_INTERVAL = 10\n\nlog = logging.getLogger(\"benchmark\")\n\n# Default tasks for each benchmark type\nDEFAULT_TASKS = {\n \"mujoco\": [\n \"Ant-v4\",\n \"HalfCheetah-v4\",\n \"Hopper-v4\",\n \"Humanoid-v4\",\n \"InvertedDoublePendulum-v4\",\n \"InvertedPendulum-v4\",\n \"Reacher-v4\",\n \"Swimmer-v4\",\n \"Walker2d-v4\",\n ],\n \"atari\": [\n \"PongNoFrameskip-v4\",\n \"BreakoutNoFrameskip-v4\",\n \"EnduroNoFrameskip-v4\",\n \"QbertNoFrameskip-v4\",\n \"MsPacmanNoFrameskip-v4\",\n \"SeaquestNoFrameskip-v4\",\n \"SpaceInvadersNoFrameskip-v4\",\n ],\n}\n\n\ndef find_script_paths(\n benchmark_type: str, exclude_filter: str | None = None, include_filter: str = \"**/*_hl.py\"\n) -> list[str]:\n \"\"\"Return all Python scripts matching the glob filter under examples/.\"\"\"\n base_dir = Path(__file__).parent.parent / \"examples\" / benchmark_type\n if not base_dir.exists():\n raise FileNotFoundError(f\"Directory '{base_dir}' does not exist.\")\n\n scripts = sorted(str(p) for p in base_dir.glob(include_filter))\n if not scripts:\n raise FileNotFoundError(\n f\"Did not find any scripts matching '{include_filter}' in '{base_dir}'.\"\n )\n\n # Apply exclusion filter if provided\n if exclude_filter:\n scripts = [s for s in scripts if not Path(s).match(exclude_filter)]\n if not scripts:\n raise FileNotFoundError(\n f\"No scripts remaining after applying exclude filter '{exclude_filter}'.\"\n )\n\n return scripts\n\n\ndef get_current_tmux_sessions(benchmark_type: str) -> list[str]:\n \"\"\"List active tmux sessions starting with TMUX_SESSION_PREFIX.\"\"\"\n try:\n output = subprocess.check_output([\"tmux\", \"list-sessions\"], stderr=subprocess.DEVNULL)\n sessions = [\n line.split(b\":\")[0].decode()\n for line in output.splitlines()\n if line.startswith(f\"{TMUX_SESSION_PREFIX}_{benchmark_type}\".encode())\n ]\n return sessions\n except subprocess.CalledProcessError:\n return []\n\n\ndef start_tmux_session(\n script_path: str,\n persistence_base_dir: Path | str,\n num_experiments: int,\n benchmark_type: str,\n task: str,\n max_epochs: int | None = None,\n epoch_num_steps: int | None = None,\n experiment_launcher: Literal[\"sequential\", \"joblib\"] | None = None,\n num_training_envs: int | None = None,\n num_test_envs: int | None = None,\n) -> bool:\n \"\"\"Start a tmux session running the given experiment script, returning True on success.\"\"\"\n # Normalize paths for Git Bash / Windows compatibility\n python_exec = sys.executable.replace(\"\\\\\", \"/\")\n script_path = script_path.replace(\"\\\\\", \"/\")\n persistence_base_dir = str(persistence_base_dir).replace(\"\\\\\", \"/\")\n\n # Include task name in session to avoid collisions when running multiple tasks\n script_name = Path(script_path).name.replace(\"_hl.py\", \"\")\n # Remove benchmark_type from name since we add it explicitly below\n script_name = script_name.replace(benchmark_type, \"\").strip(\"_\")\n\n session_name = f\"{TMUX_SESSION_PREFIX}_{benchmark_type}_{task}_{script_name}\"\n\n # Build command with optional max_epochs and epoch_num_steps\n cmd_args = f\"{python_exec} {script_path} --num_experiments {num_experiments} --persistence_base_dir {persistence_base_dir} --task {task}\"\n if max_epochs is not None:\n cmd_args += f\" --max_epochs {max_epochs}\"\n if epoch_num_steps is not None:\n cmd_args += f\" --epoch_num_steps {epoch_num_steps}\"\n if experiment_launcher is not None:\n cmd_args += f\" --experiment_launcher {experiment_launcher}\"\n if num_training_envs is not None:\n cmd_args += f\" --num_training_envs {num_training_envs}\"\n if num_test_envs is not None:\n cmd_args += f\" --num_test_envs {num_test_envs}\"\n\n cmd = [\n \"tmux\",\n \"new-session\",\n \"-d\",\n \"-s\",\n session_name,\n f\"{cmd_args}; echo 'Finished {script_path}'; tmux kill-session -t {session_name}\",\n ]\n try:\n subprocess.run(cmd, check=True)\n log.info(\n f\"Started {script_path} in session '{session_name}'. Attach with:\\ntmux attach -t {session_name}\"\n )\n return True\n except subprocess.CalledProcessError as e:\n log.error(f\"Failed to start {script_path} (session {session_name}): {e}\")\n return False\n\n\ndef aggregate_rliable_results(task_results_dir: str | Path) -> None:\n \"\"\"Aggregate rliable results from all experiments into a single results.json per environment.\n\n This form is expected by `benchmark.js` in the docs.\n \"\"\"\n task_results_dir = Path(task_results_dir)\n if not task_results_dir.exists():\n log.warning(f\"Benchmark results directory does not exist: '{task_results_dir}'\")\n return\n\n experiment_dirs = [d for d in task_results_dir.iterdir() if d.is_dir()]\n aggregated_results = []\n for experiment_dir in experiment_dirs:\n agent_name = experiment_dir.name.split(\"Experiment\")[0]\n if not agent_name:\n log.warning(\n f\"Could not extract agent name from directory: '{experiment_dir.name}', skipping...\"\n )\n continue\n\n rliable_file = experiment_dir / \"rliable_evaluation_test.json\"\n if not rliable_file.exists():\n log.warning(f\"Missing rliable results file: '{rliable_file}', skipping...\")\n continue\n\n try:\n with open(rliable_file) as f:\n result_entries = json.load(f)\n for result_entry in result_entries:\n result_entry[\"agent\"] = agent_name\n aggregated_results.append(result_entry)\n except (OSError, json.JSONDecodeError) as e:\n log.error(f\"Failed to read or parse '{rliable_file}': {e}\")\n continue\n\n if not aggregated_results:\n log.warning(f\"No results to aggregate for directory '{task_results_dir}'\")\n return\n\n aggregated_results_path = task_results_dir / \"results.json\"\n try:\n with open(aggregated_results_path, \"w\") as f:\n json.dump(aggregated_results, f, indent=4)\n log.info(f\"Aggregated {len(aggregated_results)} results to '{aggregated_results_path}'.\")\n except OSError as e:\n log.error(f\"Failed to write aggregated results to '{aggregated_results_path}': {e}\")\n\n\ndef main(\n max_concurrent_sessions: int | None = None,\n benchmark_type: Literal[\"mujoco\", \"atari\"] = \"atari\",\n num_experiments: int = 1,\n max_scripts: int = -1,\n tasks: list[str] | None = None,\n max_tasks: int = -1,\n max_epochs: int | None = None,\n epoch_num_steps: int | None = None,\n num_training_envs: int | None = None,\n num_test_envs: int | None = None,\n experiment_launcher: Literal[\"sequential\", \"joblib\"] | None = \"sequential\",\n include_filter: str = \"**/*_hl.py\",\n exclude_filter: str | None = None,\n) -> None:\n \"\"\"\n Run the benchmarking by executing each selected script in its default configuration\n (apart from explicitly overridden parameters) in its own tmux session in parallel.\n Note that if you have unclosed tmux sessions from previous runs, they might count\n towards the max_concurrent_sessions limit. You can terminate all sessions with\n `tmux kill-server`.\n\n :param max_concurrent_sessions: optionally restrict how many tmux sessions to open in parallel,\n each script will run in a tmux session\n :param benchmark_type: mujoco or atari\n :param num_experiments: number of experiments to run per script\n :param max_scripts: maximum number of scripts to run, -1 for all. Set this to a low number for testing.\n :param tasks: optional list of task names to run benchmarks on. If None, uses default tasks for the benchmark_type.\n :param max_tasks: maximum number of tasks to run, -1 for all. Set this to a low number for testing.\n :param max_epochs: optional maximum number of training epochs to pass to all scripts. If None, uses script defaults.\n :param epoch_num_steps: optional number of environment steps per epoch to pass to all scripts. If None, uses script defaults.\n :param num_training_envs: optional number of training environments to pass to all scripts. If None, uses script defaults.\n :param num_test_envs: optional number of test environments to pass to all scripts. If None, uses script defaults.\n :param experiment_launcher: type of experiment launcher to use, only has an effect if `num_experiments>1`.\n By default, will use the experiment launchers defined in the individual scripts.\n :param include_filter: glob pattern to include scripts\n :param exclude_filter: optional glob pattern to exclude scripts (e.g., \"*ddpg*\")\n :return:\n \"\"\"\n # Use default tasks if none provided\n if tasks is None:\n tasks = DEFAULT_TASKS.get(benchmark_type, [])\n if not tasks:\n raise ValueError(\n f\"No default tasks found for benchmark_type '{benchmark_type}'. Please provide tasks manually.\"\n )\n\n # Limit number of tasks if specified\n if max_tasks > 0:\n log.info(f\"Limiting to first {max_tasks}/{len(tasks)} tasks.\")\n tasks = tasks[:max_tasks]\n\n log.info(f\"Running benchmarks for {len(tasks)} task(s): {tasks}\")\n\n persistence_base_dir = Path(__file__).parent / \"logs\" / benchmark_type / datetime_tag()\n\n # file logger for the global benchmarking logs, each individual experiment will log to its own file\n log_file = persistence_base_dir / \"benchmarking_run.txt\"\n log_file.parent.mkdir(parents=True, exist_ok=True)\n logging.add_file_logger(log_file, append=False)\n\n scripts = find_script_paths(\n benchmark_type, exclude_filter=exclude_filter, include_filter=include_filter\n )\n if max_scripts > 0:\n log.info(f\"Limiting to first {max_scripts}/{len(scripts)} scripts.\")\n scripts = scripts[:max_scripts]\n if max_concurrent_sessions is None:\n max_concurrent_sessions = len(scripts)\n\n # Run benchmarks for each task\n for i_task, task in enumerate(tasks, 1):\n log.info(\n f\"=== Starting benchmark batch for '{benchmark_type}' on task '{task}' ({i_task}/{len(tasks)}) \"\n f\"for {len(scripts)} scripts with {max_concurrent_sessions} concurrent jobs ===\"\n )\n for i_script, script in enumerate(scripts, start=1):\n # Wait for free slot\n has_printed_waiting_message = False\n while len(get_current_tmux_sessions(benchmark_type)) >= max_concurrent_sessions:\n if not has_printed_waiting_message:\n log.info(\n f\"Max concurrent sessions reached ({max_concurrent_sessions}). \"\n f\"Current sessions:\\n{get_current_tmux_sessions(benchmark_type)}\\nWaiting for a free slot...\"\n )\n has_printed_waiting_message = True\n time.sleep(SESSION_CHECK_INTERVAL)\n\n log.info(f\"Starting script {i_script}/{len(scripts)} for task '{task}'\")\n session_started = start_tmux_session(\n script,\n benchmark_type=benchmark_type,\n persistence_base_dir=persistence_base_dir,\n num_experiments=num_experiments,\n task=task,\n max_epochs=max_epochs,\n epoch_num_steps=epoch_num_steps,\n experiment_launcher=experiment_launcher,\n num_training_envs=num_training_envs,\n num_test_envs=num_test_envs,\n )\n if session_started:\n time.sleep(TMUX_SESSION_START_DELAY) # Give tmux a moment to start the session\n\n has_printed_final_waiting_message = False\n # Wait for all sessions to complete before moving to next task\n while len(get_current_tmux_sessions(benchmark_type)) > 0:\n if not has_printed_final_waiting_message:\n log.info(\n f\"All scripts for task '{task}' have been started, waiting for completion of remaining tmux sessions:\\n\"\n f\"{get_current_tmux_sessions(benchmark_type)}\"\n )\n has_printed_final_waiting_message = True\n time.sleep(COMPLETION_CHECK_INTERVAL)\n log.info(f\"All tmux sessions for task '{task}' have completed.\")\n # Aggregate results for this specific task (scripts create task-named directory automatically)\n task_results_dir = persistence_base_dir / task\n log.info(f\"Aggregating results for task '{task}' from directory: {task_results_dir}\")\n try:\n aggregate_rliable_results(str(task_results_dir))\n except Exception as e:\n log.error(f\"Failed to aggregate rliable results for task '{task}': {e}\\nContinuing...\")\n\n log.info(\n f\"=== Benchmark batch completed for all {len(scripts)} scripts and all {len(tasks)} task(s) ===\"\n )\n\n\nif __name__ == \"__main__\":\n logging.run_cli(main)\n" }, { "path": "docs/.gitignore", "content": "/03_api/*\njupyter_execute\n_toc.yml\n.jupyter_cache\n" }, { "path": "docs/01_user_guide/00_training_process.md", "content": "# The Reinforcement Learning Process\n\nThe following diagram illustrates the key mechanisms underlying the learning process in model-free reinforcement learning algorithms.\nIt shows how the agent interacts with the environment, collects experiences, and periodically updates its policy based on those experiences.\n\n
\n \"RL\n
\n \n \n
\n
\n
\n\n\n\nAccordingly, the key entities involved in the learning process are:\n * The **environment**: This is the system the agent interacts with. \n It provides the agent with observable states and rewards based on the actions taken by the agent.\n * The agent's **policy**: This is the strategy used by the agent to decide which action to take in a given state. \n The policy can be deterministic or stochastic and is typically represented by a neural network in deep reinforcement learning.\n * The **replay buffer**: This is a data structure used to store the agent's experiences, which consist of state transitions, \n actions taken, and rewards received. \n The agent learns from past experience by sampling mini-batches from the buffer during the policy update phase.\n * The **learning algorithm**: This defines how the agent updates its policy based on the experiences stored in the replay buffer. \n Different algorithms have different update mechanisms, which can significantly affect the learning performance. \n In some cases, the algorithm may also involve additional components (specifically neural networks), such as target networks or value \n functions.\n\nThese entities have direct correspondences in Tianshou's codebase:\n * The environment is represented by an instance of a class that inherits from `gymnasium.Env`, which is a standard interface for \n reinforcement learning environments. \n In practice, environments are typically vectorized to enable parallel interactions, increasing efficiency.\n * The policy is encapsulated in the {class}`~tianshou.algorithm.algorithm_base.Policy` class, which provides methods for action selection.\n * The replay buffer is implemented in the {class}`~tianshou.data.buffer.buffer_base.ReplayBuffer` class.\n A {class}`~tianshou.data.collector.Collector` instance is used to manage the addition of new experiences to the replay buffer as the agent interacts with the \n environment. \n During the learning phase, the replay buffer may be sampled, providing an instance of {class}`~tianshou.data.batch.Batch` for the policy update.\n * The abstraction for learning algorithms is given by the {class}`~tianshou.algorithm.algorithm_base.Algorithm` class, which defines how to update the policy using data from the \n replay buffer.\n\n(structuring-the-process)=\n## Structuring the Process\n\nThe learning process itself is reified in Tianshou's {class}`~tianshou.trainer.trainer.Trainer` class, which orchestrates the interaction between the agent and the \nenvironment, manages the replay buffer, and coordinates the policy updates according to the specified learning algorithm. \n\nIn general, the process can be described as executing a number of epochs as follows:\n\n* **epoch**:\n * repeat until a sufficient number of steps is reached (for online learning, typically environment step count)\n * **training step**:\n * for online learning algorithms …\n * **collection step**: collect state transitions in the environment by running the agent\n * (optionally) conduct a test step if collected data indicates promising behaviour\n * **update step**: apply gradient updates using the algorithm’s update logic. \n The update is based on … \n * data from the preceding collection step only (on-policy learning)\n * data from the collection step and previous data (off-policy learning)\n * data from a user-provided replay buffer (offline learning)\n * **test step**\n * collect test episodes from dedicated test environments and evaluate agent performance\n * (optionally) stop training early if performance is sufficiently high\n\n```{admonition} Glossary\n:class: note\nThe above introduces some of the key terms used throughout Tianshou. \n```\n\nNote that the above description encompasses several modes of model-free reinforcement learning, including:\n * online learning (where the agent continuously interacts with the environment in order to collect new experiences)\n * on-policy learning (where the policy is updated based on data collected using the current policy only)\n * off-policy learning (where the policy is updated based on data collected using the current and previous policies)\n * offline learning (where the replay buffer is pre-filled and not updated during training)\n\nIn Tianshou, the {class}`~tianshou.trainer.trainer.Trainer` and {class}`~tianshou.algorithm.algorithm_base.Algorithm` classes are specialised to handle these different modes accordingly.\n" }, { "path": "docs/01_user_guide/01_apis.md", "content": "# Dual APIs\n\nTianshou provides two distinct APIs to serve different use cases and user preferences:\n\n1. **high-level API**: a declarative, configuration-based interface designed for ease of use\n2. **procedural API**: a flexible, imperative interface providing maximum control\n\nBoth APIs access the same underlying algorithm implementations, allowing you to choose the level \nof abstraction that best fits your needs without sacrificing functionality.\n\n## Overview\n\n### High-Level API\n\nThe high-level API is built around the **builder pattern** and **declarative semantics**. \nInstead of writing procedural code that sequentially constructs and connects components, \nyou declare _what_ you want through configuration objects and let Tianshou handle _how_ to \nbuild and execute the experiment.\n\n**Key characteristics:**\n- centered around {class}`~tianshou.highlevel.experiment.ExperimentBuilder` classes (e.g., {class}`~tianshou.highlevel.experiment.DQNExperimentBuilder`, {class}`~tianshou.highlevel.experiment.PPOExperimentBuilder`, etc.)\n- uses configuration dataclasses and factories for all relevant parameters\n- automatically handles component creation and \"wiring\"\n- provides sensible defaults that adapt to the nature of your environment\n- includes built-in persistence, logging, and experiment management\n- full type hints (but object structure is not flat; a proper IDE is required for seamless user experience) \n\n### Procedural API\n\nThe procedural API provides explicit control over every component in the RL pipeline. \nYou manually create environments, networks, policies, algorithms, collectors, and \ntrainers, then wire them together.\n\n**Key characteristics:**\n- direct instantiation of all components\n- explicit control over the training loop\n- lower-level access to internal mechanisms\n- minimal abstraction (closer to the implementation)\n- ideal for algorithm development and research\n\n## When to Use Which API\n\nUse the high-level API when ...\n\n- **you're applying existing algorithms** to new problems\n- **you want to get started quickly** with minimal boilerplate\n- **you need experiment management** with persistence, logging, and reproducibility\n- **you prefer declarative code** that focuses on configuration\n- **you're building applications** rather than developing new algorithms\n\nUse the procedural API when:\n\n- **you're developing new algorithms** or modifying existing ones\n- **you need fine-grained control** over the training process\n- **you want to understand** the internal workings of Tianshou\n- **you're implementing custom components** not supported by the high-level API\n- **you prefer imperative programming** where each step is explicit\n- **you need maximum flexibility** for experimental research\n\n## Comparison by Example\n\nLet's compare both APIs by implementing the same DQN learning task on the CartPole environment.\n\n### High-Level API Example\n\n```python\nfrom tianshou.highlevel.config import OffPolicyTrainingConfig\nfrom tianshou.highlevel.env import EnvFactoryRegistered, VectorEnvType\nfrom tianshou.highlevel.experiment import DQNExperimentBuilder, ExperimentConfig\nfrom tianshou.highlevel.params.algorithm_params import DQNParams\nfrom tianshou.highlevel.trainer import EpochStopCallbackRewardThreshold\n\n# Build the experiment through configuration\nexperiment = (\n DQNExperimentBuilder(\n # Environment configuration\n EnvFactoryRegistered(\n task=\"CartPole-v1\",\n venv_type=VectorEnvType.DUMMY,\n training_seed=0,\n test_seed=10,\n ),\n # Experiment settings\n ExperimentConfig(\n persistence_enabled=False,\n watch=True,\n watch_render=1 / 35,\n watch_num_episodes=100,\n ),\n # Training configuration\n OffPolicyTrainingConfig(\n max_epochs=10,\n epoch_num_steps=10000,\n batch_size=64,\n num_training_envs=10,\n num_test_envs=100,\n buffer_size=20000,\n collection_step_num_env_steps=10,\n update_step_num_gradient_steps_per_sample=1 / 10,\n ),\n )\n # Algorithm-specific parameters\n .with_dqn_params(\n DQNParams(\n lr=1e-3,\n gamma=0.9,\n n_step_return_horizon=3,\n target_update_freq=320,\n eps_training=0.3,\n eps_inference=0.0,\n ),\n )\n # Network architecture\n .with_model_factory_default(hidden_sizes=(64, 64))\n # Stop condition\n .with_epoch_stop_callback(EpochStopCallbackRewardThreshold(195))\n .build()\n)\n\n# Run the experiment\nexperiment.run()\n```\n\n**What's happening here:**\n1. We create an {class}`~tianshou.highlevel.experiment.ExperimentBuilder` with three main configuration objects\n2. We chain builder methods to specify algorithm parameters, model architecture, and callbacks\n3. We call `.build()` to construct the experiment\n4. We call `.run()` to execute the entire training pipeline\n\nThe high-level API handles ...\n- creating and configuring environments\n- building the neural network\n- instantiating the policy and algorithm\n- setting up collectors and replay buffer\n- managing the training loop\n- watching the trained agent\n\n### Procedural API Example\n\n```python\nimport gymnasium as gym\nimport tianshou as ts\nfrom tianshou.algorithm.modelfree.dqn import DiscreteQLearningPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import CollectStats\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.space_info import SpaceInfo\nfrom torch.utils.tensorboard import SummaryWriter\n\n# Define hyperparameters\ntask = \"CartPole-v1\"\nlr, epoch, batch_size = 1e-3, 10, 64\nnum_training_envs, num_test_envs = 10, 100\ngamma, n_step, target_freq = 0.9, 3, 320\nbuffer_size = 20000\neps_train, eps_test = 0.1, 0.05\nepoch_num_steps, collection_step_num_env_steps = 10000, 10\n\n# Set up logging\nlogger = ts.utils.TensorboardLogger(SummaryWriter(\"log/dqn\"))\n\n# Create environments\ntraining_envs = ts.env.DummyVectorEnv([lambda: gym.make(task) for _ in range(num_training_envs)])\ntest_envs = ts.env.DummyVectorEnv([lambda: gym.make(task) for _ in range(num_test_envs)])\n\n# Build the network\nenv = gym.make(task, render_mode=\"human\")\nspace_info = SpaceInfo.from_env(env)\nstate_shape = space_info.observation_info.obs_shape\naction_shape = space_info.action_info.action_shape\nnet = Net(state_shape=state_shape, action_shape=action_shape, hidden_sizes=[128, 128, 128])\n\n# Create policy and algorithm\npolicy = DiscreteQLearningPolicy(\n model=net,\n action_space=env.action_space,\n eps_training=eps_train,\n eps_inference=eps_test,\n)\nalgorithm = ts.algorithm.DQN(\n policy=policy,\n optim=AdamOptimizerFactory(lr=lr),\n gamma=gamma,\n n_step_return_horizon=n_step,\n target_update_freq=target_freq,\n)\n\n# Set up collectors\ntraining_collector = ts.data.Collector[CollectStats](\n algorithm,\n training_envs,\n ts.data.VectorReplayBuffer(buffer_size, num_training_envs),\n exploration_noise=True,\n)\ntest_collector = ts.data.Collector[CollectStats](\n algorithm,\n test_envs,\n exploration_noise=True,\n)\n\n\n# Define stop condition\ndef stop_fn(mean_rewards: float) -> bool:\n if env.spec and env.spec.reward_threshold:\n return mean_rewards >= env.spec.reward_threshold\n return False\n\n\n# Train the algorithm\nresult = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n collection_step_num_env_steps=collection_step_num_env_steps,\n test_step_num_episodes=num_test_envs,\n batch_size=batch_size,\n update_step_num_gradient_steps_per_sample=1 / collection_step_num_env_steps,\n stop_fn=stop_fn,\n logger=logger,\n test_in_training=True,\n )\n)\nprint(f\"Finished training in {result.timing.total_time} seconds\")\n\n# Watch the trained agent\ncollector = ts.data.Collector[CollectStats](algorithm, env, exploration_noise=True)\ncollector.collect(n_episode=100, render=1 / 35)\n```\n\n**What's happening here:**\n1. We explicitly define all hyperparameters as variables\n2. We manually create the logger\n3. We construct training and test environments\n4. We build the neural network by extracting space information from the environment\n5. We create the policy and algorithm objects\n6. We set up collectors with a replay buffer\n7. We define callback functions\n8. We call `algorithm.run_training()` with explicit parameters\n9. We manually set up and run the evaluation collector\n\nThe procedural API requires ...\n- explicit creation of every component\n- manual extraction of environment properties\n- direct specification of all connections\n\n## Key Concepts in the High-Level API\n\n### ExperimentBuilder\n\nThe {class}`~tianshou.highlevel.experiment.ExperimentBuilder` is the core abstraction. \nEach algorithm has its own builder (e.g., {class}`~tianshou.highlevel.experiment.DQNExperimentBuilder`, {class}`~tianshou.highlevel.experiment.PPOExperimentBuilder`, {class}`~tianshou.highlevel.experiment.SACExperimentBuilder`).\n\n**Some methods you will find in experiment builders:**\n- `.with__params()` - Set algorithm-specific parameters\n- `.with_model_factory()`, `.with_model_factory_default()` - Configure network architecture\n- `.with_critic_factory()` - Configure critic network (for actor-critic methods)\n- `.with_epoch_train_callback()` - Add function to be called at the beginning of the training step in each epoch\n- `.with_epoch_test_callback()` - Add function to be called at the beginning of the test step in each epoch\n- `.with_epoch_stop_callback()` - Define stopping conditions\n- `.with_algorithm_wrapper_factory()` - Add algorithm wrappers (e.g., ICM)\n\n### Configuration Objects\n\nThree main configuration objects are required when constructing an experiment builder:\n\n1. **Environment Configuration** ({class}`~tianshou.highlevel.env.EnvFactory` subclasses)\n - Defines how to create and configure environments\n - Existing factories:\n - {class}`~tianshou.highlevel.env.EnvFactoryRegistered` - For the creation of environments registered in Gymnasium\n - {class}`~tianshou.highlevel.env.atari.atari_wrapper.AtariEnvFactory` - For Atari environments with preprocessing\n - Custom factories for your own environments can be created by subclassing {class}`~tianshou.highlevel.env.EnvFactory`\n\n2. **Experiment Configuration** ({class}`~tianshou.highlevel.experiment.ExperimentConfig`): \n General settings for the experiment, particularly related to \n - logging\n - randomization\n - persistence\n - watching the trained agent's performance after training\n\n3. **Training Configuration** ({class}`~tianshou.highlevel.config.OffPolicyTrainingConfig`, {class}`~tianshou.highlevel.config.OnPolicyTrainingConfig`): \n Defines all parameters related to the training process\n\n### Parameter Classes\n\nAlgorithm parameters are defined in dataclasses specific to each algorithm (e.g., {class}`~tianshou.highlevel.params.algorithm_params.DQNParams`, {class}`~tianshou.highlevel.params.algorithm_params.PPOParams`).\nThe parameters are extensively documented.\n\n```{note}\nMake sure to use a modern IDE to take advantage of auto-completion and inline documentation!\n```\n\n### Factories\n\nThe high-level API uses factories extensively:\n- **Model Factories**: Create neural networks (e.g., {class}`~tianshou.highlevel.module.intermediate.IntermediateModuleFactoryAtariDQN`)\n- **Environment Factories**: Create and configure environments\n- **Optimizer Factories**: Create optimizers with specific configurations\n\n### Extensibility\n\nThe high-level API is designed to be extensible. \nYou can create custom factories (e.g. for your own models or your own environments) by subclassing the appropriate base classes\nand then use them in the experiment builder.\n\nIf we have created a torch module in `CustomNetwork`, which we want to use within our policy, \nwe simply need to define a factory for it in order to apply it in the high-level API:\n\n```python\nfrom tianshou.highlevel.env import Environments\nfrom tianshou.highlevel.module.core import TDevice\nfrom tianshou.highlevel.module.intermediate import IntermediateModuleFactory, IntermediateModule\n\nclass CustomNetFactory(IntermediateModuleFactory):\n def __init__(self, hidden_sizes: tuple[int, ...] = (128, 128)):\n self.hidden_sizes = hidden_sizes\n \n def create_intermediate_module(self, envs: Environments, device: TDevice) -> IntermediateModule:\n obs_shape = envs.get_observation_shape()\n action_shape = envs.get_action_shape()\n \n # Your custom network creation logic\n net = CustomNetwork(\n obs_shape=obs_shape,\n action_shape=action_shape,\n hidden_sizes=self.hidden_sizes,\n ).to(device)\n \n return IntermediateModule(net, net.output_dim)\n\nexperiment = (\n DQNExperimentBuilder(...)\n .with_model_factory(CustomNetFactory(hidden_sizes=(256, 256)))\n .build()\n)\n```\n\n## Key Concepts in the Procedural API\n\n### Core Components\n\nYou manually create and connect ...\n\n1. **environments**: e.g. using `gym.make()` and vectorization ({class}`~tianshou.env.DummyVectorEnv`, {class}`~tianshou.env.SubprocVectorEnv`)\n2. **networks**: using {class}`~tianshou.utils.net.common.Net` or other PyTorch modules\n3. **policies**: using algorithm-specific policy classes (e.g., {class}`~tianshou.algorithm.modelfree.dqn.DiscreteQLearningPolicy`)\n4. **algorithms**: using algorithm classes (e.g., {class}`~tianshou.algorithm.modelfree.dqn.DQN`, {class}`~tianshou.algorithm.modelfree.ppo.PPO`, {class}`~tianshou.algorithm.modelfree.sac.SAC`)\n5. **collectors**: using {class}`~tianshou.data.Collector` to gather experience\n6. **buffers**: using {class}`~tianshou.data.buffer.VectorReplayBuffer` or {class}`~tianshou.data.buffer.ReplayBuffer`\n7. **trainers**: using the respective trainer class and corresponding parameter class (e.g., {class}`~tianshou.trainer.OffPolicyTrainer` and {class}`~tianshou.trainer.OffPolicyTrainerParams`)\n\n### Training Loop\n\nThe training is executed via `algorithm.run_training()`, which takes a trainer parameter object. \nYou can alternatively implement custom training loops (or even your own trainer class) for maximum flexibility.\n\n\n## Additional Resources\n\n- **high-Level API examples**: See `examples/` directory (scripts ending in `_hl.py`)\n- **procedural API examples**: See `examples/` directory (scripts without suffix)\n" }, { "path": "docs/01_user_guide/02_core_abstractions.md", "content": "# Core Abstractions\n\nTianshou's architecture is built around a number of key abstractions that work together to provide a modular and flexible reinforcement learning framework. \nThis document describes the conceptual foundation and functionality of each abstraction, helping you understand how they interact to enable RL agent training.\n\nKnowing these abstractions is primarily relevant when using the procedural API – and particularly when implementing one's own learning algoriithms.\n\n## Algorithm\n\nThe **{class}`~tianshou.algorithm.algorithm_base.Algorithm`** is the central abstraction representing the core of a reinforcement learning method (such as DQN, PPO, or SAC). \nIt implements the key steps within the {ref}`learning process `, containing a {ref}`policy` and defining how to update it from experience data.\n\nSince an Algorithm contains neural networks and manages their training, the class inherits from `torch.nn.Module`. \n\n### Core Responsibilities\n\nAn Algorithm implements the details of an {ref}`update step `:\n\n1. **preprocessing**: Before the actual update begins, the algorithm prepares the training data. \n This includes computing derived quantities that depend on temporal sequences, such as n-step returns, GAE advantages, or terminal state handling. \n The {meth}`~tianshou.algorithm.algorithm_base.Algorithm._preprocess_batch` method handles this phase, often leveraging static methods like \n {meth}`~tianshou.algorithm.algorithm_base.Algorithm.compute_nstep_return` and \n {meth}`~tianshou.algorithm.algorithm_base.Algorithm.compute_episodic_return` to \n efficiently compute returns using the buffer's temporal structure.\n\n2. **network update**: The algorithm performs the actual neural network updates based on its specific learning method. \n Each algorithm implements its own {meth}`~tianshou.algorithm.algorithm_base.Algorithm._update_with_batch` logic that defines how to update \n the policy networks using the preprocessed batch data.\n\n3. **postprocessing**: After the update, the algorithm may perform cleanup operations, such as updating prioritized replay buffer weights or other \n algorithm-specific bookkeeping.\n\n### Learning Orchestration\n\nThe Algorithm orchestrates the {ref}`update step ` through its \n{meth}`~tianshou.algorithm.algorithm_base.Algorithm.update` method, which ensures these three phases execute in proper sequence. \nIt also manages optimizer state and learning rate schedulers, making them available for state persistence through\n{meth}`~tianshou.algorithm.algorithm_base.Algorithm.state_dict` and\n{meth}`~tianshou.algorithm.algorithm_base.Algorithm.load_state_dict` methods.\n\nEach algorithm type (on-policy, off-policy, offline) creates its appropriate trainer through the\n{meth}`~tianshou.algorithm.algorithm_base.Algorithm.create_trainer` method, \nestablishing the connection between the learning logic and the training loop.\n\n(policy)=\n## Policy\n\nThe **{class}`~tianshou.algorithm.algorithm_base.Policy`** represents the agent's decision-making component, i.e. the mapping from observations to actions. \nWhile the Algorithm defines how to learn, the Policy defines what is learned and how to act.\n\nLike Algorithm, the class inherits from `torch.nn.Module`.\n\n### States of Operation\n\nA Policy operates in two main modes:\n\n- **training mode**: During training, the policy may employ exploration strategies, sample from action distributions, or add noise to encourage discovery. \n Training mode is further divided into:\n - *collecting state*: When gathering experience from environment interaction\n - *updating state*: When performing network updates during learning\n \n- **testing/inference mode**: During evaluation, the policy typically acts deterministically or uses the mode of predicted distributions to showcase \n learned behavior without exploration.\n\nThe flag `is_within_training_step` controls the collection strategy, distinguishing between training and inference behavior.\n\n### Key Methods\n\nThe Policy provides several essential methods:\n\n- **{meth}`~tianshou.algorithm.algorithm_base.Policy.forward`**: \n The core computation method that processes batched observations to produce action distributions or Q-values. \n It takes a batch of environment data and optional hidden state (for recurrent policies), \n returning a batch containing at minimum the \"act\" key, \n and potentially \"state\" (hidden state) and \"policy\" (intermediate results to be stored in the buffer).\n\n- **{meth}`~tianshou.algorithm.algorithm_base.Policy.compute_action`**: \n A convenient method for inference that takes a single observation and returns a concrete action suitable for the environment. \n This method internally calls `forward` with proper batching and unbatching.\n\n- **{meth}`~tianshou.algorithm.algorithm_base.Policy.map_action`**: Transforms the raw neural network output to the environment's action space format, handling any necessary scaling or discretization.\n\nThe separation between `forward` (which works with batches) and \n{meth}`~tianshou.algorithm.algorithm_base.Policy.compute_action` (which works with single observations) provides efficiency \nduring training and convenience during inference.\n\n## Collector\n\nThe class **{class}`~tianshou.data.Collector`** bridges the gap between the policy and the environment(s), \nmanaging the process of gathering experience data. \nIt enables efficient interaction with both single environments and vectorized environments (multiple parallel environments).\n\n### Data Collection\n\nThe Collector's primary method, {meth}`~tianshou.data.Collector.collect`, orchestrates the environment interaction loop. It can collect either:\n- a specified number of steps (`n_step`): useful for maintaining consistent training batch sizes\n- a specified number of episodes (`n_episode`): useful for evaluation or when episode-level statistics are important\n\nDuring collection, the Collector ...\n1. obtains observations from the environment(s),\n2. calls the policy to compute actions,\n3. steps the environment(s) with these actions,\n4. stores the resulting transitions (observation, action, reward, next observation, termination flags, and info) in the replay buffer,\n5. manages episode boundaries and reset logic,\n6. collects statistics such as episode returns, lengths, and collection speed.\n\n### Hooks and Extensibility\n\nThe Collector supports customization through hooks that can be triggered at different points in the collection process:\n- **step hooks**: called after each environment step\n- **episode done hooks**: called when episodes complete\n\nThese hooks enable custom logging, curriculum learning, or other dynamic behaviors during data collection.\n\n### Vectorized Environments\n\nThe Collector seamlessly handles vectorized environments, where multiple environment instances run in parallel. \nThis significantly speeds up data collection while maintaining correct episode boundaries and statistics for each environment instance.\n\n## Trainer\n\nThe **{class}`~tianshou.trainer.Trainer`** orchestrates the complete training loop, coordinating data collection, policy updates, and evaluation. \nIt provides the high-level control flow that brings all components together.\n\n### Trainer Types\n\nTianshou provides three main trainer types, each suited to different algorithm families:\n\n- **{class}`~tianshou.trainer.OnPolicyTrainer`**: for algorithms that must learn from freshly collected data (e.g., PPO, A2C). \n After each collection phase, the buffer is used for updates and thereafter is cleared.\n\n- **{class}`~tianshou.trainer.OffPolicyTrainer`**: for algorithms that can learn from any past experience (e.g., DQN, SAC, DDPG).\n Data accumulates in the replay buffer over time, and updates sample from this growing pool of experience.\n\n- **{class}`~tianshou.trainer.OfflineTrainer`**: for algorithms that learn exclusively from a fixed dataset without any environment interaction (e.g., BCQ, CQL).\n\n### Training Loop Structure\n\nThe training process is organized into epochs, where each epoch consists of:\n\n1. **data collection**: The trainer uses the train collector to gather experience according to its algorithm type's needs\n2. **policy update**: The algorithm performs one or more update steps using the collected data\n3. **evaluation**: Periodically, the trainer uses the test collector to evaluate the current policy's performance\n4. **logging**: Statistics from collection, updates, and evaluation are logged\n5. **checkpointing**: The best policy (according to a scoring function) is saved\n\nThe trainer handles the detailed choreography of these steps, including determining when to collect more data, \nhow many update steps to perform, when to evaluate, and when to stop training (based on maximum epochs, timesteps, or early stopping criteria).\n\n### Configuration\n\nTrainers are configured through parameter dataclasses \n({class}`~tianshou.trainer.OnPolicyTrainerParams`, {class}`~tianshou.trainer.OffPolicyTrainerParams`, {class}`~tianshou.trainer.OfflineTrainerParams`) \nthat specify in particular:\n- training duration (number of epochs, steps per epoch)\n- collectors for training and testing\n- update frequency and batch size\n- evaluation frequency\n- logging and checkpointing settings\n- early stopping criteria\n\n## Batch\n\nThe class **{class}`~tianshou.data.Batch`** is Tianshou's flexible data structure for passing information between components. \nIt serves as the lingua franca of the framework, carrying everything from raw environment observations to computed returns and policy outputs.\n\n### Design Philosophy\n\nBatch is designed to be ...\n- **flexible**: can contain any key-value pairs, with nested structures supported,\n- **numpy/torch-compatible**: automatically converts lists to arrays and seamlessly works with both NumPy arrays and PyTorch tensors,\n- **sliceable**: supports indexing and slicing operations that work across all contained data,\n- **composable**: can be concatenated, stacked, and split to support batching operations.\n\n### Type Safety with BatchProtocol\n\nWhile `Batch` provides a flexible, dictionary-like structure for holding arbitrary data, this flexibility can make it challenging to statically type-check which attributes are present in a batch at any given point in the code. To address this, Tianshou uses **{class}`~tianshou.data.batch.BatchProtocol`** and derived protocols to specify the expected attributes while keeping the actual runtime type as `Batch`.\n\nBatchProtocol is a Python `Protocol` (from `typing.Protocol`) that defines the interface of a Batch object, specifying which operations and attributes should be available. More importantly, Tianshou provides a rich set of derived protocols in {mod}`tianshou.data.types` that describe batches with specific sets of attributes commonly used throughout the framework:\n\n- **{class}`~tianshou.data.types.ObsBatchProtocol`**: Contains `obs` and `info` - the minimal batch for policy forward passes\n- **{class}`~tianshou.data.types.RolloutBatchProtocol`**: Adds `obs_next`, `act`, `rew`, `terminated`, and `truncated` - typical data from replay buffer sampling\n- **{class}`~tianshou.data.types.BatchWithReturnsProtocol`**: Extends RolloutBatchProtocol with `returns` computed from rewards\n- **{class}`~tianshou.data.types.BatchWithAdvantagesProtocol`**: Includes `adv` (advantages) and `v_s` (value estimates) for policy gradient methods\n- **{class}`~tianshou.data.types.ActStateBatchProtocol`**: Contains `act` and `state` for policy outputs, especially with RNN support\n- **{class}`~tianshou.data.types.ModelOutputBatchProtocol`**: Adds `logits` to action and state information\n- **{class}`~tianshou.data.types.DistBatchProtocol`**: Contains action distributions (`dist`) for stochastic policies\n- **{class}`~tianshou.data.types.PrioBatchProtocol`**: Includes `weight` for prioritized experience replay\n\nThese protocols serve as type hints in function signatures throughout Tianshou, making it explicit what attributes are expected and available. For example, a policy's `forward` method might accept an `ObsBatchProtocol` and return an `ActStateBatchProtocol`, clearly documenting the data contract. Despite these type annotations, the actual objects remain flexible `Batch` instances at runtime, preserving Tianshou's dynamic nature while improving code clarity and IDE support.\n\n### Common Use Cases\n\nBatches flow through the system carrying different types of information:\n\n1. **environment data**: observations, rewards, done flags, and info from environment steps\n2. **policy outputs**: actions, hidden states, and intermediate computations\n3. **training data**: returns, advantages, and other computed quantities needed for learning\n4. **sampling results**: batches sampled from the replay buffer for training\n\n### Operations\n\nKey operations on batches include:\n- **attribute access**: dot notation (`batch.obs`) or dictionary-style access (`batch['obs']`)\n- **slicing**: extract subsets with standard indexing (`batch[0:10]`, `batch[[1,3,5]]`)\n- **stacking**: combine multiple batches along a new dimension\n- **type conversion**: convert between NumPy and PyTorch with `to_numpy()` and `to_torch()`\n- **null handling**: detect and remove null values with `hasnull()`, `isnull()`, and `dropnull()`\n\nThe first dimension of all data in a Batch represents the batch size, enabling vectorized operations.\n\n## Buffer\n\nA **buffer** (i.e. class {class}`~tianshou.data.buffer.ReplayBuffer` and its variants) manages the storage and retrieval of experience data. \nIt acts as the memory of the learning system, preserving the temporal structure of episodes while providing efficient access patterns.\n\n### Storage Structure\n\nBuffers store data in a circular queue fashion with a fixed maximum size. When the buffer fills, new data overwrites the oldest stored experiences. \nAll data is stored within a single underlying Batch object, with the buffer managing:\n- **pointer tracking**: current insertion position\n- **episode boundaries**: which transitions belong to which episodes\n- **temporal relationships**: the sequential order of transitions\n\n### Reserved Keys\n\nBuffers use a standard set of keys for storing transitions:\n- `obs`: Observation at time t\n- `act`: Action taken at time t\n- `rew`: Reward received at time t\n- `terminated`: True if the episode ended naturally at time t\n- `truncated`: True if the episode was cut off at time t (e.g., time limit)\n- `done`: Automatically inferred as `terminated or truncated`\n- `obs_next`: Observation at time t+1\n- `info`: Additional information from the environment\n- `policy`: Intermediate policy computations to be stored\n\n### Core Operations\n\n**adding data**: The {meth}`~tianshou.data.buffer.buffer_base.ReplayBuffer.add` method stores new transitions, \nautomatically handling episode boundaries and computing episode statistics (return, length) \nwhen episodes complete.\n\n**sampling**: The {meth}`~tianshou.data.buffer.buffer_base.ReplayBuffer.sample` method retrieves batches of experiences for training, \nreturning both the sampled batch and the corresponding indices. \nThe sample size can be specified, or set to 0 to retrieve all available data.\n\n**temporal navigation**: The {meth}`~tianshou.data.buffer.buffer_base.ReplayBuffer.prev` and {meth}`~tianshou.data.buffer.ReplayBuffer.next` \nmethods enable traversal along the temporal sequence, respecting episode boundaries. \nThis is essential for computing n-step returns and other time-dependent quantities.\n\n**persistence**: Buffers support saving and loading via pickle or HDF5 format, enabling dataset collection and offline learning.\n\n### Buffer Variants\n\nTianshou provides specialized buffer types:\n\n- **{class}`~tianshou.data.buffer.buffer_base.ReplayBuffer`**: the standard buffer for single environments\n- **{class}`~tianshou.data.buffer.vecbuf.VectorReplayBuffer`**: manages separate sub-buffers for multiple parallel environments while maintaining chronological order\n- **{class}`~tianshou.data.buffer.prio.PrioritizedReplayBuffer`**: samples transitions based on their TD-error or other priority metrics, using an efficient segment tree implementation\n\n### Advanced Features\n\nBuffers support sophisticated use cases:\n- **frame stacking**: automatically stacks consecutive observations (useful for RNN inputs or Atari)\n- **memory optimization**: option to skip storing next observations (useful for Atari where they can be inferred)\n- **multi-modal observations**: handle observations with multiple components (e.g., image + vector)\n\n## Logger\n\nThe **{class}`~tianshou.utils.logger.logger_base.BaseLogger`** abstraction provides a unified interface for recording and tracking training progress, metrics, and statistics. \nIt decouples the training loop from the specifics of where and how data is logged.\n\n### Purpose\n\nLoggers serve several essential functions:\n- **progress tracking**: record timesteps, episodes, and epochs as training progresses\n- **metric collection**: store performance indicators like rewards, losses, and success rates\n- **experiment organization**: manage different data scopes (training, testing, updating)\n- **reproducibility**: save training curves and hyperparameters for later analysis\n\n### Logging Scopes\n\nThe framework organizes logged data into distinct scopes:\n- **train data**: metrics from the training collector (episode returns, steps, collection speed)\n- **test data**: evaluation metrics from the test collector\n- **update data**: learning statistics from the algorithm (losses, gradients, learning rates)\n- **info data**: additional custom metrics or metadata\n\nEach scope has a corresponding log method (`log_train_data`, `log_test_data`, `log_update_data`, `log_info_data`) that the trainer calls at appropriate times.\n\n### Implementations\n\nTianshou provides several logger implementations:\n- **{class}`~tianshou.utils.logger.tensorboard.TensorboardLogger`**: writes to TensorBoard format for visualization with TensorBoard\n- **{class}`~tianshou.utils.logger.wandb.WandbLogger`**: integrates with Weights & Biases for cloud-based experiment tracking\n\nAll implementations inherit from {class}`~tianshou.utils.logger.logger_base.BaseLogger` and share a common interface, \nmaking it easy to switch between logging backends or use multiple loggers simultaneously.\n\n### Data Preparation\n\nBefore writing, loggers prepare data through the `prepare_dict_for_logging` method, which can filter, transform, or aggregate metrics. \nThe `write` method then persists the prepared data to the logging backend with an associated step count.\n\n## How They Work Together\n\nThese seven abstractions collaborate to enable reinforcement learning:\n\n1. The **Trainer** initializes and orchestrates the training process.\n2. The **Collector** uses the **Policy** to gather experience from environments.\n3. Collected transitions are stored in the **Buffer** extracted as **Batches**.\n4. The **Algorithm** samples from the **Buffer**, preprocesses the data, and updates the **Policy**.\n5. The **Logger** records metrics throughout the process.\n6. The cycle repeats until training completes.\n\nThis modular design allows each component to focus on its specific responsibility while maintaining clean interfaces. \nYou can customize individual components (e.g., implementing a new Algorithm or Buffer) without affecting the others, \nmaking Tianshou both powerful and flexible.\n" }, { "path": "docs/01_user_guide/index.rst", "content": "User Guide\n==========\n\nThe user guide provides an introduction to core concepts, establishes the glossary of terms,\nintroduces Tianshou's dual API architecture and provides an overview of important abstractions." }, { "path": "docs/02_deep_dives/0_intro.md", "content": "# Deep Dives\n\nOur deep dives are a collection of executable tutorials on some of the internal representations used by Tianshou.\nProvided as notebooks, you can run them directly in Colab or download them to run them locally.\n" }, { "path": "docs/02_deep_dives/L1_Batch.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Batch: Tianshou's Core Data Structure\\n\",\n \"\\n\",\n \"The `Batch` class is Tianshou's fundamental data structure for efficiently storing and manipulating heterogeneous data in reinforcement learning. This tutorial provides comprehensive guidance on understanding its conceptual foundations, operational behavior, and best practices.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import pickle\\n\",\n \"from typing import cast\\n\",\n \"\\n\",\n \"import numpy as np\\n\",\n \"import torch\\n\",\n \"from torch.distributions import Categorical, Normal\\n\",\n \"\\n\",\n \"from tianshou.data import Batch\\n\",\n \"from tianshou.data.types import ActBatchProtocol, ObsBatchProtocol, RolloutBatchProtocol\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## 1. Introduction: Why Batch?\\n\",\n \"\\n\",\n \"### The Challenge in Reinforcement Learning\\n\",\n \"\\n\",\n \"Reinforcement learning algorithms face a fundamental data management challenge:\\n\",\n \"\\n\",\n \"1. **Diverse Data Requirements**: Different RL algorithms need different data fields:\\n\",\n \" - Basic algorithms: `state`, `action`, `reward`, `done`, `next_state`\\n\",\n \" - Actor-Critic: additionally `advantages`, `returns`, `values`\\n\",\n \" - Policy Gradient: additionally `log_probs`, `old_log_probs`\\n\",\n \" - Off-policy: additionally `priority_weights`\\n\",\n \"\\n\",\n \"2. **Heterogeneous Observation Spaces**: Environments return diverse observation types:\\n\",\n \" - Simple: vectors (`np.array([1.0, 2.0, 3.0])`)\\n\",\n \" - Complex: images (`np.array(shape=(84, 84, 3))`)\\n\",\n \" - Hybrid: dictionaries combining multiple modalities\\n\",\n \" ```python\\n\",\n \" obs = {\\n\",\n \" 'camera': np.array(shape=(64, 64, 3)),\\n\",\n \" 'velocity': np.array([1.2, 0.5]),\\n\",\n \" 'inventory': np.array([5, 2, 0])\\n\",\n \" }\\n\",\n \" ```\\n\",\n \"\\n\",\n \"3. **Data Flow Across Components**: Data must flow seamlessly through:\\n\",\n \" - Collectors (gathering experience from environments)\\n\",\n \" - Replay Buffers (storing and sampling transitions)\\n\",\n \" - Policies and Algorithms (learning and inference)\\n\",\n \"\\n\",\n \"### Why Not Alternatives?\\n\",\n \"\\n\",\n \"#### Plain Dictionaries\\n\",\n \"Dictionaries lack essential features\\n\",\n \"```python\\n\",\n \"data = {'obs': np.array([1, 2]), 'reward': np.array([1.0, 2.0])}\\n\",\n \"```\\n\",\n \"\\n\",\n \"They would work in principle but has no shape/length semantics, no indexing, and no type safety.\\n\",\n \"\\n\",\n \"#### TensorDict\\n\",\n \"While `TensorDict` (used in `pytorch-rl`) is a powerful alternative:\\n\",\n \"- **Batch supports arbitrary objects**, not just tensors (useful for object-dtype arrays, custom types)\\n\",\n \"- **Batch has better type checking** via `BatchProtocol` (enables IDE autocompletion)\\n\",\n \"- **Batch preceded TensorDict** and provides a stable foundation for Tianshou\\n\",\n \"- **TensorDict isn't part of core PyTorch** (external dependency)\\n\",\n \"\\n\",\n \"### What is Batch?\\n\",\n \"\\n\",\n \"**Batch = Dictionary + Array hybrid with RL-specific features**\\n\",\n \"\\n\",\n \"Key capabilities:\\n\",\n \"- **Dict-like**: Key-value storage with attribute access (`batch.obs`, `batch.reward`)\\n\",\n \"- **Array-like**: Shape, indexing, slicing (`batch[0]`, `batch[:10]`, `batch.shape`)\\n\",\n \"- **Hierarchical**: Nested structures for complex data\\n\",\n \"- **Type-safe**: Protocol-based typing for IDE support\\n\",\n \"- **RL-aware**: Special handling for distributions, missing values, heterogeneous aggregation\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## 2. Core Concepts\\n\",\n \"\\n\",\n \"### Hierarchical Named Tensors\\n\",\n \"\\n\",\n \"Batch stores **hierarchical named tensors** - collections of tensors whose identifiers form a structured hierarchy. Consider tensors `[t1, t2, t3, t4]` with names `[name1, name2, name3, name4]`, where `name1` and `name2` are under namespace `name0`. The fully qualified name of `t1` is `name0.name1`.\\n\",\n \"\\n\",\n \"### Tree Structure Visualization\\n\",\n \"\\n\",\n \"The structure can be visualized as a tree with:\\n\",\n \"- **Root**: The Batch object itself\\n\",\n \"- **Internal nodes**: Keys (names)\\n\",\n \"- **Leaf nodes**: Values (scalars, arrays, tensors)\\n\",\n \"\\n\",\n \"```mermaid\\n\",\n \"graph TD\\n\",\n \" root[\\\"Batch (root)\\\"]\\n\",\n \" root --> obs[\\\"obs\\\"]\\n\",\n \" root --> act[\\\"act\\\"]\\n\",\n \" root --> rew[\\\"rew\\\"]\\n\",\n \" obs --> camera[\\\"camera\\\"]\\n\",\n \" obs --> sensory[\\\"sensory\\\"]\\n\",\n \" camera --> cam_data[\\\"np.array(3,3)\\\"]\\n\",\n \" sensory --> sens_data[\\\"np.array(5,)\\\"]\\n\",\n \" act --> act_data[\\\"np.array(2,)\\\"]\\n\",\n \" rew --> rew_data[\\\"3.66\\\"]\\n\",\n \" \\n\",\n \" style root fill:#e1f5ff\\n\",\n \" style obs fill:#fff4e1\\n\",\n \" style act fill:#fff4e1\\n\",\n \" style rew fill:#fff4e1\\n\",\n \" style camera fill:#ffe1f5\\n\",\n \" style sensory fill:#ffe1f5\\n\",\n \" style cam_data fill:#e8f5e1\\n\",\n \" style sens_data fill:#e8f5e1\\n\",\n \" style act_data fill:#e8f5e1\\n\",\n \" style rew_data fill:#e8f5e1\\n\",\n \"```\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Example: hierarchical structure\\n\",\n \"data = {\\n\",\n \" \\\"action\\\": np.array([1.0, 2.0, 3.0]),\\n\",\n \" \\\"reward\\\": 3.66,\\n\",\n \" \\\"obs\\\": {\\n\",\n \" \\\"camera\\\": np.zeros((3, 3)),\\n\",\n \" \\\"sensory\\\": np.ones(5),\\n\",\n \" },\\n\",\n \"}\\n\",\n \"\\n\",\n \"batch = Batch(data)\\n\",\n \"print(batch)\\n\",\n \"print(\\\"\\\\nAccessing nested values:\\\")\\n\",\n \"print(f\\\"batch.obs.camera.shape = {batch.obs.camera.shape}\\\")\\n\",\n \"print(f\\\"batch.obs.sensory = {batch.obs.sensory}\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Data Flow in RL Pipeline\\n\",\n \"\\n\",\n \"Batch facilitates data flow throughout the RL pipeline:\\n\",\n \"\\n\",\n \"```mermaid\\n\",\n \"graph LR\\n\",\n \" A[Collector] -->|ActBatchProtocol| B[Environment]\\n\",\n \" B[Environment + Action] -->|RolloutBatchProtocol| C[Replay Buffer]\\n\",\n \" C -->|RolloutBatchProtocol| D[Policy]\\n\",\n \" D -->|ActBatchProtocol| A\\n\",\n \" D -->|BatchWithAdvantages| E[Algorithm/Trainer]\\n\",\n \" E --> D\\n\",\n \" \\n\",\n \" style A fill:#e1f5ff\\n\",\n \" style B fill:#fff4e1\\n\",\n \" style C fill:#ffe1f5\\n\",\n \" style D fill:#e8f5e1\\n\",\n \" style E fill:#f5e1e1\\n\",\n \"```\\n\",\n \"\\n\",\n \"Each arrow represents a specific `BatchProtocol` that defines what fields are expected at that stage.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## 3. Basic Operations\\n\",\n \"\\n\",\n \"### 3.1 Construction\\n\",\n \"\\n\",\n \"Batch objects can be constructed in several ways:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# From keyword arguments\\n\",\n \"batch1 = Batch(a=4, b=[5, 5], c=\\\"hello\\\")\\n\",\n \"print(\\\"From kwargs:\\\", batch1)\\n\",\n \"\\n\",\n \"# From dictionary\\n\",\n \"batch2 = Batch({\\\"a\\\": 4, \\\"b\\\": [5, 5], \\\"c\\\": \\\"hello\\\"})\\n\",\n \"print(\\\"\\\\nFrom dict:\\\", batch2)\\n\",\n \"\\n\",\n \"# From list of dictionaries (automatically stacked)\\n\",\n \"batch3 = Batch([{\\\"a\\\": 1, \\\"b\\\": 2}, {\\\"a\\\": 3, \\\"b\\\": 4}])\\n\",\n \"print(\\\"\\\\nFrom list of dicts:\\\", batch3)\\n\",\n \"\\n\",\n \"# Nested batch\\n\",\n \"batch4 = Batch(obs=Batch(x=1, y=2), act=5)\\n\",\n \"print(\\\"\\\\nNested:\\\", batch4)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 3.2 Content Rules\\n\",\n \"\\n\",\n \"Understanding what Batch can store and how it converts data:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Keys must be strings\\n\",\n \"batch = Batch()\\n\",\n \"batch.key1 = \\\"value\\\"\\n\",\n \"batch.key2 = np.array([1, 2, 3])\\n\",\n \"print(\\\"Keys:\\\", list(batch.keys()))\\n\",\n \"\\n\",\n \"# Automatic conversions\\n\",\n \"demo = Batch(\\n\",\n \" scalar_int=5, # → np.array(5)\\n\",\n \" scalar_float=3.14, # → np.array(3.14)\\n\",\n \" list_nums=[1, 2, 3], # → np.array([1, 2, 3])\\n\",\n \" list_mixed=[1, \\\"hello\\\", None], # → np.array([1, \\\"hello\\\", None], dtype=object)\\n\",\n \" dict_val={\\\"x\\\": 1, \\\"y\\\": 2}, # → Batch(x=1, y=2)\\n\",\n \")\\n\",\n \"\\n\",\n \"print(\\\"\\\\nAutomatic conversions:\\\")\\n\",\n \"print(f\\\"scalar_int type: {type(demo.scalar_int)}, value: {demo.scalar_int}\\\")\\n\",\n \"print(f\\\"list_nums type: {type(demo.list_nums)}, dtype: {demo.list_nums.dtype}\\\")\\n\",\n \"print(f\\\"list_mixed dtype: {demo.list_mixed.dtype}\\\")\\n\",\n \"print(f\\\"dict_val type: {type(demo.dict_val)}\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"**Important conversions:**\\n\",\n \"- Lists of numbers → NumPy arrays\\n\",\n \"- Lists with mixed types → Object-dtype arrays\\n\",\n \"- Dictionaries → Batch objects (recursively)\\n\",\n \"- Scalars → NumPy scalars\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 3.3 Access Patterns\\n\",\n \"\\n\",\n \"**Important: Understanding Iteration**\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"batch = Batch(a=[1, 2, 3], b=[4, 5, 6])\\n\",\n \"\\n\",\n \"# Attribute vs dictionary access (equivalent)\\n\",\n \"print(\\\"Attribute access:\\\", batch.a)\\n\",\n \"print(\\\"Dict access:\\\", batch[\\\"a\\\"])\\n\",\n \"\\n\",\n \"# Getting keys\\n\",\n \"print(\\\"\\\\nKeys:\\\", list(batch.keys()))\\n\",\n \"\\n\",\n \"# Gotcha: Iteration is array like, not over keys\\n\",\n \"print(\\\"\\\\nIteration behavior:\\\")\\n\",\n \"print(\\\"for x in batch iterates over batch[0], batch[1], ..., NOT keys!\\\")\\n\",\n \"for i, item in enumerate(batch):\\n\",\n \" print(f\\\"batch[{i}] = {item}\\\")\\n\",\n \"\\n\",\n \"# This is different from dict behavior!\\n\",\n \"regular_dict = {\\\"a\\\": [1, 2, 3], \\\"b\\\": [4, 5, 6]}\\n\",\n \"print(\\\"\\\\nCompare with dict iteration (iterates over keys):\\\")\\n\",\n \"for key in regular_dict:\\n\",\n \" print(f\\\"key = {key}\\\")\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": \"\"\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 3.4 Indexing & Slicing\\n\",\n \"\\n\",\n \"Batch supports NumPy-like indexing and slicing:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"batch = Batch(a=np.array([[0.0, 2.0], [1.0, 3.0]]), b=[[5.0, -5.0], [1.0, -2.0]])\\n\",\n \"\\n\",\n \"print(\\\"Original batch shape:\\\", batch.shape)\\n\",\n \"print(\\\"Original batch length:\\\", len(batch))\\n\",\n \"\\n\",\n \"# Single index\\n\",\n \"print(\\\"\\\\nbatch[0]:\\\")\\n\",\n \"print(batch[0])\\n\",\n \"\\n\",\n \"# Slicing\\n\",\n \"print(\\\"\\\\nbatch[:1]:\\\")\\n\",\n \"print(batch[:1])\\n\",\n \"\\n\",\n \"# Advanced indexing\\n\",\n \"print(\\\"\\\\nbatch[[0, 1]]:\\\")\\n\",\n \"print(batch[[0, 1]])\\n\",\n \"\\n\",\n \"# Multi-dimensional indexing\\n\",\n \"print(\\\"\\\\nbatch[:, 0] (first column of all arrays):\\\")\\n\",\n \"print(batch[:, 0])\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Broadcasting and in-place operations\\n\",\n \"batch[:, 1] += 10\\n\",\n \"print(\\\"After batch[:, 1] += 10:\\\")\\n\",\n \"print(batch)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 3.5 Stack, Concatenate, and Split\\n\",\n \"\\n\",\n \"Combining and splitting batches:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": \"# Stack: adds a new dimension\\nbatch1 = Batch(a=np.array([1, 2]), b=np.array([5, 6]))\\nbatch2 = Batch(a=np.array([3, 4]), b=np.array([7, 8]))\\n\\nstacked = Batch.stack([batch1, batch2])\\nprint(\\\"Stacked:\\\")\\nprint(stacked)\\nprint(f\\\"Shape: {stacked.shape}\\\")\\n\\n# Concatenate: extends along existing dimension\\nconcatenated = Batch.cat([batch1, batch2])\\nprint(\\\"\\\\nConcatenated:\\\")\\nprint(concatenated)\\nprint(f\\\"Shape: {concatenated.shape}\\\")\"\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Split\\n\",\n \"batch = Batch(a=np.arange(10), b=np.arange(10, 20))\\n\",\n \"splits = list(batch.split(size=3, shuffle=False))\\n\",\n \"print(f\\\"Split into {len(splits)} batches:\\\")\\n\",\n \"for i, split in enumerate(splits):\\n\",\n \" print(f\\\"Split {i}: a={split.a}, length={len(split)}\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 3.6 Data Type Conversion\\n\",\n \"\\n\",\n \"Converting between NumPy and PyTorch:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Create batch with NumPy arrays\\n\",\n \"batch = Batch(a=np.zeros((3, 4)), b=np.ones(5))\\n\",\n \"print(\\\"Original (NumPy):\\\")\\n\",\n \"print(f\\\"batch.a type: {type(batch.a)}\\\")\\n\",\n \"\\n\",\n \"# Convert to PyTorch (in-place)\\n\",\n \"batch.to_torch_(dtype=torch.float32, device=\\\"cpu\\\")\\n\",\n \"print(\\\"\\\\nAfter to_torch_():\\\")\\n\",\n \"print(f\\\"batch.a type: {type(batch.a)}\\\")\\n\",\n \"print(f\\\"batch.a dtype: {batch.a.dtype}\\\")\\n\",\n \"\\n\",\n \"# Convert back to NumPy (in-place)\\n\",\n \"batch.to_numpy_()\\n\",\n \"print(\\\"\\\\nAfter to_numpy_():\\\")\\n\",\n \"print(f\\\"batch.a type: {type(batch.a)}\\\")\\n\",\n \"\\n\",\n \"# Non-in-place versions return a new batch\\n\",\n \"batch_torch = batch.to_torch()\\n\",\n \"print(\\\"\\\\nOriginal batch unchanged:\\\", type(batch.a))\\n\",\n \"print(\\\"New batch:\\\", type(batch_torch.a))\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## 4. Type Safety with Protocols\\n\",\n \"\\n\",\n \"### Why Protocols?\\n\",\n \"\\n\",\n \"Batch needs to be **flexible** (not fixed fields like dataclasses) but we still want **type safety** and **IDE autocompletion**. Protocols provide the best of both worlds:\\n\",\n \"\\n\",\n \"- **Runtime flexibility**: Add any fields dynamically\\n\",\n \"- **Static type checking**: Type checkers (mypy, pyright) verify correct usage\\n\",\n \"- **IDE support**: Autocompletion for expected fields\\n\",\n \"\\n\",\n \"### What is BatchProtocol?\\n\",\n \"\\n\",\n \"A `Protocol` defines an interface without implementation. Think of it as a contract: \\\"any object with these fields is valid.\\\"\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Creating a typed batch using cast\\n\",\n \"# This enables IDE autocompletion and type checking\\n\",\n \"\\n\",\n \"# ActBatchProtocol: just needs 'act' field\\n\",\n \"act_batch = cast(ActBatchProtocol, Batch(act=np.array([1, 2, 3])))\\n\",\n \"print(\\\"ActBatchProtocol:\\\", act_batch.act)\\n\",\n \"\\n\",\n \"# ObsBatchProtocol: needs 'obs' and 'info' fields\\n\",\n \"obs_batch = cast(\\n\",\n \" ObsBatchProtocol,\\n\",\n \" Batch(obs=np.array([[1.0, 2.0], [3.0, 4.0]]), info=np.array([{}, {}], dtype=object)),\\n\",\n \")\\n\",\n \"print(\\\"\\\\nObsBatchProtocol:\\\", obs_batch.obs)\\n\",\n \"\\n\",\n \"# RolloutBatchProtocol: needs obs, obs_next, act, rew, terminated, truncated\\n\",\n \"rollout_batch = cast(\\n\",\n \" RolloutBatchProtocol,\\n\",\n \" Batch(\\n\",\n \" obs=np.array([[1.0, 2.0], [3.0, 4.0]]),\\n\",\n \" obs_next=np.array([[2.0, 3.0], [4.0, 5.0]]),\\n\",\n \" act=np.array([0, 1]),\\n\",\n \" rew=np.array([1.0, 2.0]),\\n\",\n \" terminated=np.array([False, True]),\\n\",\n \" truncated=np.array([False, False]),\\n\",\n \" info=np.array([{}, {}], dtype=object),\\n\",\n \" ),\\n\",\n \")\\n\",\n \"print(\\\"\\\\nRolloutBatchProtocol reward:\\\", rollout_batch.rew)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Protocol Hierarchy\\n\",\n \"\\n\",\n \"Tianshou defines a hierarchy of protocols for different use cases:\\n\",\n \"\\n\",\n \"```mermaid\\n\",\n \"graph TD\\n\",\n \" BP[BatchProtocol
Base protocol] --> OBP[ObsBatchProtocol
obs, info]\\n\",\n \" BP --> ABP[ActBatchProtocol
act]\\n\",\n \" ABP --> ASBP[ActStateBatchProtocol
act, state]\\n\",\n \" OBP --> RBP[RolloutBatchProtocol
+obs_next, act, rew,
terminated, truncated]\\n\",\n \" RBP --> BWRP[BatchWithReturnsProtocol
+returns]\\n\",\n \" BWRP --> BWAP[BatchWithAdvantagesProtocol
+adv, v_s]\\n\",\n \" ASBP --> MOBP[ModelOutputBatchProtocol
+logits]\\n\",\n \" MOBP --> DBP[DistBatchProtocol
+dist]\\n\",\n \" DBP --> DLPBP[DistLogProbBatchProtocol
+log_prob]\\n\",\n \" BWAP --> LOPBP[LogpOldProtocol
+logp_old]\\n\",\n \" \\n\",\n \" style BP fill:#e1f5ff\\n\",\n \" style OBP fill:#fff4e1\\n\",\n \" style ABP fill:#fff4e1\\n\",\n \" style RBP fill:#ffe1f5\\n\",\n \" style BWRP fill:#e8f5e1\\n\",\n \" style BWAP fill:#e8f5e1\\n\",\n \" style DBP fill:#f5e1e1\\n\",\n \" style LOPBP fill:#e1e1f5\\n\",\n \"```\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Using Protocols in Functions\\n\",\n \"\\n\",\n \"Protocols enable type-safe function signatures:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"def process_observations(batch: ObsBatchProtocol) -> np.ndarray:\\n\",\n \" \\\"\\\"\\\"Function that expects observations.\\n\",\n \"\\n\",\n \" IDE will autocomplete batch.obs and batch.info!\\n\",\n \" Type checker will verify these fields exist.\\n\",\n \" \\\"\\\"\\\"\\n\",\n \" # IDE knows batch.obs exists\\n\",\n \" return batch.obs if isinstance(batch.obs, np.ndarray) else np.array(batch.obs)\\n\",\n \"\\n\",\n \"\\n\",\n \"def compute_advantage(batch: RolloutBatchProtocol) -> np.ndarray:\\n\",\n \" \\\"\\\"\\\"Function that expects rollout data.\\n\",\n \"\\n\",\n \" IDE will autocomplete batch.rew, batch.obs_next, etc.\\n\",\n \" \\\"\\\"\\\"\\n\",\n \" # Simplified advantage computation\\n\",\n \" return batch.rew # IDE knows this exists\\n\",\n \"\\n\",\n \"\\n\",\n \"# Example usage\\n\",\n \"obs_data = Batch(obs=np.array([1, 2, 3]), info=np.array([{}], dtype=object))\\n\",\n \"result = process_observations(obs_data)\\n\",\n \"print(\\\"Processed obs:\\\", result)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"**Key Protocol Types:**\\n\",\n \"\\n\",\n \"- `ActBatchProtocol`: Just actions (for simple policies)\\n\",\n \"- `ObsBatchProtocol`: Observations and info\\n\",\n \"- `RolloutBatchProtocol`: Complete transitions (obs, act, rew, done, obs_next)\\n\",\n \"- `BatchWithReturnsProtocol`: Rollouts + computed returns\\n\",\n \"- `BatchWithAdvantagesProtocol`: Returns + advantages and values\\n\",\n \"- `DistBatchProtocol`: Contains distribution objects\\n\",\n \"- `LogpOldProtocol`: For importance sampling (PPO, etc.)\\n\",\n \"\\n\",\n \"See `tianshou/data/types.py` for the complete list!\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## 5. Distribution Slicing\\n\",\n \"\\n\",\n \"### Why Special Handling?\\n\",\n \"\\n\",\n \"PyTorch `Distribution` objects need special slicing because they're not simple arrays. When you slice `batch[0:2]`, Tianshou needs to slice the underlying distribution parameters correctly.\\n\",\n \"\\n\",\n \"### Supported Distributions\\n\",\n \"\\n\",\n \"Tianshou supports slicing for:\\n\",\n \"- `Categorical`: Discrete distributions\\n\",\n \"- `Normal`: Continuous Gaussian distributions\\n\",\n \"- `Independent`: Wraps other distributions\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Categorical distribution\\n\",\n \"probs = torch.tensor([[0.3, 0.7], [0.4, 0.6], [0.5, 0.5]])\\n\",\n \"dist = Categorical(probs=probs)\\n\",\n \"batch = Batch(dist=dist, values=np.array([1, 2, 3]))\\n\",\n \"\\n\",\n \"print(\\\"Original batch length:\\\", len(batch))\\n\",\n \"print(\\\"Original dist probs shape:\\\", batch.dist.probs.shape)\\n\",\n \"\\n\",\n \"# Slicing automatically handles the distribution\\n\",\n \"sliced = batch[0:2]\\n\",\n \"print(\\\"\\\\nSliced batch length:\\\", len(sliced))\\n\",\n \"print(\\\"Sliced dist probs shape:\\\", sliced.dist.probs.shape)\\n\",\n \"print(\\\"Sliced values:\\\", sliced.values)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Normal distribution\\n\",\n \"loc = torch.tensor([0.0, 1.0, 2.0])\\n\",\n \"scale = torch.tensor([1.0, 1.0, 1.0])\\n\",\n \"normal_dist = Normal(loc=loc, scale=scale)\\n\",\n \"batch_normal = Batch(dist=normal_dist, actions=np.array([0.5, 1.5, 2.5]))\\n\",\n \"\\n\",\n \"print(\\\"Normal distribution batch:\\\")\\n\",\n \"print(f\\\"Original mean: {batch_normal.dist.mean}\\\")\\n\",\n \"\\n\",\n \"# Index a single element\\n\",\n \"single = batch_normal[1]\\n\",\n \"print(f\\\"\\\\nSingle element mean: {single.dist.mean}\\\")\\n\",\n \"print(f\\\"Single element action: {single.actions}\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Converting to At Least 2D\\n\",\n \"\\n\",\n \"Sometimes you need to ensure distributions have a batch dimension:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"from tianshou.data.batch import dist_to_atleast_2d\\n\",\n \"\\n\",\n \"# Scalar distribution (no batch dimension)\\n\",\n \"scalar_dist = Categorical(probs=torch.tensor([0.3, 0.7]))\\n\",\n \"print(\\\"Scalar dist batch_shape:\\\", scalar_dist.batch_shape)\\n\",\n \"\\n\",\n \"# Convert to have batch dimension\\n\",\n \"batched_dist = dist_to_atleast_2d(scalar_dist)\\n\",\n \"print(\\\"Batched dist batch_shape:\\\", batched_dist.batch_shape)\\n\",\n \"\\n\",\n \"# For entire batch\\n\",\n \"scalar_batch = Batch(a=1, b=2, dist=Categorical(probs=torch.ones(3)))\\n\",\n \"print(\\\"\\\\nBefore to_at_least_2d:\\\", scalar_batch.dist.batch_shape)\\n\",\n \"\\n\",\n \"batch_2d = scalar_batch.to_at_least_2d()\\n\",\n \"print(\\\"After to_at_least_2d:\\\", batch_2d.dist.batch_shape)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Use Cases\\n\",\n \"\\n\",\n \"Distribution slicing is used in:\\n\",\n \"- **Policy sampling**: When policies output distributions, slicing batches preserves distribution structure\\n\",\n \"- **Replay buffer sampling**: Distributions are stored and retrieved correctly\\n\",\n \"- **Advantage computation**: Computing log probabilities on subsets of data\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## 6. Advanced Topics\\n\",\n \"\\n\",\n \"### 6.1 Key Reservation\\n\",\n \"\\n\",\n \"Sometimes you know what keys you'll need but don't have values yet. Reserve keys using empty `Batch()` objects:\\n\",\n \"\\n\",\n \"```mermaid\\n\",\n \"graph TD\\n\",\n \" root[\\\"Batch\\\"]\\n\",\n \" root --> a[\\\"key1: np.array([1,2,3])\\\"]\\n\",\n \" root --> b[\\\"key2: Batch() (reserved)\\\"]\\n\",\n \" root --> c[\\\"key3\\\"]\\n\",\n \" c --> c1[\\\"subkey1: Batch() (reserved)\\\"]\\n\",\n \" c --> c2[\\\"subkey2: np.array([4,5])\\\"]\\n\",\n \" \\n\",\n \" style root fill:#e1f5ff\\n\",\n \" style a fill:#e8f5e1\\n\",\n \" style b fill:#ffcccc\\n\",\n \" style c fill:#fff4e1\\n\",\n \" style c1 fill:#ffcccc\\n\",\n \" style c2 fill:#e8f5e1\\n\",\n \"```\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Reserving keys\\n\",\n \"batch = Batch(\\n\",\n \" known_field=np.array([1, 2]),\\n\",\n \" future_field=Batch(), # Reserved for later\\n\",\n \")\\n\",\n \"print(\\\"Batch with reserved key:\\\")\\n\",\n \"print(batch)\\n\",\n \"\\n\",\n \"# Later, assign actual data\\n\",\n \"batch.future_field = np.array([3, 4])\\n\",\n \"print(\\\"\\\\nAfter assignment:\\\")\\n\",\n \"print(batch)\\n\",\n \"\\n\",\n \"# Nested reservation\\n\",\n \"batch2 = Batch(\\n\",\n \" obs=Batch(\\n\",\n \" camera=Batch(), # Reserved\\n\",\n \" lidar=np.zeros(10),\\n\",\n \" )\\n\",\n \")\\n\",\n \"print(\\\"\\\\nNested reservation:\\\")\\n\",\n \"print(batch2)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 6.2 Length and Shape Semantics\\n\",\n \"\\n\",\n \"Understanding when `len()` works and what `shape` means:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Normal case: all tensors same length\\n\",\n \"batch1 = Batch(a=[1, 2, 3], b=np.array([4, 5, 6]))\\n\",\n \"print(\\\"Normal batch:\\\")\\n\",\n \"print(f\\\"len(batch1) = {len(batch1)}\\\")\\n\",\n \"print(f\\\"batch1.shape = {batch1.shape}\\\")\\n\",\n \"\\n\",\n \"# Scalars have no length\\n\",\n \"batch2 = Batch(a=5, b=10)\\n\",\n \"print(\\\"\\\\nScalar batch:\\\")\\n\",\n \"print(f\\\"batch2.shape = {batch2.shape}\\\")\\n\",\n \"try:\\n\",\n \" print(f\\\"len(batch2) = {len(batch2)}\\\")\\n\",\n \"except TypeError as e:\\n\",\n \" print(f\\\"len(batch2) raises TypeError: {e}\\\")\\n\",\n \"\\n\",\n \"# Mixed lengths: returns minimum\\n\",\n \"batch3 = Batch(a=[1, 2], b=[3, 4, 5])\\n\",\n \"print(\\\"\\\\nMixed length batch:\\\")\\n\",\n \"print(f\\\"len(batch3) = {len(batch3)} (minimum of 2 and 3)\\\")\\n\",\n \"\\n\",\n \"# Reserved keys are ignored\\n\",\n \"batch4 = Batch(a=[1, 2, 3], reserved=Batch())\\n\",\n \"print(\\\"\\\\nBatch with reserved key:\\\")\\n\",\n \"print(f\\\"len(batch4) = {len(batch4)} (reserved key ignored)\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 6.3 Empty Batches\\n\",\n \"\\n\",\n \"Understanding different meanings of \\\"empty\\\":\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# 1. No keys at all\\n\",\n \"empty1 = Batch()\\n\",\n \"print(\\\"No keys:\\\")\\n\",\n \"print(f\\\"len(empty1.get_keys()) = {len(list(empty1.get_keys()))}\\\")\\n\",\n \"print(f\\\"len(empty1) = {len(empty1)}\\\")\\n\",\n \"\\n\",\n \"# 2. Has keys but they're all reserved\\n\",\n \"empty2 = Batch(a=Batch(), b=Batch())\\n\",\n \"print(\\\"\\\\nReserved keys only:\\\")\\n\",\n \"print(f\\\"len(empty2.get_keys()) = {len(list(empty2.get_keys()))}\\\")\\n\",\n \"print(f\\\"len(empty2) = {len(empty2)}\\\")\\n\",\n \"\\n\",\n \"# 3. Has data but length is 0\\n\",\n \"empty3 = Batch(a=np.array([]), b=np.array([]))\\n\",\n \"print(\\\"\\\\nZero-length arrays:\\\")\\n\",\n \"print(f\\\"len(empty3.get_keys()) = {len(list(empty3.get_keys()))}\\\")\\n\",\n \"print(f\\\"len(empty3) = {len(empty3)}\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"**Checking emptiness:**\\n\",\n \"- `len(batch.get_keys()) == 0`: No keys (completely empty)\\n\",\n \"- `len(batch) == 0`: No data elements (may have reserved keys)\\n\",\n \"\\n\",\n \"**The `.empty()` and `.empty_()` methods:**\\n\",\n \"These reset values to zeros/None, different from checking emptiness:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"batch = Batch(a=[1, 2, 3], b=[\\\"x\\\", \\\"y\\\", \\\"z\\\"])\\n\",\n \"print(\\\"Original:\\\", batch)\\n\",\n \"\\n\",\n \"# Empty specific index\\n\",\n \"batch[0] = Batch.empty(batch[0])\\n\",\n \"print(\\\"\\\\nAfter emptying index 0:\\\")\\n\",\n \"print(batch)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 6.4 Heterogeneous Aggregation\\n\",\n \"\\n\",\n \"Stacking/concatenating batches with different keys:\\n\",\n \"\\n\",\n \"```mermaid\\n\",\n \"graph LR\\n\",\n \" A[\\\"Batch(a=[1,2], c=5)\\\"] --> C[\\\"Batch.stack\\\"]\\n\",\n \" B[\\\"Batch(b=[3,4], c=6)\\\"] --> C\\n\",\n \" C --> D[\\\"Batch(a=[[1,2],[0,0]],
b=[[0,0],[3,4]],
c=[5,6])\\\"]\\n\",\n \" \\n\",\n \" style A fill:#e1f5ff\\n\",\n \" style B fill:#fff4e1\\n\",\n \" style C fill:#ffe1f5\\n\",\n \" style D fill:#e8f5e1\\n\",\n \"```\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Stack with different keys (missing keys padded with zeros)\\n\",\n \"batch_a = Batch(a=np.ones((2, 3)), shared=np.array([1, 2]))\\n\",\n \"batch_b = Batch(b=np.zeros((2, 4)), shared=np.array([3, 4]))\\n\",\n \"\\n\",\n \"stacked = Batch.stack([batch_a, batch_b])\\n\",\n \"print(\\\"Stacked batch:\\\")\\n\",\n \"print(f\\\"a.shape = {stacked.a.shape} (padded with zeros for batch_b)\\\")\\n\",\n \"print(f\\\"b.shape = {stacked.b.shape} (padded with zeros for batch_a)\\\")\\n\",\n \"print(f\\\"shared.shape = {stacked.shared.shape} (in both batches)\\\")\\n\",\n \"print(stacked)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 6.5 Missing Values\\n\",\n \"\\n\",\n \"Handling `None` and `NaN` values:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Batch with missing values\\n\",\n \"batch = Batch(a=[1, 2, None, 4], b=[5.0, np.nan, 7.0, 8.0], c=[[1, 2], [3, 4], [5, 6], [7, 8]])\\n\",\n \"\\n\",\n \"# Check for nulls\\n\",\n \"print(\\\"Has null?\\\", batch.hasnull())\\n\",\n \"\\n\",\n \"# Get null mask\\n\",\n \"null_mask = batch.isnull()\\n\",\n \"print(\\\"\\\\nNull mask:\\\")\\n\",\n \"print(f\\\"a: {null_mask.a}\\\")\\n\",\n \"print(f\\\"b: {null_mask.b}\\\")\\n\",\n \"\\n\",\n \"# Drop rows with any null\\n\",\n \"clean_batch = batch.dropnull()\\n\",\n \"print(\\\"\\\\nAfter dropnull() (keeps rows 0 and 3):\\\")\\n\",\n \"print(f\\\"Length: {len(clean_batch)}\\\")\\n\",\n \"print(f\\\"a: {clean_batch.a}\\\")\\n\",\n \"print(f\\\"b: {clean_batch.b}\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 6.6 Value Transformations\\n\",\n \"\\n\",\n \"Applying functions to all values recursively:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"batch = Batch(a=np.array([1, 2, 3]), nested=Batch(b=np.array([4.0, 5.0]), c=np.array([6, 7, 8])))\\n\",\n \"\\n\",\n \"# Apply transformation (returns new batch)\\n\",\n \"doubled = batch.apply_values_transform(lambda x: x * 2)\\n\",\n \"print(\\\"Original batch a:\\\", batch.a)\\n\",\n \"print(\\\"Doubled batch a:\\\", doubled.a)\\n\",\n \"print(\\\"Doubled nested.b:\\\", doubled.nested.b)\\n\",\n \"\\n\",\n \"# In-place transformation\\n\",\n \"batch.apply_values_transform(lambda x: x + 10, inplace=True)\\n\",\n \"print(\\\"\\\\nAfter in-place +10:\\\")\\n\",\n \"print(\\\"a:\\\", batch.a)\\n\",\n \"print(\\\"nested.b:\\\", batch.nested.b)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## 7. Surprising Behaviors & Gotchas\\n\",\n \"\\n\",\n \"### Iteration Does NOT Iterate Over Keys!\\n\",\n \"\\n\",\n \"**This is the most common source of confusion:**\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"batch = Batch(a=[1, 2, 3], b=[4, 5, 6])\\n\",\n \"\\n\",\n \"print(\\\"WRONG: This doesn't iterate over keys!\\\")\\n\",\n \"for item in batch:\\n\",\n \" print(f\\\"item = {item}\\\") # Prints batch[0], batch[1], batch[2]\\n\",\n \"\\n\",\n \"print(\\\"\\\\nCORRECT: To iterate over keys:\\\")\\n\",\n \"for key in batch.keys():\\n\",\n \" print(f\\\"key = {key}\\\")\\n\",\n \"\\n\",\n \"print(\\\"\\\\nCORRECT: To iterate over key-value pairs:\\\")\\n\",\n \"for key, value in batch.items():\\n\",\n \" print(f\\\"{key} = {value}\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Automatic Type Conversions\\n\",\n \"\\n\",\n \"Be aware of these automatic conversions:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Lists become arrays\\n\",\n \"batch = Batch(a=[1, 2, 3])\\n\",\n \"print(\\\"List → array:\\\", type(batch.a), batch.a.dtype)\\n\",\n \"\\n\",\n \"# Dicts become Batch\\n\",\n \"batch = Batch(a={\\\"x\\\": 1, \\\"y\\\": 2})\\n\",\n \"print(\\\"Dict → Batch:\\\", type(batch.a))\\n\",\n \"\\n\",\n \"# Scalars become numpy scalars\\n\",\n \"batch = Batch(a=5)\\n\",\n \"print(\\\"Scalar → np.ndarray:\\\", type(batch.a), batch.a)\\n\",\n \"\\n\",\n \"# Mixed types → object dtype\\n\",\n \"batch = Batch(a=[1, \\\"hello\\\", None])\\n\",\n \"print(\\\"Mixed → object:\\\", batch.a.dtype, batch.a)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Length Edge Cases\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# 1. Scalars have no length\\n\",\n \"batch_scalar = Batch(a=5, b=10)\\n\",\n \"try:\\n\",\n \" len(batch_scalar)\\n\",\n \"except TypeError as e:\\n\",\n \" print(f\\\"Scalar batch: {e}\\\")\\n\",\n \"\\n\",\n \"# 2. Empty nested batches ignored in len()\\n\",\n \"batch_empty_nested = Batch(a=[1, 2, 3], b=Batch())\\n\",\n \"print(f\\\"\\\\nWith empty nested: len = {len(batch_empty_nested)} (ignores b)\\\")\\n\",\n \"\\n\",\n \"# 3. Different lengths: returns minimum\\n\",\n \"batch_different = Batch(a=[1, 2], b=[1, 2, 3, 4])\\n\",\n \"print(f\\\"Different lengths: len = {len(batch_different)} (minimum)\\\")\\n\",\n \"\\n\",\n \"# 4. None values don't affect length\\n\",\n \"batch_none = Batch(a=[1, 2, 3], b=None)\\n\",\n \"print(f\\\"With None: len = {len(batch_none)} (None ignored)\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### String Keys Only\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Integer keys not allowed\\n\",\n \"try:\\n\",\n \" batch = Batch({1: \\\"value\\\", 2: \\\"other\\\"})\\n\",\n \"except AssertionError as e:\\n\",\n \" print(\\\"Integer keys not allowed:\\\", e)\\n\",\n \"\\n\",\n \"# String keys work\\n\",\n \"batch = Batch({\\\"key1\\\": \\\"value\\\", \\\"key2\\\": \\\"other\\\"})\\n\",\n \"print(\\\"\\\\nString keys work:\\\", list(batch.keys()))\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Cat vs Stack Behavior\\n\",\n \"\\n\",\n \"Recent changes have made concatenation stricter about structure:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Stack pads missing keys with zeros\\n\",\n \"b1 = Batch(a=[1, 2])\\n\",\n \"b2 = Batch(b=[3, 4])\\n\",\n \"stacked = Batch.stack([b1, b2])\\n\",\n \"print(\\\"Stack (different keys):\\\")\\n\",\n \"print(f\\\" a: {stacked.a} (b2.a padded with 0)\\\")\\n\",\n \"print(f\\\" b: {stacked.b} (b1.b padded with 0)\\\")\\n\",\n \"\\n\",\n \"# Cat requires same structure now\\n\",\n \"b3 = Batch(a=[1, 2], b=[3, 4])\\n\",\n \"b4 = Batch(a=[5, 6], b=[7, 8])\\n\",\n \"concatenated = Batch.cat([b3, b4])\\n\",\n \"print(\\\"\\\\nCat (same keys):\\\")\\n\",\n \"print(f\\\" a: {concatenated.a}\\\")\\n\",\n \"print(f\\\" b: {concatenated.b}\\\")\\n\",\n \"\\n\",\n \"# Cat with different structures raises error\\n\",\n \"try:\\n\",\n \" Batch.cat([b1, b2]) # Different keys!\\n\",\n \"except ValueError:\\n\",\n \" print(\\\"\\\\nCat with different keys: ValueError raised\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## 8. Best Practices\\n\",\n \"\\n\",\n \"### When to Use Batch\\n\",\n \"\\n\",\n \"**Good use cases:**\\n\",\n \"- Collecting environment data (transitions, episodes)\\n\",\n \"- Storing replay buffer data\\n\",\n \"- Passing data between components (collector → buffer → policy)\\n\",\n \"- Handling heterogeneous observations (dict spaces)\\n\",\n \"\\n\",\n \"**Consider alternatives:**\\n\",\n \"- Simple scalar tracking (use regular variables)\\n\",\n \"- Pure tensor operations (use PyTorch tensors directly)\\n\",\n \"- Deeply nested arbitrary structures (use dataclasses)\\n\",\n \"\\n\",\n \"### Structuring Your Batches\\n\",\n \"\\n\",\n \"**Use protocols for type safety:**\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Good: Use protocols for clear interfaces\\n\",\n \"def train_step(batch: RolloutBatchProtocol) -> float:\\n\",\n \" \\\"\\\"\\\"IDE knows what fields exist.\\\"\\\"\\\"\\n\",\n \" loss = ((batch.rew - 0.5) ** 2).mean() # Type-safe\\n\",\n \" return float(loss)\\n\",\n \"\\n\",\n \"\\n\",\n \"# Create properly typed batch\\n\",\n \"train_batch = cast(\\n\",\n \" RolloutBatchProtocol,\\n\",\n \" Batch(\\n\",\n \" obs=np.random.randn(10, 4),\\n\",\n \" obs_next=np.random.randn(10, 4),\\n\",\n \" act=np.random.randint(0, 2, 10),\\n\",\n \" rew=np.random.randn(10),\\n\",\n \" terminated=np.zeros(10, dtype=bool),\\n\",\n \" truncated=np.zeros(10, dtype=bool),\\n\",\n \" info=np.array([{}] * 10, dtype=object),\\n\",\n \" ),\\n\",\n \")\\n\",\n \"\\n\",\n \"loss = train_step(train_batch)\\n\",\n \"print(f\\\"Loss: {loss:.4f}\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"**Consistent key naming:**\\n\",\n \"- Follow Tianshou conventions: `obs`, `act`, `rew`, `terminated`, `truncated`\\n\",\n \"- Use descriptive names: `camera_obs` not `co`\\n\",\n \"- Avoid name collisions with Batch methods: don't use `keys`, `items`, `get`, etc.\\n\",\n \"\\n\",\n \"**When to nest vs flatten:**\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Good: Nest related data\\n\",\n \"batch_nested = Batch(\\n\",\n \" obs=Batch(\\n\",\n \" camera=np.zeros((32, 64, 64, 3)), lidar=np.zeros((32, 100)), position=np.zeros((32, 3))\\n\",\n \" ),\\n\",\n \" act=np.zeros(32),\\n\",\n \")\\n\",\n \"print(\\\"Nested structure for related obs:\\\")\\n\",\n \"print(f\\\" Access: batch.obs.camera.shape = {batch_nested.obs.camera.shape}\\\")\\n\",\n \"\\n\",\n \"# Less good: Flat structure loses semantic grouping\\n\",\n \"batch_flat = Batch(\\n\",\n \" camera=np.zeros((32, 64, 64, 3)),\\n\",\n \" lidar=np.zeros((32, 100)),\\n\",\n \" position=np.zeros((32, 3)),\\n\",\n \" act=np.zeros(32),\\n\",\n \")\\n\",\n \"print(\\\"\\\\nFlat structure (works but less clear):\\\")\\n\",\n \"print(f\\\" Access: batch.camera.shape = {batch_flat.camera.shape}\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Performance Tips\\n\",\n \"\\n\",\n \"**Use in-place operations:**\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import time\\n\",\n \"\\n\",\n \"batch = Batch(a=np.random.randn(1000, 100))\\n\",\n \"\\n\",\n \"# Creates copy\\n\",\n \"start = time.time()\\n\",\n \"for _ in range(100):\\n\",\n \" _ = batch.to_torch()\\n\",\n \"time_copy = time.time() - start\\n\",\n \"\\n\",\n \"# In-place (faster)\\n\",\n \"start = time.time()\\n\",\n \"for _ in range(100):\\n\",\n \" batch.to_torch_()\\n\",\n \" batch.to_numpy_()\\n\",\n \"time_inplace = time.time() - start\\n\",\n \"\\n\",\n \"print(f\\\"Copy: {time_copy:.4f}s\\\")\\n\",\n \"print(f\\\"In-place: {time_inplace:.4f}s\\\")\\n\",\n \"print(f\\\"Speedup: {time_copy / time_inplace:.1f}x\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"**Be mindful of copies:**\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"arr = np.array([1, 2, 3])\\n\",\n \"\\n\",\n \"# Default: creates reference (be careful!)\\n\",\n \"batch1 = Batch(a=arr)\\n\",\n \"batch1.a[0] = 999\\n\",\n \"print(f\\\"Original array modified: {arr}\\\") # Changed!\\n\",\n \"\\n\",\n \"# Explicit copy when needed\\n\",\n \"arr = np.array([1, 2, 3])\\n\",\n \"batch2 = Batch(a=arr, copy=True)\\n\",\n \"batch2.a[0] = 999\\n\",\n \"print(f\\\"Original array preserved: {arr}\\\") # Unchanged\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"**Avoid unnecessary conversions:**\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Inefficient: multiple conversions\\n\",\n \"batch = Batch(a=np.random.randn(100, 10))\\n\",\n \"batch.to_torch_()\\n\",\n \"batch.to_numpy_() # Unnecessary if we just need NumPy\\n\",\n \"\\n\",\n \"# Efficient: convert once, use many times\\n\",\n \"batch = Batch(a=np.random.randn(100, 10))\\n\",\n \"batch.to_torch_() # Convert once\\n\",\n \"# ... do torch operations ...\\n\",\n \"# Keep as torch if that's what you need!\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Common Patterns\\n\",\n \"\\n\",\n \"**Pattern 1: Building batches incrementally**\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Collect data from multiple steps\\n\",\n \"step_data = []\\n\",\n \"for i in range(5):\\n\",\n \" step_data.append({\\\"obs\\\": np.random.randn(4), \\\"act\\\": i, \\\"rew\\\": np.random.randn()})\\n\",\n \"\\n\",\n \"# Convert to batch (automatically stacks)\\n\",\n \"episode_batch = Batch(step_data)\\n\",\n \"print(\\\"Episode batch shape:\\\", episode_batch.shape)\\n\",\n \"print(\\\"obs shape:\\\", episode_batch.obs.shape)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"**Pattern 2: Slicing for mini-batches**\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Large batch\\n\",\n \"large_batch = Batch(obs=np.random.randn(100, 4), act=np.random.randint(0, 2, 100))\\n\",\n \"\\n\",\n \"# Split into mini-batches\\n\",\n \"batch_size = 32\\n\",\n \"for mini_batch in large_batch.split(batch_size, shuffle=True):\\n\",\n \" print(f\\\"Mini-batch size: {len(mini_batch)}\\\")\\n\",\n \" # Train on mini_batch...\\n\",\n \" break # Just show one iteration\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"**Pattern 3: Extending batches**\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Start with some data\\n\",\n \"batch = Batch(obs=np.array([[1, 2], [3, 4]]), act=np.array([0, 1]))\\n\",\n \"print(\\\"Initial:\\\", len(batch))\\n\",\n \"\\n\",\n \"# Add more data\\n\",\n \"new_data = Batch(obs=np.array([[5, 6]]), act=np.array([1]))\\n\",\n \"batch.cat_(new_data)\\n\",\n \"print(\\\"After cat_:\\\", len(batch))\\n\",\n \"print(\\\"obs:\\\", batch.obs)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## 9. Summary\\n\",\n \"\\n\",\n \"### Key Takeaways\\n\",\n \"\\n\",\n \"1. **Batch = Dict + Array**: Combines key-value storage with array operations\\n\",\n \"2. **Hierarchical Structure**: Perfect for complex RL data (nested observations, etc.)\\n\",\n \"3. **Type Safety via Protocols**: Use `BatchProtocol` subclasses for IDE support and type checking\\n\",\n \"4. **Special RL Features**: Distribution slicing, heterogeneous aggregation, missing value handling\\n\",\n \"5. **Remember**: Iteration is over indices, NOT keys!\\n\",\n \"\\n\",\n \"### Quick Reference\\n\",\n \"\\n\",\n \"| Operation | Code | Notes |\\n\",\n \"|-----------|------|-------|\\n\",\n \"| Create | `Batch(a=1, b=[2, 3])` | Auto-converts types |\\n\",\n \"| Access | `batch.a` or `batch[\\\"a\\\"]` | Equivalent |\\n\",\n \"| Index | `batch[0]`, `batch[:10]` | Returns sliced Batch |\\n\",\n \"| Iterate indices | `for item in batch:` | Yields batch[0], batch[1], ... |\\n\",\n \"| Iterate keys | `for k in batch.keys():` | Like dict |\\n\",\n \"| Stack | `Batch.stack([b1, b2])` | Adds dimension |\\n\",\n \"| Concatenate | `Batch.cat([b1, b2])` | Extends dimension |\\n\",\n \"| Split | `batch.split(size=10)` | Returns iterator |\\n\",\n \"| To PyTorch | `batch.to_torch_()` | In-place |\\n\",\n \"| To NumPy | `batch.to_numpy_()` | In-place |\\n\",\n \"| Transform | `batch.apply_values_transform(fn)` | Recursive |\\n\",\n \"\\n\",\n \"### Next Steps\\n\",\n \"\\n\",\n \"- **Collector Deep Dive**: See how Batch flows through data collection\\n\",\n \"- **Buffer Deep Dive**: Understand how Batch is stored and sampled\\n\",\n \"- **Policy Guide**: Learn how policies work with BatchProtocol\\n\",\n \"- **API Reference**: Full details at [Batch API documentation](https://tianshou.org/en/stable/api/tianshou.data.html#tianshou.data.Batch)\\n\",\n \"\\n\",\n \"### Questions?\\n\",\n \"\\n\",\n \"- Check the [Tianshou GitHub discussions](https://github.com/thu-ml/tianshou/discussions)\\n\",\n \"- Review [issue tracker](https://github.com/thu-ml/tianshou/issues) for known gotchas\\n\",\n \"- Read the [source code](https://github.com/thu-ml/tianshou/blob/master/tianshou/data/batch.py) - it's well-documented!\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Appendix: Serialization & Advanced Topics\\n\",\n \"\\n\",\n \"### Pickle Support\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Batch objects are picklable\\n\",\n \"original = Batch(obs=Batch(a=0.0, c=torch.Tensor([1.0, 2.0])), np=np.zeros([3, 4]))\\n\",\n \"\\n\",\n \"# Serialize and deserialize\\n\",\n \"serialized = pickle.dumps(original)\\n\",\n \"restored = pickle.loads(serialized)\\n\",\n \"\\n\",\n \"print(\\\"Original obs.a:\\\", original.obs.a)\\n\",\n \"print(\\\"Restored obs.a:\\\", restored.obs.a)\\n\",\n \"print(\\\"Equal:\\\", original == restored)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Advanced Indexing\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Multi-dimensional data\\n\",\n \"batch = Batch(a=np.random.randn(5, 3, 2))\\n\",\n \"print(\\\"Original shape:\\\", batch.a.shape)\\n\",\n \"\\n\",\n \"# Various indexing operations\\n\",\n \"print(\\\"batch[0].a.shape:\\\", batch[0].a.shape)\\n\",\n \"print(\\\"batch[:, 0].a.shape:\\\", batch[:, 0].a.shape)\\n\",\n \"print(\\\"batch[[0, 2, 4]].a.shape:\\\", batch[[0, 2, 4]].a.shape)\"\n ]\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.11.0\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 4\n}\n" }, { "path": "docs/02_deep_dives/L2_Buffer.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Buffer: Experience Replay in Tianshou\\n\",\n \"\\n\",\n \"The replay buffer is a fundamental component in reinforcement learning, particularly for off-policy algorithms. Tianshou's buffer implementation extends beyond simple data storage to provide sophisticated trajectory tracking, efficient sampling, and seamless integration with the RL training pipeline.\\n\",\n \"\\n\",\n \"This tutorial provides comprehensive coverage of Tianshou's buffer system, from basic concepts to advanced features and integration patterns.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import pickle\\n\",\n \"import tempfile\\n\",\n \"\\n\",\n \"import numpy as np\\n\",\n \"\\n\",\n \"from tianshou.data import Batch, PrioritizedReplayBuffer, ReplayBuffer, VectorReplayBuffer\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## 1. Introduction: Why Buffers in Reinforcement Learning?\\n\",\n \"\\n\",\n \"### The Role of Experience Replay\\n\",\n \"\\n\",\n \"Experience replay is a critical technique in modern reinforcement learning that addresses three fundamental challenges:\\n\",\n \"\\n\",\n \"1. **Breaking Temporal Correlation**: Sequential experiences from an agent are highly correlated. Training directly on these sequences can lead to unstable learning. By storing experiences and sampling randomly, we break these correlations.\\n\",\n \"\\n\",\n \"2. **Sample Efficiency**: In RL, collecting data through environment interaction is often expensive. Experience replay allows us to reuse each experience multiple times for training, dramatically improving sample efficiency.\\n\",\n \"\\n\",\n \"3. **Mini-batch Training**: Modern deep learning requires mini-batch gradient descent. Buffers enable efficient batching of experiences for neural network training.\\n\",\n \"\\n\",\n \"### Why Not Alternatives?\\n\",\n \"\\n\",\n \"**Plain Python Lists**\\n\",\n \"- No efficient random sampling\\n\",\n \"- No automatic circular queue behavior\\n\",\n \"- No trajectory boundary tracking\\n\",\n \"- Poor memory management for large datasets\\n\",\n \"\\n\",\n \"**Simple Batch Storage**\\n\",\n \"- No automatic overwriting when full\\n\",\n \"- No episode metadata (returns, lengths)\\n\",\n \"- No methods for boundary navigation (prev/next)\\n\",\n \"- No specialized sampling strategies\\n\",\n \"\\n\",\n \"### Buffer = Batch + Trajectory Management + Sampling\\n\",\n \"\\n\",\n \"Tianshou's buffers build on the `Batch` class to provide:\\n\",\n \"- **Circular queue storage**: Automatic overwriting of oldest data\\n\",\n \"- **Trajectory tracking**: Episode boundaries, returns, and lengths\\n\",\n \"- **Efficient sampling**: Random access with various strategies\\n\",\n \"- **Integration utilities**: Seamless connection to Collector and Policy\\n\",\n \"\\n\",\n \"### Use Cases\\n\",\n \"\\n\",\n \"- **Off-policy algorithms**: DQN, SAC, TD3, DDPG require experience replay\\n\",\n \"- **On-policy with replay**: Some PPO implementations reuse buffer data\\n\",\n \"- **Offline RL**: Loading and using pre-collected datasets\\n\",\n \"- **Multi-environment training**: VectorReplayBuffer for parallel collection\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## 2. Buffer Types and Hierarchy\\n\",\n \"\\n\",\n \"Tianshou provides several buffer implementations, each designed for specific use cases. Understanding this hierarchy is crucial for choosing the right buffer.\\n\",\n \"\\n\",\n \"### Buffer Hierarchy\\n\",\n \"\\n\",\n \"```mermaid\\n\",\n \"graph TD\\n\",\n \" RB[ReplayBuffer
Single environment
Circular queue] --> RBM[ReplayBufferManager
Manages multiple buffers
Contiguous memory]\\n\",\n \" RBM --> VRB[VectorReplayBuffer
Parallel environments
Maintains temporal order]\\n\",\n \" \\n\",\n \" RB --> PRB[PrioritizedReplayBuffer
TD-error based sampling
Importance weights]\\n\",\n \" PRB --> PVRB[PrioritizedVectorReplayBuffer
Prioritized + Parallel]\\n\",\n \" \\n\",\n \" RB --> CRB[CachedReplayBuffer
Primary + auxiliary caches
Imitation learning]\\n\",\n \" \\n\",\n \" RB --> HERB[HERReplayBuffer
Hindsight Experience Replay
Goal-conditioned RL]\\n\",\n \" HERB --> HVRB[HERVectorReplayBuffer
HER + Parallel]\\n\",\n \" \\n\",\n \" style RB fill:#e1f5ff\\n\",\n \" style RBM fill:#fff4e1\\n\",\n \" style VRB fill:#ffe1f5\\n\",\n \" style PRB fill:#e8f5e1\\n\",\n \" style CRB fill:#f5e1e1\\n\",\n \" style HERB fill:#e1e1f5\\n\",\n \"```\\n\",\n \"\\n\",\n \"### When to Use Which Buffer\\n\",\n \"\\n\",\n \"**ReplayBuffer**: Single environment scenarios\\n\",\n \"- Simple setup and testing\\n\",\n \"- Debugging algorithms\\n\",\n \"- Low-parallelism training\\n\",\n \"\\n\",\n \"**VectorReplayBuffer**: Multiple parallel environments (most common)\\n\",\n \"- Standard production use case\\n\",\n \"- Efficient parallel data collection\\n\",\n \"- Maintains per-environment episode boundaries\\n\",\n \"\\n\",\n \"**PrioritizedReplayBuffer**: DQN variants with prioritization\\n\",\n \"- Rainbow DQN\\n\",\n \"- Algorithms requiring importance sampling\\n\",\n \"- When some transitions are more valuable than others\\n\",\n \"\\n\",\n \"**CachedReplayBuffer**: Separate primary and auxiliary caches\\n\",\n \"- Imitation learning (expert + agent data)\\n\",\n \"- GAIL and similar algorithms\\n\",\n \"- When you need different sampling strategies for different data sources\\n\",\n \"\\n\",\n \"**HERReplayBuffer**: Goal-conditioned reinforcement learning\\n\",\n \"- Sparse reward environments\\n\",\n \"- Robotics tasks with explicit goals\\n\",\n \"- Relabeling failed experiences with achieved goals\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## 3. Basic Operations\\n\",\n \"\\n\",\n \"### 3.1 Construction and Configuration\\n\",\n \"\\n\",\n \"The ReplayBuffer constructor accepts several important parameters that control its behavior:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Create a buffer with all configuration options\\n\",\n \"buf = ReplayBuffer(\\n\",\n \" size=20, # Maximum capacity (transitions)\\n\",\n \" stack_num=1, # Frame stacking for RNNs (default: 1, no stacking)\\n\",\n \" ignore_obs_next=False, # Save memory by not storing obs_next\\n\",\n \" save_only_last_obs=False, # For temporal stacking (Atari-style)\\n\",\n \" sample_avail=False, # Sample only valid indices for frame stacking\\n\",\n \" random_seed=42, # Reproducible sampling\\n\",\n \")\\n\",\n \"\\n\",\n \"print(f\\\"Buffer created: {buf}\\\")\\n\",\n \"print(f\\\"Max size: {buf.maxsize}\\\")\\n\",\n \"print(f\\\"Current length: {len(buf)}\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"**Parameter Explanations**:\\n\",\n \"\\n\",\n \"- `size`: Maximum number of transitions the buffer can hold. When full, oldest data is overwritten.\\n\",\n \"- `stack_num`: Number of consecutive frames to stack. Used for RNN inputs or frame-based policies (Atari).\\n\",\n \"- `ignore_obs_next`: If True, obs_next is not stored, saving memory. The buffer reconstructs it from the next obs when needed.\\n\",\n \"- `save_only_last_obs`: For temporal stacking. Only saves the last observation in a stack.\\n\",\n \"- `sample_avail`: When True with stack_num > 1, only samples indices where a complete stack is available.\\n\",\n \"- `random_seed`: Seeds the random number generator for reproducible sampling.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 3.2 Reserved Keys and the Done Flag System\\n\",\n \"\\n\",\n \"ReplayBuffer uses nine reserved keys that integrate with Gymnasium conventions. Understanding the done flag system is critical.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# The nine reserved keys\\n\",\n \"print(\\\"Reserved keys:\\\")\\n\",\n \"print(ReplayBuffer._reserved_keys)\\n\",\n \"print(\\\"\\\\nKeys required for add():\\\")\\n\",\n \"print(ReplayBuffer._required_keys_for_add)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"**Important: Understanding done, terminated, and truncated**\\n\",\n \"\\n\",\n \"Gymnasium (the successor to OpenAI Gym) introduced a crucial distinction:\\n\",\n \"\\n\",\n \"- `terminated`: Episode ended naturally (agent reached goal or failed)\\n\",\n \" - Examples: CartPole fell over, agent reached goal state\\n\",\n \" - Should be used for bootstrapping calculations\\n\",\n \"\\n\",\n \"- `truncated`: Episode was cut off artificially (time limit, external interruption)\\n\",\n \" - Examples: Maximum episode length reached, environment reset externally \\n\",\n \" - Should NOT be used for bootstrapping (the episode could have continued)\\n\",\n \"\\n\",\n \"- `done`: Computed automatically as `terminated OR truncated`\\n\",\n \" - Used internally for episode boundary tracking\\n\",\n \" - You should NEVER manually set this field\\n\",\n \"\\n\",\n \"**Best Practice**: Always use the `info` dictionary for custom metadata rather than adding top-level keys:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# GOOD: Custom metadata in info dictionary\\n\",\n \"good_batch = Batch(\\n\",\n \" obs=np.array([1.0, 2.0]),\\n\",\n \" act=0,\\n\",\n \" rew=1.0,\\n\",\n \" terminated=False,\\n\",\n \" truncated=False,\\n\",\n \" obs_next=np.array([1.5, 2.5]),\\n\",\n \" info={\\\"custom_metric\\\": 0.95, \\\"step_count\\\": 10}, # Custom data here\\n\",\n \")\\n\",\n \"\\n\",\n \"# BAD: Don't add custom top-level keys (may conflict with future buffer features)\\n\",\n \"# bad_batch = Batch(..., custom_metric=0.95) # Don't do this!\\n\",\n \"\\n\",\n \"print(\\\"Good batch structure:\\\")\\n\",\n \"print(good_batch)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 3.3 Circular Queue Storage\\n\",\n \"\\n\",\n \"The buffer implements a circular queue: when it reaches maximum capacity, new data overwrites the oldest entries.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Create a small buffer to demonstrate circular behavior\\n\",\n \"demo_buf = ReplayBuffer(size=5)\\n\",\n \"\\n\",\n \"print(\\\"Adding 3 transitions:\\\")\\n\",\n \"for i in range(3):\\n\",\n \" demo_buf.add(\\n\",\n \" Batch(\\n\",\n \" obs=i,\\n\",\n \" act=i,\\n\",\n \" rew=float(i),\\n\",\n \" terminated=False,\\n\",\n \" truncated=False,\\n\",\n \" obs_next=i + 1,\\n\",\n \" info={},\\n\",\n \" )\\n\",\n \" )\\n\",\n \"print(f\\\"Length: {len(demo_buf)}, Max: {demo_buf.maxsize}\\\")\\n\",\n \"print(f\\\"Observations: {demo_buf.obs[: len(demo_buf)]}\\\")\\n\",\n \"\\n\",\n \"print(\\\"\\\\nAdding 5 more transitions (total 8, exceeds capacity 5):\\\")\\n\",\n \"for i in range(3, 8):\\n\",\n \" demo_buf.add(\\n\",\n \" Batch(\\n\",\n \" obs=i,\\n\",\n \" act=i,\\n\",\n \" rew=float(i),\\n\",\n \" terminated=False,\\n\",\n \" truncated=False,\\n\",\n \" obs_next=i + 1,\\n\",\n \" info={},\\n\",\n \" )\\n\",\n \" )\\n\",\n \"print(f\\\"Length: {len(demo_buf)}, Max: {demo_buf.maxsize}\\\")\\n\",\n \"print(f\\\"Observations: {demo_buf.obs[: len(demo_buf)]}\\\")\\n\",\n \"print(\\\"\\\\nNotice: First 3 transitions (0,1,2) were overwritten by (3,4,5)\\\")\\n\",\n \"print(\\\"Buffer now contains: [3, 4, 5, 6, 7]\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 3.4 Batch-Compatible Operations\\n\",\n \"\\n\",\n \"Since ReplayBuffer extends Batch functionality, it supports standard indexing and slicing:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Indexing and slicing\\n\",\n \"print(\\\"Last transition:\\\")\\n\",\n \"print(demo_buf[-1])\\n\",\n \"\\n\",\n \"print(\\\"\\\\nLast 3 transitions:\\\")\\n\",\n \"print(demo_buf[-3:])\\n\",\n \"\\n\",\n \"print(\\\"\\\\nSpecific indices [0, 2, 4]:\\\")\\n\",\n \"print(demo_buf[np.array([0, 2, 4])])\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## 4. Trajectory Management\\n\",\n \"\\n\",\n \"A key distinguishing feature of ReplayBuffer is its automatic tracking of episode boundaries and metadata.\\n\",\n \"\\n\",\n \"### 4.1 Episode Tracking and Metadata\\n\",\n \"\\n\",\n \"The `add()` method returns four values that provide episode information:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Create a fresh buffer for trajectory demonstration\\n\",\n \"traj_buf = ReplayBuffer(size=20)\\n\",\n \"\\n\",\n \"print(\\\"Episode 1: 4 steps, terminates naturally\\\")\\n\",\n \"for i in range(4):\\n\",\n \" idx, ep_rew, ep_len, ep_start = traj_buf.add(\\n\",\n \" Batch(\\n\",\n \" obs=i,\\n\",\n \" act=i,\\n\",\n \" rew=float(i + 1), # Rewards: 1, 2, 3, 4\\n\",\n \" terminated=i == 3, # Last step terminates\\n\",\n \" truncated=False,\\n\",\n \" obs_next=i + 1,\\n\",\n \" info={},\\n\",\n \" )\\n\",\n \" )\\n\",\n \" print(f\\\" Step {i}: idx={idx}, ep_rew={ep_rew}, ep_len={ep_len}, ep_start={ep_start}\\\")\\n\",\n \"\\n\",\n \"print(\\\"\\\\nNotice: Episode return (10.0) and length (4) only appear at the end!\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"**Return Values Explained**:\\n\",\n \"\\n\",\n \"1. `idx`: Index where the transition was inserted (np.ndarray of shape (1,))\\n\",\n \"2. `ep_rew`: Episode return, only non-zero when `done=True` (np.ndarray of shape (1,))\\n\",\n \"3. `ep_len`: Episode length, only non-zero when `done=True` (np.ndarray of shape (1,))\\n\",\n \"4. `ep_start`: Index where the episode started (np.ndarray of shape (1,))\\n\",\n \"\\n\",\n \"This automatic computation eliminates manual episode tracking during data collection.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Continue with Episode 2: 5 steps\\n\",\n \"print(\\\"Episode 2: 5 steps, truncated (time limit)\\\")\\n\",\n \"for i in range(4, 9):\\n\",\n \" idx, ep_rew, ep_len, ep_start = traj_buf.add(\\n\",\n \" Batch(\\n\",\n \" obs=i,\\n\",\n \" act=i,\\n\",\n \" rew=float(i + 1),\\n\",\n \" terminated=False,\\n\",\n \" truncated=i == 8, # Last step truncated\\n\",\n \" obs_next=i + 1,\\n\",\n \" info={},\\n\",\n \" )\\n\",\n \" )\\n\",\n \" if i == 8:\\n\",\n \" print(\\n\",\n \" f\\\" Final step: idx={idx}, ep_rew={ep_rew[0]:.1f}, ep_len={ep_len[0]}, ep_start={ep_start}\\\"\\n\",\n \" )\\n\",\n \"\\n\",\n \"# Episode 3: Ongoing (not finished)\\n\",\n \"print(\\\"\\\\nEpisode 3: 3 steps, ongoing (not done)\\\")\\n\",\n \"for i in range(9, 12):\\n\",\n \" idx, ep_rew, ep_len, ep_start = traj_buf.add(\\n\",\n \" Batch(\\n\",\n \" obs=i,\\n\",\n \" act=i,\\n\",\n \" rew=float(i + 1),\\n\",\n \" terminated=False,\\n\",\n \" truncated=False, # Episode continues\\n\",\n \" obs_next=i + 1,\\n\",\n \" info={},\\n\",\n \" )\\n\",\n \" )\\n\",\n \" if i == 11:\\n\",\n \" print(\\n\",\n \" f\\\" Latest step: idx={idx}, ep_rew={ep_rew}, ep_len={ep_len} (zeros because not done)\\\"\\n\",\n \" )\\n\",\n \"\\n\",\n \"print(f\\\"\\\\nBuffer state: {len(traj_buf)} transitions across 2 complete + 1 ongoing episode\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 4.2 Boundary Navigation: prev() and next()\\n\",\n \"\\n\",\n \"The buffer provides methods to navigate within episodes while respecting episode boundaries:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Examine the buffer structure\\n\",\n \"print(\\\"Buffer contents:\\\")\\n\",\n \"print(f\\\"Indices: {np.arange(len(traj_buf))}\\\")\\n\",\n \"print(f\\\"Obs: {traj_buf.obs[: len(traj_buf)]}\\\")\\n\",\n \"print(f\\\"Terminated: {traj_buf.terminated[: len(traj_buf)]}\\\")\\n\",\n \"print(f\\\"Truncated: {traj_buf.truncated[: len(traj_buf)]}\\\")\\n\",\n \"print(f\\\"Done: {traj_buf.done[: len(traj_buf)]}\\\")\\n\",\n \"print(\\\"\\\\nEpisode boundaries: indices 3 (terminated) and 8 (truncated)\\\")\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# prev() returns the previous index within the same episode\\n\",\n \"# It STOPS at episode boundaries\\n\",\n \"test_indices = np.array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11])\\n\",\n \"prev_indices = traj_buf.prev(test_indices)\\n\",\n \"\\n\",\n \"print(\\\"prev() behavior:\\\")\\n\",\n \"print(f\\\"Index: {test_indices}\\\")\\n\",\n \"print(f\\\"Prev: {prev_indices}\\\")\\n\",\n \"print(\\\"\\\\nObservations:\\\")\\n\",\n \"print(\\\"- Index 0 stays at 0 (start of episode 1)\\\")\\n\",\n \"print(\\\"- Index 4 stays at 4 (start of episode 2, can't go back to episode 1)\\\")\\n\",\n \"print(\\\"- Index 9 stays at 9 (start of episode 3, can't go back to episode 2)\\\")\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# next() returns the next index within the same episode\\n\",\n \"# It STOPS at episode boundaries\\n\",\n \"next_indices = traj_buf.next(test_indices)\\n\",\n \"\\n\",\n \"print(\\\"next() behavior:\\\")\\n\",\n \"print(f\\\"Index: {test_indices}\\\")\\n\",\n \"print(f\\\"Next: {next_indices}\\\")\\n\",\n \"print(\\\"\\\\nObservations:\\\")\\n\",\n \"print(\\\"- Index 3 stays at 3 (end of episode 1, terminated)\\\")\\n\",\n \"print(\\\"- Index 8 stays at 8 (end of episode 2, truncated)\\\")\\n\",\n \"print(\\\"- Indices 9-11 advance normally (episode 3 ongoing)\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"**Use Cases for prev() and next()**:\\n\",\n \"\\n\",\n \"These methods are essential for computing algorithmic quantities:\\n\",\n \"- **N-step returns**: Use prev() to look back N steps within an episode\\n\",\n \"- **GAE (Generalized Advantage Estimation)**: Navigate backwards through episodes\\n\",\n \"- **Episode extraction**: Find episode start/end indices\\n\",\n \"- **Temporal difference targets**: Ensure you don't bootstrap across episode boundaries\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 4.3 Identifying Unfinished Episodes\\n\",\n \"\\n\",\n \"The `unfinished_index()` method returns indices of ongoing episodes:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"unfinished = traj_buf.unfinished_index()\\n\",\n \"print(f\\\"Unfinished episode indices: {unfinished}\\\")\\n\",\n \"print(f\\\"Latest step of ongoing episode: obs={traj_buf.obs[unfinished[0]]}\\\")\\n\",\n \"\\n\",\n \"# After finishing episode 3\\n\",\n \"traj_buf.add(\\n\",\n \" Batch(\\n\",\n \" obs=12,\\n\",\n \" act=12,\\n\",\n \" rew=13.0,\\n\",\n \" terminated=True,\\n\",\n \" truncated=False,\\n\",\n \" obs_next=13,\\n\",\n \" info={},\\n\",\n \" )\\n\",\n \")\\n\",\n \"\\n\",\n \"unfinished_after = traj_buf.unfinished_index()\\n\",\n \"print(\\\"\\\\nAfter finishing episode 3:\\\")\\n\",\n \"print(f\\\"Unfinished episodes: {unfinished_after} (empty array)\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## 5. Sampling Strategies\\n\",\n \"\\n\",\n \"Efficient sampling is critical for RL training. The buffer provides several sampling methods and strategies.\\n\",\n \"\\n\",\n \"### 5.1 Basic Sampling\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Create a buffer with some data\\n\",\n \"sample_buf = ReplayBuffer(size=100)\\n\",\n \"for i in range(50):\\n\",\n \" sample_buf.add(\\n\",\n \" Batch(\\n\",\n \" obs=i,\\n\",\n \" act=i % 4,\\n\",\n \" rew=np.random.random(),\\n\",\n \" terminated=(i + 1) % 10 == 0,\\n\",\n \" truncated=False,\\n\",\n \" obs_next=i + 1,\\n\",\n \" info={},\\n\",\n \" )\\n\",\n \" )\\n\",\n \"\\n\",\n \"# Sample with batch_size\\n\",\n \"batch, indices = sample_buf.sample(batch_size=8)\\n\",\n \"print(f\\\"Sampled batch size: {len(batch)}\\\")\\n\",\n \"print(f\\\"Sampled indices: {indices}\\\")\\n\",\n \"print(f\\\"Sampled observations: {batch.obs}\\\")\\n\",\n \"\\n\",\n \"# batch_size=None: return all data in random order\\n\",\n \"all_data, all_indices = sample_buf.sample(batch_size=None)\\n\",\n \"print(f\\\"\\\\nSample all (batch_size=None): {len(all_data)} transitions\\\")\\n\",\n \"\\n\",\n \"# batch_size=0: return all data in buffer order\\n\",\n \"ordered_data, ordered_indices = sample_buf.sample(batch_size=0)\\n\",\n \"print(f\\\"Get all in order (batch_size=0): {len(ordered_data)} transitions\\\")\\n\",\n \"print(f\\\"Indices in order: {ordered_indices[:10]}...\\\") # Show first 10\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"**Sampling Behavior Summary**:\\n\",\n \"\\n\",\n \"- `batch_size > 0`: Random sample of specified size\\n\",\n \"- `batch_size = None`: All data in random order \\n\",\n \"- `batch_size = 0`: All data in insertion order\\n\",\n \"- `batch_size < 0`: Empty array (edge case handling)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 5.2 Frame Stacking\\n\",\n \"\\n\",\n \"The `stack_num` parameter enables automatic frame stacking, useful for RNN inputs or Atari-style environments where temporal context matters:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Create buffer with frame stacking\\n\",\n \"stack_buf = ReplayBuffer(size=20, stack_num=4)\\n\",\n \"\\n\",\n \"# Add observations: 0, 1, 2, ..., 9\\n\",\n \"for i in range(10):\\n\",\n \" stack_buf.add(\\n\",\n \" Batch(\\n\",\n \" obs=np.array([i]), # Single frame\\n\",\n \" act=0,\\n\",\n \" rew=1.0,\\n\",\n \" terminated=i == 9,\\n\",\n \" truncated=False,\\n\",\n \" obs_next=np.array([i + 1]),\\n\",\n \" info={},\\n\",\n \" )\\n\",\n \" )\\n\",\n \"\\n\",\n \"# Get stacked frames for index 6\\n\",\n \"# Should return [3, 4, 5, 6] (4 consecutive frames ending at 6)\\n\",\n \"stacked = stack_buf.get(index=6, key=\\\"obs\\\")\\n\",\n \"print(\\\"Frame stacking demo:\\\")\\n\",\n \"print(\\\"Requested index: 6\\\")\\n\",\n \"print(f\\\"Stacked frames shape: {stacked.shape}\\\")\\n\",\n \"print(f\\\"Stacked frames: {stacked.flatten()}\\\")\\n\",\n \"print(\\\"\\\\nExplanation: stack_num=4, so index 6 returns [obs[3], obs[4], obs[5], obs[6]]\\\")\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Demonstrate episode boundary handling with frame stacking\\n\",\n \"boundary_buf = ReplayBuffer(size=20, stack_num=4)\\n\",\n \"\\n\",\n \"# Episode 1: indices 0-4\\n\",\n \"for i in range(5):\\n\",\n \" boundary_buf.add(\\n\",\n \" Batch(\\n\",\n \" obs=np.array([i]),\\n\",\n \" act=0,\\n\",\n \" rew=1.0,\\n\",\n \" terminated=i == 4,\\n\",\n \" truncated=False,\\n\",\n \" obs_next=np.array([i + 1]),\\n\",\n \" info={},\\n\",\n \" )\\n\",\n \" )\\n\",\n \"\\n\",\n \"# Episode 2: indices 5-9\\n\",\n \"for i in range(5, 10):\\n\",\n \" boundary_buf.add(\\n\",\n \" Batch(\\n\",\n \" obs=np.array([i]),\\n\",\n \" act=0,\\n\",\n \" rew=1.0,\\n\",\n \" terminated=i == 9,\\n\",\n \" truncated=False,\\n\",\n \" obs_next=np.array([i + 1]),\\n\",\n \" info={},\\n\",\n \" )\\n\",\n \" )\\n\",\n \"\\n\",\n \"# Try to get stacked frames at episode boundary\\n\",\n \"boundary_stack = boundary_buf.get(index=6, key=\\\"obs\\\") # Early in episode 2\\n\",\n \"print(\\\"\\\\nFrame stacking at episode boundary:\\\")\\n\",\n \"print(f\\\"Index 6 stacked frames: {boundary_stack.flatten()}\\\")\\n\",\n \"print(\\\"Notice: Frames don't cross episode boundary (5,5,5,6 not 3,4,5,6)\\\")\\n\",\n \"print(\\\"The buffer uses prev() internally, which respects episode boundaries\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"**Frame Stacking Use Cases**:\\n\",\n \"\\n\",\n \"- **RNN/LSTM inputs**: Provide temporal context to recurrent networks\\n\",\n \"- **Atari games**: Stack 4 frames to capture motion (as in DQN paper)\\n\",\n \"- **Velocity estimation**: Multiple frames allow computing derivatives\\n\",\n \"- **Partially observable environments**: Build up state estimates\\n\",\n \"\\n\",\n \"**Important Notes**:\\n\",\n \"- Frame stacking respects episode boundaries (won't stack across episodes)\\n\",\n \"- Set `sample_avail=True` to only sample indices where full stacks are available\\n\",\n \"- `save_only_last_obs=True` saves memory in Atari-style setups\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## 6. VectorReplayBuffer: Parallel Environment Support\\n\",\n \"\\n\",\n \"VectorReplayBuffer is essential for modern RL training with parallel environments. It maintains separate subbuffers for each environment while providing a unified interface.\\n\",\n \"\\n\",\n \"### 6.1 Motivation and Architecture\\n\",\n \"\\n\",\n \"When training with multiple parallel environments (e.g., 8 environments running simultaneously), we need:\\n\",\n \"- **Per-environment episode tracking**: Each environment has its own episode boundaries\\n\",\n \"- **Temporal ordering**: Preserve the sequence of events within each environment\\n\",\n \"- **Unified sampling**: Sample uniformly across all environments for training\\n\",\n \"\\n\",\n \"```mermaid\\n\",\n \"graph LR\\n\",\n \" E1[Env 1] --> B1[Subbuffer 1
2500 capacity]\\n\",\n \" E2[Env 2] --> B2[Subbuffer 2
2500 capacity]\\n\",\n \" E3[Env 3] --> B3[Subbuffer 3
2500 capacity]\\n\",\n \" E4[Env 4] --> B4[Subbuffer 4
2500 capacity]\\n\",\n \" \\n\",\n \" B1 --> VRB[VectorReplayBuffer
Total: 10000
Unified Sampling]\\n\",\n \" B2 --> VRB\\n\",\n \" B3 --> VRB\\n\",\n \" B4 --> VRB\\n\",\n \" \\n\",\n \" VRB --> Policy[Policy Training]\\n\",\n \" \\n\",\n \" style E1 fill:#e1f5ff\\n\",\n \" style E2 fill:#e1f5ff\\n\",\n \" style E3 fill:#e1f5ff\\n\",\n \" style E4 fill:#e1f5ff\\n\",\n \" style B1 fill:#fff4e1\\n\",\n \" style B2 fill:#fff4e1\\n\",\n \" style B3 fill:#fff4e1\\n\",\n \" style B4 fill:#fff4e1\\n\",\n \" style VRB fill:#ffe1f5\\n\",\n \" style Policy fill:#e8f5e1\\n\",\n \"```\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Create VectorReplayBuffer for 4 parallel environments\\n\",\n \"vec_buf = VectorReplayBuffer(\\n\",\n \" total_size=100, # Total capacity across all subbuffers\\n\",\n \" buffer_num=4, # Number of parallel environments\\n\",\n \")\\n\",\n \"\\n\",\n \"print(\\\"VectorReplayBuffer created:\\\")\\n\",\n \"print(f\\\"Total size: {vec_buf.maxsize}\\\")\\n\",\n \"print(f\\\"Number of subbuffers: {vec_buf.buffer_num}\\\")\\n\",\n \"print(f\\\"Size per subbuffer: {vec_buf.maxsize // vec_buf.buffer_num}\\\")\\n\",\n \"print(f\\\"Subbuffer edges: {vec_buf.subbuffer_edges}\\\")\\n\",\n \"print(\\\"\\\\nSubbuffer edges define the boundary indices: [0, 25, 50, 75, 100]\\\")\\n\",\n \"print(\\\"Subbuffer 0: indices 0-24, Subbuffer 1: indices 25-49, etc.\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 6.2 The buffer_ids Parameter\\n\",\n \"\\n\",\n \"This is one of the most confusing aspects for new users. The `buffer_ids` parameter specifies which subbuffer each transition belongs to.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Simulate data from 4 parallel environments\\n\",\n \"# Each environment produces one transition\\n\",\n \"parallel_batch = Batch(\\n\",\n \" obs=np.array([[0.1, 0.2], [1.1, 1.2], [2.1, 2.2], [3.1, 3.2]]), # 4 observations\\n\",\n \" act=np.array([0, 1, 0, 1]), # 4 actions\\n\",\n \" rew=np.array([1.0, 2.0, 3.0, 4.0]), # 4 rewards\\n\",\n \" terminated=np.array([False, False, False, False]),\\n\",\n \" truncated=np.array([False, False, False, False]),\\n\",\n \" obs_next=np.array([[0.2, 0.3], [1.2, 1.3], [2.2, 2.3], [3.2, 3.3]]),\\n\",\n \" info=np.array([{}, {}, {}, {}], dtype=object),\\n\",\n \")\\n\",\n \"\\n\",\n \"print(\\\"Parallel batch shape:\\\", parallel_batch.obs.shape)\\n\",\n \"print(\\\"This represents 4 transitions, one from each environment\\\")\\n\",\n \"\\n\",\n \"# Add with buffer_ids specifying which subbuffer each transition goes to\\n\",\n \"indices, ep_rews, ep_lens, ep_starts = vec_buf.add(\\n\",\n \" parallel_batch,\\n\",\n \" buffer_ids=[0, 1, 2, 3], # Transition 0→Subbuf 0, 1→Subbuf 1, etc.\\n\",\n \")\\n\",\n \"\\n\",\n \"print(f\\\"\\\\nAdded to indices: {indices}\\\")\\n\",\n \"print(\\\"Notice: Indices are in different subbuffers:\\\")\\n\",\n \"print(f\\\" Index {indices[0]} in subbuffer 0 (range 0-24)\\\")\\n\",\n \"print(f\\\" Index {indices[1]} in subbuffer 1 (range 25-49)\\\")\\n\",\n \"print(f\\\" Index {indices[2]} in subbuffer 2 (range 50-74)\\\")\\n\",\n \"print(f\\\" Index {indices[3]} in subbuffer 3 (range 75-99)\\\")\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Add more data to demonstrate buffer_ids\\n\",\n \"# Environments don't always produce data in order 0,1,2,3\\n\",\n \"# For example, if only environments 1 and 3 are ready:\\n\",\n \"partial_batch = Batch(\\n\",\n \" obs=np.array([[1.2, 1.3], [3.2, 3.3]]), # Only 2 observations\\n\",\n \" act=np.array([0, 1]),\\n\",\n \" rew=np.array([2.5, 4.5]),\\n\",\n \" terminated=np.array([False, False]),\\n\",\n \" truncated=np.array([False, False]),\\n\",\n \" obs_next=np.array([[1.3, 1.4], [3.3, 3.4]]),\\n\",\n \" info=np.array([{}, {}], dtype=object),\\n\",\n \")\\n\",\n \"\\n\",\n \"# Only environments 1 and 3 produced data\\n\",\n \"indices2, _, _, _ = vec_buf.add(\\n\",\n \" partial_batch,\\n\",\n \" buffer_ids=[1, 3], # Only these two subbuffers receive data\\n\",\n \")\\n\",\n \"\\n\",\n \"print(\\\"Added partial batch (only envs 1 and 3):\\\")\\n\",\n \"print(f\\\"Indices: {indices2}\\\")\\n\",\n \"print(f\\\"Subbuffer 1 received data at index {indices2[0]}\\\")\\n\",\n \"print(f\\\"Subbuffer 3 received data at index {indices2[1]}\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"**Important: buffer_ids Requirements**:\\n\",\n \"\\n\",\n \"For `VectorReplayBuffer`:\\n\",\n \"- `buffer_ids` length must match batch size\\n\",\n \"- Values must be in range [0, buffer_num)\\n\",\n \"- Can be partial (not all environments at once)\\n\",\n \"\\n\",\n \"For regular `ReplayBuffer`:\\n\",\n \"- If `buffer_ids` is not None, it must be [0]\\n\",\n \"- Batch must have shape (1, data_length)\\n\",\n \"- This is for API compatibility with VectorReplayBuffer\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 6.3 Subbuffer Edges and Episode Handling\\n\",\n \"\\n\",\n \"Subbuffer edges prevent episodes from spanning across subbuffers, ensuring data from different environments doesn't get mixed:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# The subbuffer_edges property defines boundaries\\n\",\n \"print(f\\\"Subbuffer edges: {vec_buf.subbuffer_edges}\\\")\\n\",\n \"print(\\\"\\\\nThis creates 4 subbuffers:\\\")\\n\",\n \"for i in range(vec_buf.buffer_num):\\n\",\n \" start = vec_buf.subbuffer_edges[i]\\n\",\n \" end = vec_buf.subbuffer_edges[i + 1]\\n\",\n \" print(f\\\"Subbuffer {i}: indices [{start}, {end})\\\")\\n\",\n \"\\n\",\n \"# Episodes cannot cross these boundaries\\n\",\n \"# prev() and next() respect subbuffer edges just like episode boundaries\\n\",\n \"test_idx = np.array([24, 25, 49, 50]) # At subbuffer edges\\n\",\n \"prev_result = vec_buf.prev(test_idx)\\n\",\n \"next_result = vec_buf.next(test_idx)\\n\",\n \"\\n\",\n \"print(\\\"\\\\nBoundary navigation test:\\\")\\n\",\n \"print(f\\\"Indices: {test_idx}\\\")\\n\",\n \"print(f\\\"prev(): {prev_result}\\\")\\n\",\n \"print(f\\\"next(): {next_result}\\\")\\n\",\n \"print(\\\"\\\\nNotice: prev/next don't cross subbuffer boundaries\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 6.4 Sampling from VectorReplayBuffer\\n\",\n \"\\n\",\n \"Sampling is uniform across all subbuffers (proportional to their current fill level):\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Add more data to have enough for sampling\\n\",\n \"for _step in range(10):\\n\",\n \" batch = Batch(\\n\",\n \" obs=np.random.randn(4, 2),\\n\",\n \" act=np.random.randint(0, 2, size=4),\\n\",\n \" rew=np.random.random(4),\\n\",\n \" terminated=np.zeros(4, dtype=bool),\\n\",\n \" truncated=np.zeros(4, dtype=bool),\\n\",\n \" obs_next=np.random.randn(4, 2),\\n\",\n \" info=np.array([{}] * 4, dtype=object),\\n\",\n \" )\\n\",\n \" vec_buf.add(batch, buffer_ids=[0, 1, 2, 3])\\n\",\n \"\\n\",\n \"# Sample batch\\n\",\n \"sampled, indices = vec_buf.sample(batch_size=16)\\n\",\n \"print(f\\\"Sampled {len(sampled)} transitions\\\")\\n\",\n \"print(f\\\"Sample indices (from different subbuffers): {indices}\\\")\\n\",\n \"print(\\\"\\\\nNotice indices span across all subbuffer ranges\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## 7. Specialized Buffer Variants\\n\",\n \"\\n\",\n \"### 7.1 PrioritizedReplayBuffer\\n\",\n \"\\n\",\n \"Implements prioritized experience replay where transitions are sampled based on their TD-error magnitudes:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": \"# Create prioritized buffer\\nprio_buf = PrioritizedReplayBuffer(\\n size=100,\\n alpha=0.6, # Prioritization exponent (0=uniform, 1=fully prioritized)\\n beta=0.4, # Importance sampling correction (annealed to 1)\\n)\\n\\n# Add some transitions\\nfor i in range(20):\\n prio_buf.add(\\n Batch(\\n obs=np.array([i]),\\n act=i % 4,\\n rew=np.random.random(),\\n terminated=False,\\n truncated=False,\\n obs_next=np.array([i + 1]),\\n info={},\\n )\\n )\\n\\n# Sample returns batch and indices\\n# Importance weights are INSIDE the batch as batch.weight\\nbatch, indices = prio_buf.sample(batch_size=8)\\nprint(f\\\"Sampled batch size: {len(batch)}\\\")\\nprint(f\\\"Indices: {indices}\\\")\\nprint(f\\\"Importance weights (batch.weight): {batch.weight}\\\")\\nprint(\\\"\\\\nWeights are stored in batch.weight and compensate for biased sampling\\\")\"\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# After computing TD-errors from the sampled batch, update priorities\\n\",\n \"# In practice, these would be actual TD-errors: |Q(s,a) - (r + γ*max Q(s',a'))|\\n\",\n \"fake_td_errors = np.random.random(len(indices)) * 10 # Simulated TD-errors\\n\",\n \"\\n\",\n \"# Update priorities (higher TD-error = higher priority)\\n\",\n \"prio_buf.update_weight(indices, fake_td_errors)\\n\",\n \"\\n\",\n \"print(\\\"Updated priorities based on TD-errors\\\")\\n\",\n \"print(\\\"Transitions with higher TD-errors will be sampled more frequently\\\")\\n\",\n \"\\n\",\n \"# Demonstrate beta annealing\\n\",\n \"prio_buf.set_beta(0.6) # Increase beta over training\\n\",\n \"print(f\\\"\\\\nAnnealed beta to: {prio_buf.options['beta']}\\\")\\n\",\n \"print(\\\"Beta typically starts at 0.4 and anneals to 1.0 over training\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"**PrioritizedReplayBuffer Use Cases**:\\n\",\n \"- Rainbow DQN and variants\\n\",\n \"- Any algorithm where some transitions are more \\\"surprising\\\" and valuable\\n\",\n \"- Environments with rare but important events\\n\",\n \"\\n\",\n \"**Key Parameters**:\\n\",\n \"- `alpha`: Controls how much prioritization affects sampling (0=uniform, 1=fully proportional to priority)\\n\",\n \"- `beta`: Importance sampling correction to remain unbiased (anneal from ~0.4 to 1.0)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 7.2 Other Specialized Buffers\\n\",\n \"\\n\",\n \"**CachedReplayBuffer**: Maintains a primary buffer plus auxiliary caches\\n\",\n \"- Use case: Imitation learning where you want separate expert and agent buffers\\n\",\n \"- Example: GAIL (Generative Adversarial Imitation Learning)\\n\",\n \"- Allows different sampling ratios from different sources\\n\",\n \"\\n\",\n \"**HERReplayBuffer**: Hindsight Experience Replay for goal-conditioned tasks\\n\",\n \"- Use case: Sparse reward robotics tasks\\n\",\n \"- Relabels failed episodes with achieved goals as if they were intended\\n\",\n \"- Dramatically improves learning in goal-reaching tasks\\n\",\n \"- See the HER documentation for detailed examples\\n\",\n \"\\n\",\n \"For detailed usage of these specialized buffers, refer to the Tianshou API documentation and algorithm-specific tutorials.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## 8. Serialization and Persistence\\n\",\n \"\\n\",\n \"Buffers support multiple serialization formats for saving and loading data.\\n\",\n \"\\n\",\n \"### 8.1 Pickle Serialization\\n\",\n \"\\n\",\n \"The simplest method, preserving all buffer state including trajectory metadata:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Create and populate a buffer\\n\",\n \"save_buf = ReplayBuffer(size=50)\\n\",\n \"for i in range(30):\\n\",\n \" save_buf.add(\\n\",\n \" Batch(\\n\",\n \" obs=np.array([i, i + 1]),\\n\",\n \" act=i % 4,\\n\",\n \" rew=float(i),\\n\",\n \" terminated=(i + 1) % 10 == 0,\\n\",\n \" truncated=False,\\n\",\n \" obs_next=np.array([i + 1, i + 2]),\\n\",\n \" info={\\\"step\\\": i},\\n\",\n \" )\\n\",\n \" )\\n\",\n \"\\n\",\n \"print(f\\\"Original buffer: {len(save_buf)} transitions\\\")\\n\",\n \"\\n\",\n \"# Serialize with pickle\\n\",\n \"pickled_data = pickle.dumps(save_buf)\\n\",\n \"print(f\\\"Serialized size: {len(pickled_data)} bytes\\\")\\n\",\n \"\\n\",\n \"# Deserialize\\n\",\n \"loaded_buf = pickle.loads(pickled_data)\\n\",\n \"print(f\\\"Loaded buffer: {len(loaded_buf)} transitions\\\")\\n\",\n \"print(f\\\"Data preserved: obs[0] = {loaded_buf.obs[0]}\\\")\\n\",\n \"print(f\\\"Metadata preserved: info[0] = {loaded_buf.info[0]}\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 8.2 HDF5 Serialization\\n\",\n \"\\n\",\n \"HDF5 is recommended for large datasets and cross-platform compatibility:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Save to HDF5\\n\",\n \"with tempfile.NamedTemporaryFile(suffix=\\\".hdf5\\\", delete=False) as tmp:\\n\",\n \" hdf5_path = tmp.name\\n\",\n \"\\n\",\n \"save_buf.save_hdf5(hdf5_path, compression=\\\"gzip\\\")\\n\",\n \"print(f\\\"Saved to HDF5: {hdf5_path}\\\")\\n\",\n \"\\n\",\n \"# Load from HDF5\\n\",\n \"loaded_hdf5_buf = ReplayBuffer.load_hdf5(hdf5_path)\\n\",\n \"print(f\\\"Loaded from HDF5: {len(loaded_hdf5_buf)} transitions\\\")\\n\",\n \"print(f\\\"Data matches: {np.array_equal(save_buf.obs, loaded_hdf5_buf.obs)}\\\")\\n\",\n \"\\n\",\n \"# Clean up\\n\",\n \"import os\\n\",\n \"\\n\",\n \"os.unlink(hdf5_path)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"**When to Use HDF5**:\\n\",\n \"- Large datasets (> 1GB)\\n\",\n \"- Offline RL with pre-collected data\\n\",\n \"- Sharing data across platforms\\n\",\n \"- Need for compression\\n\",\n \"- Integration with external tools (many scientific tools read HDF5)\\n\",\n \"\\n\",\n \"**When to Use Pickle**:\\n\",\n \"- Quick saves during development\\n\",\n \"- Small buffers\\n\",\n \"- Python-only workflow\\n\",\n \"- Simpler serialization needs\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 8.3 Loading from Raw Data with from_data()\\n\",\n \"\\n\",\n \"For offline RL, you can create a buffer from raw arrays:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Simulate pre-collected offline dataset\\n\",\n \"import h5py\\n\",\n \"\\n\",\n \"# Create temporary HDF5 file with raw data\\n\",\n \"with tempfile.NamedTemporaryFile(suffix=\\\".hdf5\\\", delete=False) as tmp:\\n\",\n \" offline_path = tmp.name\\n\",\n \"\\n\",\n \"with h5py.File(offline_path, \\\"w\\\") as f:\\n\",\n \" # Create datasets\\n\",\n \" n = 100\\n\",\n \" f.create_dataset(\\\"obs\\\", data=np.random.randn(n, 4))\\n\",\n \" f.create_dataset(\\\"act\\\", data=np.random.randint(0, 2, n))\\n\",\n \" f.create_dataset(\\\"rew\\\", data=np.random.randn(n))\\n\",\n \" f.create_dataset(\\\"terminated\\\", data=np.random.random(n) < 0.1)\\n\",\n \" f.create_dataset(\\\"truncated\\\", data=np.zeros(n, dtype=bool))\\n\",\n \" f.create_dataset(\\\"done\\\", data=np.random.random(n) < 0.1)\\n\",\n \" f.create_dataset(\\\"obs_next\\\", data=np.random.randn(n, 4))\\n\",\n \"\\n\",\n \"# Load into buffer\\n\",\n \"with h5py.File(offline_path, \\\"r\\\") as f:\\n\",\n \" offline_buf = ReplayBuffer.from_data(\\n\",\n \" obs=f[\\\"obs\\\"],\\n\",\n \" act=f[\\\"act\\\"],\\n\",\n \" rew=f[\\\"rew\\\"],\\n\",\n \" terminated=f[\\\"terminated\\\"],\\n\",\n \" truncated=f[\\\"truncated\\\"],\\n\",\n \" done=f[\\\"done\\\"],\\n\",\n \" obs_next=f[\\\"obs_next\\\"],\\n\",\n \" )\\n\",\n \"\\n\",\n \"print(f\\\"Loaded offline dataset: {len(offline_buf)} transitions\\\")\\n\",\n \"print(f\\\"Observation shape: {offline_buf.obs.shape}\\\")\\n\",\n \"\\n\",\n \"# Clean up\\n\",\n \"os.unlink(offline_path)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"This is the standard approach for offline RL where you have pre-collected datasets from other sources.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## 9. Integration with the RL Pipeline\\n\",\n \"\\n\",\n \"Understanding how buffers integrate with other Tianshou components is essential for effective usage.\\n\",\n \"\\n\",\n \"### 9.1 Data Flow in RL Training\\n\",\n \"\\n\",\n \"```mermaid\\n\",\n \"graph LR\\n\",\n \" ENV[Vectorized
Environments] -->|observations| COL[Collector]\\n\",\n \" POL[Policy] -->|actions| COL\\n\",\n \" COL -->|transitions| BUF[Buffer]\\n\",\n \" BUF -->|sampled batches| POL\\n\",\n \" POL -->|forward pass| ALG[Algorithm]\\n\",\n \" ALG -->|loss & gradients| POL\\n\",\n \" \\n\",\n \" style ENV fill:#e1f5ff\\n\",\n \" style COL fill:#fff4e1\\n\",\n \" style BUF fill:#ffe1f5\\n\",\n \" style POL fill:#e8f5e1\\n\",\n \" style ALG fill:#f5e1e1\\n\",\n \"```\\n\",\n \"\\n\",\n \"### 9.2 Typical Training Loop Pattern\\n\",\n \"\\n\",\n \"Here's how buffers are typically used in a training loop:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Pseudocode for typical RL training loop\\n\",\n \"# (This is illustrative; actual implementation would use Trainer)\\n\",\n \"\\n\",\n \"\\n\",\n \"def training_loop_pseudocode():\\n\",\n \" \\\"\\\"\\\"\\n\",\n \" Illustrative training loop showing buffer integration.\\n\",\n \"\\n\",\n \" In practice, use Tianshou's Trainer class which handles this.\\n\",\n \" \\\"\\\"\\\"\\n\",\n \" # Setup (illustration only)\\n\",\n \" # env = make_vectorized_env(num_envs=8)\\n\",\n \" # policy = make_policy()\\n\",\n \" # buffer = VectorReplayBuffer(total_size=100000, buffer_num=8)\\n\",\n \" # collector = Collector(policy, env, buffer)\\n\",\n \"\\n\",\n \" # Training loop\\n\",\n \" # for epoch in range(num_epochs):\\n\",\n \" # # 1. Collect data from environments\\n\",\n \" # collect_result = collector.collect(n_step=1000)\\n\",\n \" # # Collector automatically adds transitions to buffer with correct buffer_ids\\n\",\n \" #\\n\",\n \" # # 2. Train on multiple batches\\n\",\n \" # for _ in range(update_per_collect):\\n\",\n \" # # Sample batch from buffer\\n\",\n \" # batch, indices = buffer.sample(batch_size=256)\\n\",\n \" #\\n\",\n \" # # Compute loss and update policy\\n\",\n \" # loss = policy.learn(batch)\\n\",\n \" #\\n\",\n \" # # For prioritized buffers, update priorities\\n\",\n \" # # if isinstance(buffer, PrioritizedReplayBuffer):\\n\",\n \" # # buffer.update_weight(indices, td_errors)\\n\",\n \"\\n\",\n \" print(\\\"This pseudocode illustrates the buffer's role:\\\")\\n\",\n \" print(\\\"1. Collector fills buffer from environment interaction\\\")\\n\",\n \" print(\\\"2. Buffer provides random samples for training\\\")\\n\",\n \" print(\\\"3. Policy learns from sampled batches\\\")\\n\",\n \" print(\\\"\\\\nIn practice, use Tianshou's Trainer for this workflow\\\")\\n\",\n \"\\n\",\n \"\\n\",\n \"training_loop_pseudocode()\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 9.3 Collector Integration\\n\",\n \"\\n\",\n \"The Collector class handles the complexity of:\\n\",\n \"- Calling policy to get actions\\n\",\n \"- Stepping environments\\n\",\n \"- Adding transitions to buffer with correct buffer_ids\\n\",\n \"- Tracking episode statistics\\n\",\n \"\\n\",\n \"When you create a Collector, you pass it a buffer, and it automatically:\\n\",\n \"- Uses VectorReplayBuffer for vectorized environments\\n\",\n \"- Sets buffer_ids based on which environments are ready\\n\",\n \"- Handles episode resets and boundary tracking\\n\",\n \"\\n\",\n \"See the Collector tutorial for detailed examples of this integration.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## 10. Advanced Topics and Edge Cases\\n\",\n \"\\n\",\n \"### 10.1 Buffer Overflow and Episode Boundaries\\n\",\n \"\\n\",\n \"What happens when the buffer fills up mid-episode?\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Small buffer to demonstrate overflow\\n\",\n \"overflow_buf = ReplayBuffer(size=8)\\n\",\n \"\\n\",\n \"# Add a long episode (12 steps, buffer size is only 8)\\n\",\n \"print(\\\"Adding 12-step episode to buffer with size 8:\\\")\\n\",\n \"for i in range(12):\\n\",\n \" idx, ep_rew, ep_len, ep_start = overflow_buf.add(\\n\",\n \" Batch(\\n\",\n \" obs=i,\\n\",\n \" act=0,\\n\",\n \" rew=1.0,\\n\",\n \" terminated=i == 11,\\n\",\n \" truncated=False,\\n\",\n \" obs_next=i + 1,\\n\",\n \" info={},\\n\",\n \" )\\n\",\n \" )\\n\",\n \" if i in [7, 11]:\\n\",\n \" print(f\\\" Step {i}: idx={idx}, buffer_len={len(overflow_buf)}\\\")\\n\",\n \"\\n\",\n \"print(\\\"\\\\nFinal buffer contents (most recent 8 steps):\\\")\\n\",\n \"print(f\\\"Observations: {overflow_buf.obs[: len(overflow_buf)]}\\\")\\n\",\n \"print(f\\\"Episode return: {ep_rew[0]} (sum of all 12 steps, tracked correctly!)\\\")\\n\",\n \"print(\\\"\\\\nNote: Buffer overwrote old data but episode statistics are still correct\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"**Important**: Episode returns and lengths are tracked internally and remain correct even when the episode spans buffer overflows. The buffer maintains `_ep_return`, `_ep_len`, and `_ep_start_idx` to track ongoing episodes.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 10.2 Episode Spanning Subbuffer Edges\\n\",\n \"\\n\",\n \"In VectorReplayBuffer, episodes can wrap around within their subbuffer:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Create small VectorReplayBuffer to demonstrate edge crossing\\n\",\n \"edge_buf = VectorReplayBuffer(total_size=20, buffer_num=2) # 10 per subbuffer\\n\",\n \"\\n\",\n \"print(f\\\"Subbuffer edges: {edge_buf.subbuffer_edges}\\\")\\n\",\n \"print(\\\"Subbuffer 0: indices 0-9, Subbuffer 1: indices 10-19\\\\n\\\")\\n\",\n \"\\n\",\n \"# Fill subbuffer 0 with 12 steps (wraps around since capacity is 10)\\n\",\n \"for i in range(12):\\n\",\n \" batch = Batch(\\n\",\n \" obs=np.array([[i]]),\\n\",\n \" act=np.array([0]),\\n\",\n \" rew=np.array([1.0]),\\n\",\n \" terminated=np.array([i == 11]),\\n\",\n \" truncated=np.array([False]),\\n\",\n \" obs_next=np.array([[i + 1]]),\\n\",\n \" info=np.array([{}], dtype=object),\\n\",\n \" )\\n\",\n \" idx, _, _, _ = edge_buf.add(batch, buffer_ids=[0])\\n\",\n \" if i >= 10:\\n\",\n \" print(f\\\"Step {i} added at index {idx[0]} (wrapped around in subbuffer 0)\\\")\\n\",\n \"\\n\",\n \"# get_buffer_indices handles this correctly\\n\",\n \"episode_indices = edge_buf.get_buffer_indices(start=8, stop=2) # Crosses edge\\n\",\n \"print(f\\\"\\\\nEpisode spanning edge (from 8 to 1): {episode_indices}\\\")\\n\",\n \"print(\\\"Correctly retrieves [8, 9, 0, 1] within subbuffer 0\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 10.3 ignore_obs_next Memory Optimization\\n\",\n \"\\n\",\n \"For memory-constrained scenarios, you can avoid storing obs_next:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Buffer that doesn't store obs_next\\n\",\n \"memory_buf = ReplayBuffer(size=10, ignore_obs_next=True)\\n\",\n \"\\n\",\n \"# Add transitions (obs_next is ignored)\\n\",\n \"for i in range(5):\\n\",\n \" memory_buf.add(\\n\",\n \" Batch(\\n\",\n \" obs=np.array([i, i + 1]),\\n\",\n \" act=i,\\n\",\n \" rew=1.0,\\n\",\n \" terminated=False,\\n\",\n \" truncated=False,\\n\",\n \" obs_next=np.array([i + 1, i + 2]), # Provided but not stored\\n\",\n \" info={},\\n\",\n \" )\\n\",\n \" )\\n\",\n \"\\n\",\n \"# When sampling, obs_next is reconstructed from next obs\\n\",\n \"sample, _ = memory_buf.sample(batch_size=1)\\n\",\n \"print(f\\\"Sampled obs: {sample.obs}\\\")\\n\",\n \"print(f\\\"Sampled obs_next: {sample.obs_next}\\\")\\n\",\n \"print(\\\"\\\\nobs_next was reconstructed, not stored directly\\\")\\n\",\n \"print(\\\"This saves memory at the cost of slightly more complex retrieval\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"This is particularly useful for Atari environments with large observation spaces (84x84x4 frames).\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## 11. Surprising Behaviors and Gotchas\\n\",\n \"\\n\",\n \"### 11.1 Most Common Mistake: buffer_ids Confusion\\n\",\n \"\\n\",\n \"The buffer_ids parameter is the most common source of errors:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# COMMON ERROR 1: Forgetting buffer_ids with VectorReplayBuffer\\n\",\n \"vec_demo = VectorReplayBuffer(total_size=100, buffer_num=4)\\n\",\n \"\\n\",\n \"parallel_data = Batch(\\n\",\n \" obs=np.random.randn(4, 2),\\n\",\n \" act=np.array([0, 1, 0, 1]),\\n\",\n \" rew=np.array([1.0, 2.0, 3.0, 4.0]),\\n\",\n \" terminated=np.array([False, False, False, False]),\\n\",\n \" truncated=np.array([False, False, False, False]),\\n\",\n \" obs_next=np.random.randn(4, 2),\\n\",\n \" info=np.array([{}, {}, {}, {}], dtype=object),\\n\",\n \")\\n\",\n \"\\n\",\n \"# WRONG: Omitting buffer_ids (defaults to [0,1,2,3] which is OK here)\\n\",\n \"# But if you have partial data, this will fail\\n\",\n \"vec_demo.add(parallel_data) # Works by default\\n\",\n \"\\n\",\n \"# CORRECT: Always explicit\\n\",\n \"vec_demo.add(parallel_data, buffer_ids=[0, 1, 2, 3])\\n\",\n \"print(\\\"Always specify buffer_ids explicitly for clarity\\\")\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# COMMON ERROR 2: Shape mismatch with buffer_ids\\n\",\n \"try:\\n\",\n \" # Trying to add 2 transitions but specifying 4 buffer_ids\\n\",\n \" wrong_batch = Batch(\\n\",\n \" obs=np.random.randn(2, 2), # Only 2 transitions!\\n\",\n \" act=np.array([0, 1]),\\n\",\n \" rew=np.array([1.0, 2.0]),\\n\",\n \" terminated=np.array([False, False]),\\n\",\n \" truncated=np.array([False, False]),\\n\",\n \" obs_next=np.random.randn(2, 2),\\n\",\n \" info=np.array([{}, {}], dtype=object),\\n\",\n \" )\\n\",\n \" vec_demo.add(wrong_batch, buffer_ids=[0, 1, 2, 3]) # MISMATCH!\\n\",\n \"except (IndexError, ValueError) as e:\\n\",\n \" print(f\\\"Error caught: {type(e).__name__}\\\")\\n\",\n \" print(\\\"Lesson: buffer_ids length must match batch size\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 11.2 Done Flag Confusion\\n\",\n \"\\n\",\n \"Never manually set the `done` flag:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# WRONG: Manually setting done\\n\",\n \"wrong_batch = Batch(\\n\",\n \" obs=1,\\n\",\n \" act=0,\\n\",\n \" rew=1.0,\\n\",\n \" terminated=True,\\n\",\n \" truncated=False,\\n\",\n \" # done=True, # DON'T DO THIS! It will be overwritten anyway\\n\",\n \" obs_next=2,\\n\",\n \" info={},\\n\",\n \")\\n\",\n \"\\n\",\n \"# CORRECT: Only set terminated and truncated\\n\",\n \"# done is automatically computed as (terminated OR truncated)\\n\",\n \"correct_batch = Batch(\\n\",\n \" obs=1,\\n\",\n \" act=0,\\n\",\n \" rew=1.0,\\n\",\n \" terminated=True, # Episode ended naturally\\n\",\n \" truncated=False, # Not cut off\\n\",\n \" obs_next=2,\\n\",\n \" info={},\\n\",\n \")\\n\",\n \"\\n\",\n \"demo = ReplayBuffer(size=10)\\n\",\n \"demo.add(correct_batch)\\n\",\n \"print(f\\\"Terminated: {demo.terminated[0]}\\\")\\n\",\n \"print(f\\\"Truncated: {demo.truncated[0]}\\\")\\n\",\n \"print(f\\\"Done (auto-computed): {demo.done[0]}\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 11.3 Sampling from Empty or Near-Empty Buffers\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Edge case: Sampling more than available\\n\",\n \"small_buf = ReplayBuffer(size=100)\\n\",\n \"for i in range(5): # Only 5 transitions\\n\",\n \" small_buf.add(\\n\",\n \" Batch(obs=i, act=0, rew=1.0, terminated=False, truncated=False, obs_next=i + 1, info={})\\n\",\n \" )\\n\",\n \"\\n\",\n \"# Request 20 but only 5 available - samples with replacement\\n\",\n \"batch, indices = small_buf.sample(batch_size=20)\\n\",\n \"print(f\\\"Requested 20, buffer has {len(small_buf)}, got {len(batch)}\\\")\\n\",\n \"print(f\\\"Indices: {indices}\\\")\\n\",\n \"print(\\\"Notice: Some indices repeat (sampling with replacement)\\\")\\n\",\n \"\\n\",\n \"# Defensive pattern: Check buffer size\\n\",\n \"if len(small_buf) >= 128:\\n\",\n \" batch, _ = small_buf.sample(128)\\n\",\n \"else:\\n\",\n \" print(f\\\"Buffer has {len(small_buf)} < 128, waiting for more data\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 11.4 Frame Stacking Valid Indices\\n\",\n \"\\n\",\n \"With stack_num > 1, not all indices are valid for sampling:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# With frame stacking, early indices can't form complete stacks\\n\",\n \"stack_demo = ReplayBuffer(size=20, stack_num=4, sample_avail=True)\\n\",\n \"\\n\",\n \"for i in range(10):\\n\",\n \" stack_demo.add(\\n\",\n \" Batch(\\n\",\n \" obs=np.array([i]),\\n\",\n \" act=0,\\n\",\n \" rew=1.0,\\n\",\n \" terminated=i == 9,\\n\",\n \" truncated=False,\\n\",\n \" obs_next=np.array([i + 1]),\\n\",\n \" info={},\\n\",\n \" )\\n\",\n \" )\\n\",\n \"\\n\",\n \"# With sample_avail=True, only valid indices are sampled\\n\",\n \"sampled, indices = stack_demo.sample(batch_size=5)\\n\",\n \"print(f\\\"Sampled indices with stack_num=4, sample_avail=True: {indices}\\\")\\n\",\n \"print(\\\"All indices >= 3 (can form complete 4-frame stacks)\\\")\\n\",\n \"\\n\",\n \"# Without sample_avail, any index can be sampled (may have incomplete stacks)\\n\",\n \"stack_demo2 = ReplayBuffer(size=20, stack_num=4, sample_avail=False)\\n\",\n \"for i in range(10):\\n\",\n \" stack_demo2.add(\\n\",\n \" Batch(\\n\",\n \" obs=np.array([i]),\\n\",\n \" act=0,\\n\",\n \" rew=1.0,\\n\",\n \" terminated=False,\\n\",\n \" truncated=False,\\n\",\n \" obs_next=np.array([i + 1]),\\n\",\n \" info={},\\n\",\n \" )\\n\",\n \" )\\n\",\n \"\\n\",\n \"sampled2, indices2 = stack_demo2.sample(batch_size=5)\\n\",\n \"print(f\\\"\\\\nSampled indices with sample_avail=False: {indices2}\\\")\\n\",\n \"print(\\\"May include indices < 3 (incomplete stacks repeated from boundary)\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## 12. Best Practices\\n\",\n \"\\n\",\n \"### 12.1 Choosing the Right Buffer\\n\",\n \"\\n\",\n \"**Decision Tree**:\\n\",\n \"\\n\",\n \"1. Are you using parallel environments?\\n\",\n \" - Yes → Use `VectorReplayBuffer`\\n\",\n \" - No → Continue to 2\\n\",\n \"\\n\",\n \"2. Do you need prioritized experience replay?\\n\",\n \" - Yes → Use `PrioritizedReplayBuffer` or `PrioritizedVectorReplayBuffer`\\n\",\n \" - No → Continue to 3\\n\",\n \"\\n\",\n \"3. Is it goal-conditioned RL with sparse rewards?\\n\",\n \" - Yes → Use `HERReplayBuffer` or `HERVectorReplayBuffer`\\n\",\n \" - No → Continue to 4\\n\",\n \"\\n\",\n \"4. Do you need separate expert and agent buffers?\\n\",\n \" - Yes → Use `CachedReplayBuffer`\\n\",\n \" - No → Use `ReplayBuffer` (single env) or `VectorReplayBuffer` (standard choice)\\n\",\n \"\\n\",\n \"**Most Common Setup**: `VectorReplayBuffer` for production training\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 12.2 Buffer Sizing Guidelines\\n\",\n \"\\n\",\n \"**Rule of Thumb by Domain**:\\n\",\n \"\\n\",\n \"- **Atari games**: 1,000,000 transitions (1e6)\\n\",\n \"- **Continuous control (MuJoCo)**: 100,000-1,000,000 (1e5-1e6)\\n\",\n \"- **Robotics**: 100,000-500,000 (1e5-5e5)\\n\",\n \"- **Simple environments (CartPole)**: 10,000-50,000 (1e4-5e4)\\n\",\n \"\\n\",\n \"**Factors to Consider**:\\n\",\n \"- Available RAM (each transition ~observation_size * 2 + metadata)\\n\",\n \"- Training time vs sample efficiency tradeoff\\n\",\n \"- Algorithm requirements (some need larger buffers)\\n\",\n \"\\n\",\n \"**Memory Estimation**:\\n\",\n \"```python\\n\",\n \"# For environments with observation shape (84, 84, 4) (Atari):\\n\",\n \"# Each transition: 2 * 84 * 84 * 4 bytes (obs + obs_next) + ~100 bytes overhead\\n\",\n \"# = ~56KB per transition\\n\",\n \"# 1M transitions = ~56GB (use ignore_obs_next to halve this!)\\n\",\n \"```\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 12.3 Configuration Best Practices\\n\",\n \"\\n\",\n \"**When to use stack_num > 1**:\\n\",\n \"- RNN/LSTM policies need temporal context\\n\",\n \"- Frame-based policies (Atari with 4-frame stacking)\\n\",\n \"- Velocity estimation from positions\\n\",\n \"\\n\",\n \"**When to use ignore_obs_next=True**:\\n\",\n \"- Memory-constrained environments\\n\",\n \"- Atari (large observation spaces)\\n\",\n \"- When obs_next can be reconstructed from next obs\\n\",\n \"\\n\",\n \"**When to use save_only_last_obs=True**:\\n\",\n \"- Atari with temporal stacking in environment wrapper\\n\",\n \"- When observations already contain frame history\\n\",\n \"\\n\",\n \"**When to use sample_avail=True**:\\n\",\n \"- Always use with stack_num > 1 for correctness\\n\",\n \"- Ensures samples have complete frame stacks\\n\",\n \"- Small performance cost but worth it for data quality\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 12.4 Integration Patterns\\n\",\n \"\\n\",\n \"**Pattern 1: Standard Off-Policy Setup**\\n\",\n \"```python\\n\",\n \"# env = make_vectorized_env(num_envs=8)\\n\",\n \"# buffer = VectorReplayBuffer(total_size=100000, buffer_num=8)\\n\",\n \"# policy = SACPolicy(...)\\n\",\n \"# collector = Collector(policy, env, buffer)\\n\",\n \"# \\n\",\n \"# # Collect and train\\n\",\n \"# collector.collect(n_step=1000)\\n\",\n \"# for _ in range(10):\\n\",\n \"# batch, indices = buffer.sample(256)\\n\",\n \"# policy.learn(batch)\\n\",\n \"```\\n\",\n \"\\n\",\n \"**Pattern 2: Pre-fill Buffer Before Training**\\n\",\n \"```python\\n\",\n \"# # Collect random exploration data\\n\",\n \"# collector.collect(n_step=10000) # Fill buffer\\n\",\n \"# \\n\",\n \"# # Then start training\\n\",\n \"# while not converged:\\n\",\n \"# collector.collect(n_step=100)\\n\",\n \"# for _ in range(10):\\n\",\n \"# batch = buffer.sample(256)\\n\",\n \"# policy.learn(batch)\\n\",\n \"```\\n\",\n \"\\n\",\n \"**Pattern 3: Offline RL**\\n\",\n \"```python\\n\",\n \"# # Load pre-collected dataset\\n\",\n \"# buffer = ReplayBuffer.load_hdf5(\\\"expert_data.hdf5\\\")\\n\",\n \"# \\n\",\n \"# # Train without further collection\\n\",\n \"# for epoch in range(num_epochs):\\n\",\n \"# for _ in range(updates_per_epoch):\\n\",\n \"# batch = buffer.sample(256)\\n\",\n \"# policy.learn(batch)\\n\",\n \"```\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### 12.5 Performance Tips\\n\",\n \"\\n\",\n \"**Tip 1: Pre-allocate buffer size appropriately**\\n\",\n \"- Don't make buffer too large (wastes memory)\\n\",\n \"- Don't make it too small (loses important old experiences)\\n\",\n \"- Start with domain defaults and adjust based on performance\\n\",\n \"\\n\",\n \"**Tip 2: Use HDF5 for large offline datasets**\\n\",\n \"- Compression saves disk space\\n\",\n \"- Faster loading than pickle for large files\\n\",\n \"- Better for sharing across systems\\n\",\n \"\\n\",\n \"**Tip 3: Batch sampling efficiently**\\n\",\n \"- Sample once and use multiple times if possible\\n\",\n \"- Don't sample more than you need\\n\",\n \"- For multi-GPU training, sample once and split\\n\",\n \"\\n\",\n \"**Tip 4: Monitor buffer usage**\\n\",\n \"```python\\n\",\n \"# print(f\\\"Buffer usage: {len(buffer)}/{buffer.maxsize}\\\")\\n\",\n \"# if len(buffer) < batch_size:\\n\",\n \"# print(\\\"Warning: Sampling with replacement!\\\")\\n\",\n \"```\\n\",\n \"\\n\",\n \"**Tip 5: Consider ignore_obs_next for large observation spaces**\\n\",\n \"- Can halve memory usage\\n\",\n \"- Small computational overhead on sampling\\n\",\n \"- Especially valuable for image-based RL\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": \"## 13. Quick Reference\\n\\n### Method Summary\\n\\n| Method | Purpose | Returns | Notes |\\n|--------|---------|---------|-------|\\n| `add(batch, buffer_ids)` | Add transition(s) | `(idx, ep_rew, ep_len, ep_start)` | ep_rew/ep_len only non-zero when done=True |\\n| `sample(size)` | Random sample | `(batch, indices)` | size=None for all (random), 0 for all (ordered) |\\n| `prev(idx)` | Previous in episode | `indices` | Stops at episode boundaries |\\n| `next(idx)` | Next in episode | `indices` | Stops at episode boundaries |\\n| `get(idx, key, stack_num)` | Get with stacking | `data` | Returns stacked frames if stack_num > 1 |\\n| `get_buffer_indices(start, stop)` | Episode range | `indices` | Handles edge-crossing episodes |\\n| `unfinished_index()` | Ongoing episodes | `indices` | Returns last step of unfinished episodes |\\n| `save_hdf5(path)` | Save to HDF5 | - | Recommended for large datasets |\\n| `load_hdf5(path)` | Load from HDF5 | `buffer` | Class method |\\n| `from_data(...)` | Create from arrays | `buffer` | For offline RL datasets |\\n| `reset()` | Clear buffer | - | Optionally keep episode statistics |\\n| `sample_indices(size)` | Get indices only | `indices` | For custom sampling logic |\\n\\n### Common Patterns Cheatsheet\\n\\n**Single Environment**:\\n```python\\nbuffer = ReplayBuffer(size=10000)\\nbuffer.add(Batch(obs=..., act=..., rew=..., terminated=..., truncated=..., obs_next=..., info={}))\\nbatch, indices = buffer.sample(batch_size=256)\\n```\\n\\n**Parallel Environments**:\\n```python\\nbuffer = VectorReplayBuffer(total_size=100000, buffer_num=8)\\nbuffer.add(parallel_batch, buffer_ids=[0,1,2,3,4,5,6,7])\\nbatch, indices = buffer.sample(batch_size=256)\\n```\\n\\n**Frame Stacking**:\\n```python\\nbuffer = ReplayBuffer(size=100000, stack_num=4, sample_avail=True)\\nstacked_obs = buffer.get(index=50, key=\\\"obs\\\") # Returns 4 stacked frames\\n```\\n\\n**Prioritized Replay**:\\n```python\\nbuffer = PrioritizedReplayBuffer(size=100000, alpha=0.6, beta=0.4)\\nbatch, indices = buffer.sample(batch_size=256)\\nweights = batch.weight # Importance weights are inside the batch\\n# ... compute TD errors ...\\nbuffer.update_weight(indices, td_errors)\\n```\\n\\n**Offline RL**:\\n```python\\nbuffer = ReplayBuffer.load_hdf5(\\\"dataset.hdf5\\\")\\n# Or:\\nwith h5py.File(\\\"dataset.hdf5\\\", \\\"r\\\") as f:\\n buffer = ReplayBuffer.from_data(obs=f[\\\"obs\\\"], act=f[\\\"act\\\"], ...)\\n```\\n\\n**Episode Retrieval**:\\n```python\\n# Find episode boundaries, then:\\nepisode_indices = buffer.get_buffer_indices(start=ep_start_idx, stop=ep_end_idx+1)\\nepisode = buffer[episode_indices]\\n```\"\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Summary and Next Steps\\n\",\n \"\\n\",\n \"This tutorial covered Tianshou's buffer system comprehensively:\\n\",\n \"\\n\",\n \"1. **Buffer fundamentals**: Why buffers are essential for RL\\n\",\n \"2. **Buffer hierarchy**: Understanding different buffer types\\n\",\n \"3. **Basic operations**: Construction, configuration, and data management\\n\",\n \"4. **Trajectory management**: Episode tracking and boundary navigation\\n\",\n \"5. **Sampling strategies**: Basic sampling and frame stacking\\n\",\n \"6. **VectorReplayBuffer**: Critical for parallel environments\\n\",\n \"7. **Specialized buffers**: Prioritized, cached, and HER variants\\n\",\n \"8. **Serialization**: Pickle and HDF5 persistence\\n\",\n \"9. **Integration**: How buffers fit in the RL pipeline\\n\",\n \"10. **Advanced topics**: Edge cases and overflow handling\\n\",\n \"11. **Gotchas**: Common mistakes and how to avoid them\\n\",\n \"12. **Best practices**: Configuration, sizing, and performance\\n\",\n \"13. **Quick reference**: Method summary and common patterns\\n\",\n \"\\n\",\n \"### Next Steps\\n\",\n \"\\n\",\n \"- **Collector Deep Dive**: Learn how Collector fills buffers from environments\\n\",\n \"- **Policy Tutorial**: Understand how policies sample from buffers for training\\n\",\n \"- **Algorithm Examples**: See buffer usage in specific algorithms (DQN, SAC, PPO)\\n\",\n \"- **API Reference**: Full details at [Buffer API documentation](https://tianshou.org/en/stable/api/tianshou.data.html)\\n\",\n \"\\n\",\n \"### Further Resources\\n\",\n \"\\n\",\n \"- [Tianshou GitHub](https://github.com/thu-ml/tianshou) for source code and examples\\n\",\n \"- [Gymnasium Documentation](https://gymnasium.farama.org/) for environment conventions\\n\",\n \"- Research papers on experience replay and prioritized sampling\"\n ]\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.11.0\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 4\n}\n" }, { "path": "docs/02_deep_dives/L3_Environments.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Environments\\n\",\n \"\\n\",\n \"In reinforcement learning, agents interact with environments to improve their performance through trial and error. This tutorial explores how Tianshou handles environments, from basic single-environment setups to advanced vectorized and parallel configurations.\\n\",\n \"\\n\",\n \"
\\n\",\n \"
\\n\",\n \"The agent-environment interaction loop\\n\",\n \"
\\n\",\n \"\\n\",\n \"Tianshou maintains full compatibility with the [Gymnasium](https://gymnasium.farama.org/) API (formerly OpenAI Gym), making it easy to use any Gymnasium-compatible environment.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## The Bottleneck Problem\\n\",\n \"\\n\",\n \"In a standard Gymnasium environment, each interaction follows a sequential pattern:\\n\",\n \"\\n\",\n \"1. Agent selects an action\\n\",\n \"2. Environment processes the action and returns observation and reward\\n\",\n \"3. Repeat\\n\",\n \"\\n\",\n \"This sequential process can become a significant bottleneck in deep reinforcement learning experiments, especially when:\\n\",\n \"- The environment simulation is computationally intensive\\n\",\n \"- Network training is fast but data collection is slow\\n\",\n \"- You have multiple CPU cores available but aren't using them\\n\",\n \"\\n\",\n \"Tianshou addresses this bottleneck through **vectorized environments**, which allow parallel sampling across multiple CPU cores.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Vectorized Environments\\n\",\n \"\\n\",\n \"Vectorized environments enable you to run multiple environment instances in parallel, dramatically accelerating data collection. Let's see this in action.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"metadata\": {},\n \"source\": [\n \"import time\\n\",\n \"\\n\",\n \"import gymnasium as gym\\n\",\n \"import numpy as np\\n\",\n \"\\n\",\n \"from tianshou.env import DummyVectorEnv, SubprocVectorEnv\"\n ],\n \"outputs\": [],\n \"execution_count\": null\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Performance Comparison\\n\",\n \"\\n\",\n \"Let's compare the sampling speed with different numbers of parallel environments:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"metadata\": {},\n \"source\": [\n \"num_cpus = [1, 2, 5]\\n\",\n \"\\n\",\n \"for num_cpu in num_cpus:\\n\",\n \" # Create vectorized environment with multiple processes\\n\",\n \" env = SubprocVectorEnv([lambda: gym.make(\\\"CartPole-v1\\\") for _ in range(num_cpu)])\\n\",\n \" env.reset()\\n\",\n \"\\n\",\n \" sampled_steps = 0\\n\",\n \" time_start = time.time()\\n\",\n \"\\n\",\n \" # Sample 1000 steps\\n\",\n \" while sampled_steps < 1000:\\n\",\n \" act = np.random.choice(2, size=num_cpu)\\n\",\n \" obs, rew, terminated, truncated, info = env.step(act)\\n\",\n \"\\n\",\n \" # Reset terminated environments\\n\",\n \" if np.sum(terminated):\\n\",\n \" env.reset(np.where(terminated)[0])\\n\",\n \"\\n\",\n \" sampled_steps += num_cpu\\n\",\n \"\\n\",\n \" time_used = time.time() - time_start\\n\",\n \" print(f\\\"Sampled 1000 steps in {time_used:.3f}s using {num_cpu} CPU(s)\\\")\\n\",\n \" print(f\\\" → Speed: {1000 / time_used:.1f} steps/second\\\")\"\n ],\n \"outputs\": [],\n \"execution_count\": null\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Understanding the Results\\n\",\n \"\\n\",\n \"You might notice that the speedup isn't perfectly linear with the number of CPUs. Several factors contribute to this:\\n\",\n \"\\n\",\n \"1. **Straggler Effect**: In synchronous mode, all environments must complete before the next batch begins. Slower environments hold back faster ones.\\n\",\n \"2. **Communication Overhead**: Inter-process communication has costs, especially for fast environments.\\n\",\n \"3. **Environment Complexity**: For simple environments like CartPole, the overhead may outweigh the benefits.\\n\",\n \"\\n\",\n \"> **Important**: `SubprocVectorEnv` should only be used when environment execution is slow. For simple, fast environments like CartPole, `DummyVectorEnv` (or even raw Gymnasium environments) can be more efficient because they avoid both the straggler effect and inter-process communication overhead.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Types of Vectorized Environments\\n\",\n \"\\n\",\n \"Tianshou provides several vectorized environment implementations, each optimized for different scenarios:\\n\",\n \"\\n\",\n \"### 1. DummyVectorEnv\\n\",\n \"**Pseudo-parallel simulation using a for-loop**\\n\",\n \"- Best for: Simple/fast environments, debugging\\n\",\n \"- Pros: No overhead, deterministic execution\\n\",\n \"- Cons: No actual parallelization\\n\",\n \"\\n\",\n \"### 2. SubprocVectorEnv\\n\",\n \"**Multiple processes for true parallel simulation**\\n\",\n \"- Best for: Most parallel simulation scenarios\\n\",\n \"- Pros: True parallelization, good balance\\n\",\n \"- Cons: Inter-process communication overhead\\n\",\n \"\\n\",\n \"### 3. ShmemVectorEnv\\n\",\n \"**Shared memory optimization of SubprocVectorEnv**\\n\",\n \"- Best for: Environments with large observations (e.g., images)\\n\",\n \"- Pros: Reduced memory footprint, faster for large states\\n\",\n \"- Cons: More complex implementation\\n\",\n \"\\n\",\n \"### 4. RayVectorEnv\\n\",\n \"**Ray-based distributed simulation**\\n\",\n \"- Best for: Cluster computing with multiple machines\\n\",\n \"- Pros: Scales to multiple machines\\n\",\n \"- Cons: Requires Ray installation and setup\\n\",\n \"\\n\",\n \"All these classes share the same API through their base class `BaseVectorEnv`, making it easy to switch between them.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Basic Usage\\n\",\n \"\\n\",\n \"### Creating a Vectorized Environment\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"metadata\": {},\n \"source\": [\n \"# Standard Gymnasium environment\\n\",\n \"gym_env = gym.make(\\\"CartPole-v1\\\")\\n\",\n \"\\n\",\n \"\\n\",\n \"# Tianshou vectorized environment\\n\",\n \"def create_cartpole_env() -> gym.Env:\\n\",\n \" return gym.make(\\\"CartPole-v1\\\")\\n\",\n \"\\n\",\n \"\\n\",\n \"# Create 5 parallel environments\\n\",\n \"vector_env = DummyVectorEnv([create_cartpole_env for _ in range(5)])\\n\",\n \"\\n\",\n \"print(f\\\"Created vectorized environment with {vector_env.env_num} environments\\\")\"\n ],\n \"outputs\": [],\n \"execution_count\": null\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Environment Interaction\\n\",\n \"\\n\",\n \"The key difference from standard Gymnasium is that actions, observations, and rewards are all vectorized:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"metadata\": {},\n \"source\": [\n \"# Standard Gymnasium: reset() returns a single observation\\n\",\n \"print(\\\"Standard Gymnasium reset:\\\")\\n\",\n \"single_obs, info = gym_env.reset()\\n\",\n \"print(f\\\" Shape: {single_obs.shape}\\\")\\n\",\n \"print(f\\\" Value: {single_obs}\\\")\\n\",\n \"\\n\",\n \"print(\\\"\\\\n\\\" + \\\"=\\\" * 50 + \\\"\\\\n\\\")\\n\",\n \"\\n\",\n \"# Vectorized environment: reset() returns stacked observations\\n\",\n \"print(\\\"Vectorized environment reset:\\\")\\n\",\n \"vector_obs, info = vector_env.reset()\\n\",\n \"print(f\\\" Shape: {vector_obs.shape}\\\")\\n\",\n \"print(f\\\" Value:\\\\n{vector_obs}\\\")\"\n ],\n \"outputs\": [],\n \"execution_count\": null\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Taking Vectorized Steps\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"metadata\": {},\n \"source\": [\n \"# Take random actions in all environments\\n\",\n \"actions = np.random.choice(2, size=vector_env.env_num)\\n\",\n \"obs, rew, terminated, truncated, info = vector_env.step(actions)\\n\",\n \"\\n\",\n \"print(f\\\"Actions taken: {actions}\\\")\\n\",\n \"print(f\\\"Rewards received: {rew}\\\")\\n\",\n \"print(f\\\"Terminated flags: {terminated}\\\")\\n\",\n \"print(\\\"Info\\\", info)\"\n ],\n \"outputs\": [],\n \"execution_count\": null\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Selective Environment Execution\\n\",\n \"\\n\",\n \"You can interact with specific environments using the `id` parameter:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"metadata\": {},\n \"source\": [\n \"# Execute only environments 0, 1, and 3\\n\",\n \"selected_actions = np.random.choice(2, size=3)\\n\",\n \"obs, rew, terminated, truncated, info = vector_env.step(selected_actions, id=[0, 1, 3])\\n\",\n \"\\n\",\n \"print(\\\"Executed actions in environments [0, 1, 3]\\\")\\n\",\n \"print(f\\\"Received {len(rew)} results\\\")\"\n ],\n \"outputs\": [],\n \"execution_count\": null\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Parallel Sampling: Synchronous vs Asynchronous\\n\",\n \"\\n\",\n \"### Synchronous Mode (Default)\\n\",\n \"\\n\",\n \"By default, vectorized environments operate synchronously: a step completes only after **all** environments finish their step. This works well when all environments take roughly the same time per step.\\n\",\n \"\\n\",\n \"### Asynchronous Mode\\n\",\n \"\\n\",\n \"When environment step times vary significantly (e.g., 90% of steps take 1s, but 10% take 10s), asynchronous mode can help. It allows faster environments to continue without waiting for slower ones.\\n\",\n \"\\n\",\n \"
\\n\",\n \"
\\n\",\n \"Comparison of synchronous and asynchronous vectorized environments
\\n\",\n \"(Steps with the same color are processed together)\\n\",\n \"
\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Enabling Asynchronous Mode\\n\",\n \"\\n\",\n \"Use the `wait_num` or `timeout` parameters (or both):\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"metadata\": {},\n \"source\": [\n \"from functools import partial\\n\",\n \"\\n\",\n \"\\n\",\n \"# Create environments with varying step times\\n\",\n \"class SlowEnv(gym.Env):\\n\",\n \" \\\"\\\"\\\"Environment with variable step duration.\\\"\\\"\\\"\\n\",\n \"\\n\",\n \" def __init__(self, sleep_time):\\n\",\n \" self.sleep_time = sleep_time\\n\",\n \" self.observation_space = gym.spaces.Box(low=0, high=1, shape=(4,))\\n\",\n \" self.action_space = gym.spaces.Discrete(2)\\n\",\n \" super().__init__()\\n\",\n \"\\n\",\n \" def reset(self, seed=None, options=None):\\n\",\n \" super().reset(seed=seed)\\n\",\n \" return np.random.rand(4), {}\\n\",\n \"\\n\",\n \" def step(self, action):\\n\",\n \" time.sleep(self.sleep_time) # Simulate slow computation\\n\",\n \" return np.random.rand(4), 0.0, False, False, {}\\n\",\n \"\\n\",\n \"\\n\",\n \"# Create async vectorized environment\\n\",\n \"env_fns = [partial(SlowEnv, sleep_time=0.01 * i) for i in [1, 2, 3, 4]]\\n\",\n \"async_env = SubprocVectorEnv(env_fns, wait_num=3, timeout=0.1)\\n\",\n \"\\n\",\n \"print(\\\"Asynchronous environment created\\\")\\n\",\n \"print(\\\" wait_num=3: Returns after 3 environments complete\\\")\\n\",\n \"print(\\\" timeout=0.1: Or after 0.1 seconds, whichever comes first\\\")\"\n ],\n \"outputs\": [],\n \"execution_count\": null\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### How Async Parameters Work\\n\",\n \"\\n\",\n \"- **`wait_num`**: Minimum number of environments to wait for (e.g., `wait_num=3` means each step returns results from at least 3 environments)\\n\",\n \"- **`timeout`**: Maximum time to wait in seconds (acts as a dynamic `wait_num`—returns whatever is ready after timeout)\\n\",\n \"- If no environment finishes within the timeout, the system waits until at least one completes\\n\",\n \"\\n\",\n \"> **Warning**: Asynchronous collectors can cause exceptions when used as `test_collector` in trainers. Always use synchronous mode for test collectors.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## EnvPool Integration\\n\",\n \"\\n\",\n \"[EnvPool](https://github.com/sail-sg/envpool/) is a C++-based vectorized environment library that provides significant performance improvements over Python-based solutions for many of the standard environments. Tianshou fully supports EnvPool with minimal code changes.\\n\",\n \"\\n\",\n \"### Why EnvPool?\\n\",\n \"\\n\",\n \"- **Performance**: 10x-100x faster than standard vectorized environments for supported environments\\n\",\n \"- **Memory Efficient**: Optimized memory usage through shared buffers\\n\",\n \"- **Drop-in Replacement**: Nearly identical API to Tianshou's vectorized environments\\n\",\n \"\\n\",\n \"### Supported Environments\\n\",\n \"\\n\",\n \"EnvPool currently supports:\\n\",\n \"- Atari games\\n\",\n \"- MuJoCo physics simulations\\n\",\n \"- VizDoom 3D environments\\n\",\n \"- Classic control environments\\n\",\n \"- Toy text environments\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Using EnvPool\\n\",\n \"\\n\",\n \"First, install EnvPool:\\n\",\n \"\\n\",\n \"```bash\\n\",\n \"pip install envpool\\n\",\n \"```\\n\",\n \"\\n\",\n \"Then use it directly with Tianshou:\\n\",\n \"\\n\",\n \"```python\\n\",\n \"import envpool\\n\",\n \"\\n\",\n \"# Create EnvPool vectorized environment\\n\",\n \"envs = envpool.make_gymnasium(\\\"CartPole-v1\\\", num_envs=10)\\n\",\n \"\\n\",\n \"print(f\\\"Created EnvPool environment with {envs.spec.config.num_envs} environments\\\")\\n\",\n \"print(\\\"Ready to use with Tianshou collectors!\\\")\\n\",\n \"\\n\",\n \"# Use directly with Tianshou\\n\",\n \"collector = Collector(algorithm, envs, buffer)\\n\",\n \"```\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### EnvPool Examples\\n\",\n \"\\n\",\n \"For complete examples of using EnvPool with Tianshou:\\n\",\n \"- [Atari with EnvPool](https://github.com/thu-ml/tianshou/tree/master/examples/atari#envpool)\\n\",\n \"- [MuJoCo with EnvPool](https://github.com/thu-ml/tianshou/tree/master/examples/mujoco#envpool)\\n\",\n \"- [VizDoom with EnvPool](https://github.com/thu-ml/tianshou/tree/master/examples/vizdoom#envpool)\\n\",\n \"- [More EnvPool Examples](https://github.com/sail-sg/envpool/tree/master/examples/tianshou_examples)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Custom Environments and State Representations\\n\",\n \"\\n\",\n \"Tianshou works seamlessly with custom environments as long as they follow the Gymnasium API. Let's explore how to handle different state representations.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Required Gymnasium API\\n\",\n \"\\n\",\n \"Your custom environment must implement:\\n\",\n \"\\n\",\n \"```python\\n\",\n \"class MyEnv(gym.Env):\\n\",\n \" def reset(self, seed=None, options=None) -> Tuple[observation, info]:\\n\",\n \" \\\"\\\"\\\"Reset environment to initial state.\\\"\\\"\\\"\\n\",\n \" pass\\n\",\n \" \\n\",\n \" def step(self, action) -> Tuple[observation, reward, terminated, truncated, info]:\\n\",\n \" \\\"\\\"\\\"Execute one step in the environment.\\\"\\\"\\\"\\n\",\n \" pass\\n\",\n \" \\n\",\n \" def seed(self, seed: int) -> List[int]:\\n\",\n \" \\\"\\\"\\\"Set random seed.\\\"\\\"\\\"\\n\",\n \" pass\\n\",\n \" \\n\",\n \" def render(self, mode='human') -> Any:\\n\",\n \" \\\"\\\"\\\"Render the environment.\\\"\\\"\\\"\\n\",\n \" pass\\n\",\n \" \\n\",\n \" def close(self) -> None:\\n\",\n \" \\\"\\\"\\\"Clean up resources.\\\"\\\"\\\"\\n\",\n \" pass\\n\",\n \" \\n\",\n \" # Required spaces\\n\",\n \" observation_space: gym.Space\\n\",\n \" action_space: gym.Space\\n\",\n \"```\\n\",\n \"\\n\",\n \"> **Important**: Make sure your `seed()` method is implemented correctly:\\n\",\n \"> ```python\\n\",\n \"> def seed(self, seed):\\n\",\n \"> np.random.seed(seed)\\n\",\n \"> # Also seed other random generators used in your environment\\n\",\n \"> ```\\n\",\n \"> Without proper seeding, parallel environments may produce identical outputs!\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Dictionary Observations\\n\",\n \"\\n\",\n \"Many environments return observations as dictionaries rather than simple arrays. Tianshou's `Batch` class handles this elegantly.\\n\",\n \"\\n\",\n \"Example with the FetchReach environment:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"metadata\": {},\n \"source\": [\n \"from tianshou.data import Batch, ReplayBuffer\\n\",\n \"\\n\",\n \"# Example: Creating a mock observation similar to FetchReach\\n\",\n \"observation = {\\n\",\n \" \\\"observation\\\": np.array([1.34, 0.75, 0.53, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]),\\n\",\n \" \\\"achieved_goal\\\": np.array([1.34, 0.75, 0.53]),\\n\",\n \" \\\"desired_goal\\\": np.array([1.24, 0.78, 0.63]),\\n\",\n \"}\\n\",\n \"\\n\",\n \"# Store in replay buffer\\n\",\n \"buffer = ReplayBuffer(size=10)\\n\",\n \"buffer.add(Batch(obs=observation, act=0, rew=0.0, terminated=False, truncated=False))\\n\",\n \"\\n\",\n \"print(\\\"Stored observation structure:\\\")\\n\",\n \"print(buffer.obs)\"\n ],\n \"outputs\": [],\n \"execution_count\": null\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Accessing Dictionary Observations\\n\",\n \"\\n\",\n \"When sampling from the buffer, you can access nested dictionary values in multiple ways:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"metadata\": {},\n \"source\": [\n \"# Sample a batch\\n\",\n \"batch, indices = buffer.sample(batch_size=1)\\n\",\n \"\\n\",\n \"print(\\\"Batch keys:\\\", list(batch.keys()))\\n\",\n \"print(\\\"\\\\nAccessing nested observation:\\\")\\n\",\n \"\\n\",\n \"# Recommended way: access through batch first\\n\",\n \"print(\\\"batch.obs.desired_goal[0]:\\\", batch.obs.desired_goal[0])\\n\",\n \"\\n\",\n \"# Alternative ways (not recommended)\\n\",\n \"print(\\\"batch.obs[0].desired_goal:\\\", batch.obs[0].desired_goal)\\n\",\n \"print(\\\"batch[0].obs.desired_goal:\\\", batch[0].obs.desired_goal)\"\n ],\n \"outputs\": [],\n \"execution_count\": null\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Using Dictionary Observations in Networks\\n\",\n \"\\n\",\n \"When designing networks for environments with dictionary observations:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"metadata\": {},\n \"source\": [\n \"import torch\\n\",\n \"import torch.nn as nn\\n\",\n \"\\n\",\n \"\\n\",\n \"class CustomNetwork(nn.Module):\\n\",\n \" \\\"\\\"\\\"Network that processes dictionary observations.\\\"\\\"\\\"\\n\",\n \"\\n\",\n \" def __init__(self, obs_dim, goal_dim, hidden_dim, action_dim):\\n\",\n \" super().__init__()\\n\",\n \"\\n\",\n \" # Separate processing for different observation components\\n\",\n \" self.obs_encoder = nn.Linear(obs_dim, hidden_dim)\\n\",\n \" self.goal_encoder = nn.Linear(goal_dim * 2, hidden_dim) # achieved + desired\\n\",\n \"\\n\",\n \" # Combined processing\\n\",\n \" self.fc = nn.Sequential(\\n\",\n \" nn.Linear(hidden_dim * 2, hidden_dim), nn.ReLU(), nn.Linear(hidden_dim, action_dim)\\n\",\n \" )\\n\",\n \"\\n\",\n \" def forward(self, obs_batch, **kwargs):\\n\",\n \" # Extract components from the batch\\n\",\n \" observation = obs_batch.observation\\n\",\n \" achieved_goal = obs_batch.achieved_goal\\n\",\n \" desired_goal = obs_batch.desired_goal\\n\",\n \"\\n\",\n \" # Process each component\\n\",\n \" obs_feat = self.obs_encoder(observation)\\n\",\n \" goal_feat = self.goal_encoder(torch.cat([achieved_goal, desired_goal], dim=-1))\\n\",\n \"\\n\",\n \" # Combine and output\\n\",\n \" combined = torch.cat([obs_feat, goal_feat], dim=-1)\\n\",\n \" return self.fc(combined)\\n\",\n \"\\n\",\n \"\\n\",\n \"# Example usage\\n\",\n \"net = CustomNetwork(obs_dim=10, goal_dim=3, hidden_dim=64, action_dim=4)\\n\",\n \"print(\\\"Network created for dictionary observations\\\")\\n\",\n \"print(\\\" Input: observation (10D) + achieved_goal (3D) + desired_goal (3D)\\\")\\n\",\n \"print(\\\" Output: actions (4D)\\\")\"\n ],\n \"outputs\": [],\n \"execution_count\": null\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Custom Object States\\n\",\n \"\\n\",\n \"For more complex state representations (e.g., graphs, custom objects), Tianshou stores references in numpy arrays. However, you must ensure deep copies to avoid state aliasing:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"metadata\": {},\n \"source\": [\n \"import copy\\n\",\n \"\\n\",\n \"import networkx as nx\\n\",\n \"\\n\",\n \"\\n\",\n \"class GraphEnv(gym.Env):\\n\",\n \" \\\"\\\"\\\"Example environment with graph-based states.\\\"\\\"\\\"\\n\",\n \"\\n\",\n \" def __init__(self):\\n\",\n \" super().__init__()\\n\",\n \" self.graph = nx.Graph()\\n\",\n \" self.action_space = gym.spaces.Discrete(5)\\n\",\n \" self.observation_space = gym.spaces.Box(low=0, high=1, shape=(10,)) # for compatibility\\n\",\n \"\\n\",\n \" def reset(self, seed=None, options=None):\\n\",\n \" super().reset(seed=seed)\\n\",\n \" self.graph = nx.erdos_renyi_graph(10, 0.3)\\n\",\n \" # IMPORTANT: Return deep copy to avoid reference issues\\n\",\n \" return copy.deepcopy(self.graph), {}\\n\",\n \"\\n\",\n \" def step(self, action):\\n\",\n \" # Modify graph based on action\\n\",\n \" if action < 4 and len(self.graph.nodes) > 0:\\n\",\n \" nodes = list(self.graph.nodes)\\n\",\n \" if len(nodes) >= 2:\\n\",\n \" self.graph.add_edge(nodes[0], nodes[1])\\n\",\n \"\\n\",\n \" # IMPORTANT: Return deep copy\\n\",\n \" return copy.deepcopy(self.graph), 0.0, False, False, {}\\n\",\n \"\\n\",\n \"\\n\",\n \"# Test storing graph objects\\n\",\n \"graph_buffer = ReplayBuffer(size=5)\\n\",\n \"env = GraphEnv()\\n\",\n \"obs, _ = env.reset()\\n\",\n \"graph_buffer.add(Batch(obs=obs, act=0, rew=0.0, terminated=False, truncated=False))\\n\",\n \"\\n\",\n \"print(\\\"Graph objects stored in buffer:\\\")\\n\",\n \"print(graph_buffer.obs)\"\n ],\n \"outputs\": [],\n \"execution_count\": null\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"> **Important**: When using custom objects as states:\\n\",\n \"> 1. Always return `copy.deepcopy(state)` in both `reset()` and `step()`\\n\",\n \"> 2. Ensure the object is numpy-compatible: `np.array([your_object])` should not result in an empty array\\n\",\n \"> 3. The object may be stored as a shallow copy in the buffer—deep copying prevents state aliasing\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Best Practices Summary\\n\",\n \"\\n\",\n \"### Choosing the Right Environment Wrapper\\n\",\n \"\\n\",\n \"| Scenario | Recommended Wrapper | Why |\\n\",\n \"|----------|-------------------|-----|\\n\",\n \"| Simple/fast environments | `DummyVectorEnv` or raw Gym | Minimal overhead |\\n\",\n \"| Most parallel scenarios | `SubprocVectorEnv` | Good balance of speed and simplicity |\\n\",\n \"| Large observations (images) | `ShmemVectorEnv` | Optimized memory usage |\\n\",\n \"| Multi-machine clusters | `RayVectorEnv` | Distributed computing support |\\n\",\n \"| Maximum performance | EnvPool | C++-based, 10x-100x speedup |\\n\",\n \"\\n\",\n \"### Performance Tips\\n\",\n \"\\n\",\n \"1. **Profile First**: Measure whether environment or training is your bottleneck before optimizing\\n\",\n \"2. **Start Simple**: Begin with `DummyVectorEnv` for debugging, then upgrade to parallel versions\\n\",\n \"3. **Use EnvPool**: If your environment is supported, EnvPool offers the best performance\\n\",\n \"4. **Async for Variable Times**: Use asynchronous mode only when environment step times vary significantly\\n\",\n \"5. **Proper Seeding**: Always implement the `seed()` method correctly in custom environments\\n\",\n \"\\n\",\n \"### Common Pitfalls\\n\",\n \"\\n\",\n \"- ❌ Using `SubprocVectorEnv` for fast environments → Use `DummyVectorEnv` instead\\n\",\n \"- ❌ Forgetting to deep-copy custom states → States will be aliased in the buffer\\n\",\n \"- ❌ Not implementing `seed()` properly → Parallel environments produce identical results\\n\",\n \"- ❌ Using async collectors for testing → Causes exceptions in trainers\\n\",\n \"- ❌ Assuming linear speedup → Account for communication overhead and straggler effects\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Further Reading\\n\",\n \"\\n\",\n \"- **Tianshou Documentation**: [Environment API Reference](https://tianshou.org/en/master/03_api/env/venvs.html)\\n\",\n \"- **EnvPool**: [Official Documentation](https://envpool.readthedocs.io/)\\n\",\n \"- **Gymnasium**: [Environment Creation Tutorial](https://gymnasium.farama.org/tutorials/gymnasium_basics/environment_creation/)\\n\",\n \"- **Ray**: [Distributed RL with Ray](https://docs.ray.io/en/latest/rllib/index.html)\"\n ]\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3 (ipykernel)\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.11.4\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 4\n}\n" }, { "path": "docs/02_deep_dives/L4_GAE.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"id\": \"QJ5krjrcbuiA\"\n },\n \"source\": [\n \"# Generalized Advantage Estimation\\n\",\n \"\\n\",\n \"\\n\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"id\": \"UPVl5LBEWJ0t\"\n },\n \"source\": [\n \"## How to compute GAE on your own?\\n\",\n \"(Note that for this reading you need to understand the calculation of [GAE](https://arxiv.org/abs/1506.02438) advantage first)\\n\",\n \"\\n\",\n \"In terms of code implementation, perhaps the most difficult and annoying part is computing GAE advantage. Just now, we use the `self.compute_episodic_return()` method inherited from `BasePolicy` to save us from all those troubles. However, it is still important that we know the details behind this.\\n\",\n \"\\n\",\n \"To compute GAE advantage, the usage of `self.compute_episodic_return()` may go like:\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab\": {\n \"base_uri\": \"https://localhost:8080/\"\n },\n \"id\": \"D34GlVvPNz08\",\n \"outputId\": \"43a4e5df-59b5-4e4a-c61c-e69090810215\"\n },\n \"source\": [\n \"```python\\n\",\n \"batch, indices = dummy_buffer.sample(0) # 0 means sampling all the data from the buffer\\n\",\n \"returns, advantage = Algorithm.compute_episodic_return(\\n\",\n \" batch=batch,\\n\",\n \" buffer=dummy_buffer,\\n\",\n \" indices=indices,\\n\",\n \" v_s_=np.zeros(10),\\n\",\n \" v_s=np.zeros(10),\\n\",\n \" gamma=1.0,\\n\",\n \" gae_lambda=1.0,\\n\",\n \")\\n\",\n \"```\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"In the code above, we sample all the 10 data in the buffer and try to compute the GAE advantage. However, the way the returns are computed here might be a bit misleading. In fact, the last episode is unfinished, but its last step saved in the batch is treated as a terminal state, since it assumes that there are no future rewards. The episode is not terminated yet, it is truncated, so the agent could still get rewards in the future. Terminated and truncated episodes should indeed be treated differently.\\n\",\n \"The return of a step is the (discounted) sum of the future rewards from that step until the end of the episode. \\n\",\n \"\\\\begin{equation}\\n\",\n \"R_{t}=\\\\sum_{t}^{T} \\\\gamma^{t} r_{t}\\n\",\n \"\\\\end{equation}\\n\",\n \"Thus, at the last step of a terminated episode the return is equal to the reward at that state, since there are no future states.\\n\",\n \"\\\\begin{equation}\\n\",\n \"R_{T,terminated}=r_{T}\\n\",\n \"\\\\end{equation}\\n\",\n \"\\n\",\n \"However, if the episode was truncated the return at the last step is usually better represented by the estimated value of that state, which is the expected return from that state onwards.\\n\",\n \"\\\\begin{align*}\\n\",\n \"R_{T,truncated}=V^{\\\\pi}\\\\left(s_{T}\\\\right) \\\\quad & \\\\text{or} \\\\quad R_{T,truncated}=Q^{\\\\pi}(s_{T},a_{T})\\n\",\n \"\\\\end{align*}\\n\",\n \"Moreover, if the next state was also observed (but not its reward), then an even better estimate would be the reward of the last step plus the discounted value of the next state.\\n\",\n \"\\\\begin{align*}\\n\",\n \"R_{T,truncated}=r_T+\\\\gamma V^{\\\\pi}\\\\left(s_{T+1}\\\\right)\\n\",\n \"\\\\end{align*}\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"id\": \"h_5Dt6XwQLXV\"\n },\n \"source\": [\n \"\\n\",\n \"As we know, we need to estimate the value function of every observation to compute GAE advantage. So in `v_s` is the value of `batch.obs`, and in `v_s_` is the value of `batch.obs_next`. This is usually computed by:\\n\",\n \"\\n\",\n \"`v_s = critic(batch.obs)`,\\n\",\n \"\\n\",\n \"`v_s_ = critic(batch.obs_next)`,\\n\",\n \"\\n\",\n \"where both `v_s` and `v_s_` are 10 dimensional arrays and `critic` is usually a neural network.\\n\",\n \"\\n\",\n \"After we've got all those values, GAE can be computed following the equation below.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"id\": \"ooHNIICGUO19\"\n },\n \"source\": [\n \"\\\\begin{aligned}\\n\",\n \"\\\\hat{A}_{t}^{\\\\mathrm{GAE}(\\\\gamma, \\\\lambda)}: =& \\\\sum_{l=0}^{\\\\infty}(\\\\gamma \\\\lambda)^{l} \\\\delta_{t+l}^{V}\\n\",\n \"\\\\end{aligned}\\n\",\n \"\\n\",\n \"where\\n\",\n \"\\n\",\n \"\\\\begin{equation}\\n\",\n \"\\\\delta_{t}^{V} \\\\quad=-V\\\\left(s_{t}\\\\right)+r_{t}+\\\\gamma V\\\\left(s_{t+1}\\\\right)\\n\",\n \"\\\\end{equation}\\n\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"id\": \"eV6XZaouU7EV\"\n },\n \"source\": [\n \"Unfortunately, if you follow this equation, which is taken from the paper, you probably will get a slightly lower performance than you expected. There are at least 3 \\\"bugs\\\" in this equation.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"id\": \"FCxD9gNNVYbd\"\n },\n \"source\": [\n \"**First** is that Gym always returns you a `obs_next` even if this is already the last step. The value of this timestep is exactly 0 and you should not let the neural network estimate it.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab\": {\n \"base_uri\": \"https://localhost:8080/\"\n },\n \"id\": \"rNZNUNgQVvRJ\",\n \"outputId\": \"44354595-c25a-4da8-b4d8-cffa31ac4b7d\"\n },\n \"source\": [\n \"```python\\n\",\n \"# Assume v_s_ is got by calling critic(batch.obs_next)\\n\",\n \"v_s_ = np.ones(10)\\n\",\n \"v_s_ *= ~batch.done\\n\",\n \"```\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"id\": \"2EtMi18QWXTN\"\n },\n \"source\": [\n \"After the fix above, we will perhaps get a more accurate estimate.\\n\",\n \"\\n\",\n \"**Secondly**, you must know when to stop bootstrapping. Usually we stop bootstrapping when we meet a `done` flag. However, in the buffer above, the last (10th) step is not marked by done=True, because the collecting has not finished. We must know all those unfinished steps so that we know when to stop bootstrapping.\\n\",\n \"\\n\",\n \"Luckily, this can be done under the assistance of buffer because buffers in Tianshou not only store data, but also help you manage data trajectories.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab\": {\n \"base_uri\": \"https://localhost:8080/\"\n },\n \"id\": \"saluvX4JU6bC\",\n \"outputId\": \"2994d178-2f33-40a0-a6e4-067916b0b5c5\"\n },\n \"source\": [\n \"```python\\n\",\n \"unfinished_indexes = dummy_buffer.unfinished_index()\\n\",\n \"done_indexes = np.where(batch.done)[0]\\n\",\n \"stop_bootstrap_ids = np.concatenate([unfinished_indexes, done_indexes])\\n\",\n \"```\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"id\": \"qp6vVE4dYWv1\"\n },\n \"source\": [\n \"**Thirdly**, there are some special indexes which are marked by done flag, however its value for obs_next should not be zero. It is again because done does not differentiate between terminated and truncated. These steps are usually those at the last step of an episode, but this episode stops not because the agent can no longer get any rewards (value=0), but because the episode is too long so we have to truncate it. These kind of steps are always marked with `info['TimeLimit.truncated']=True` in Gym.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"id\": \"tWkqXRJfZTvV\"\n },\n \"source\": [\n \"As a result, we need to rewrite the equation above\\n\",\n \"\\n\",\n \"`v_s_ *= ~batch.done`\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"id\": \"kms-QtxKZe-M\"\n },\n \"source\": [\n \"to\\n\",\n \"\\n\",\n \"```\\n\",\n \"mask = batch.info['TimeLimit.truncated'] | (~batch.done)\\n\",\n \"v_s_ *= mask\\n\",\n \"\\n\",\n \"```\\n\",\n \"\\n\",\n \"\\n\",\n \"\\n\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"id\": \"u_aPPoKraBu6\"\n },\n \"source\": [\n \"## Summary\\n\",\n \"If you already felt bored by now, simply remember that Tianshou can help handle all these little details so that you can focus on the algorithm itself. Just call `Algorithm.compute_episodic_return()`.\\n\",\n \"\\n\",\n \"If you still feel interested, we would recommend you check Appendix C in this [paper](https://arxiv.org/abs/2107.14171v2) and implementation of `Algorithm.value_mask()` and `Algorithm.compute_episodic_return()` for details.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"id\": \"2cPnUXRBWKD9\"\n },\n \"source\": [\n \"
\\n\",\n \"\\n\",\n \"
\\n\",\n \"
\\n\",\n \"\\n\",\n \"
\"\n ]\n }\n ],\n \"metadata\": {\n \"colab\": {\n \"provenance\": []\n },\n \"kernelspec\": {\n \"display_name\": \"Python 3 (ipykernel)\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.11.7\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 4\n}\n" }, { "path": "docs/02_deep_dives/L5_Collector.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"id\": \"M98bqxdMsTXK\"\n },\n \"source\": [\n \"# Collector\\n\",\n \"\\n\",\n \"The Collector serves as the orchestration layer between the policy (agent) and the environment in Tianshou's architecture. It manages the interaction loop, persists collected experiences to a replay buffer, and computes episode-level statistics. This module is fundamental to both training data collection and policy evaluation workflows.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"id\": \"OX5cayLv4Ziu\"\n },\n \"source\": [\n \"## Core Applications\\n\",\n \"\\n\",\n \"The Collector supports two primary use cases in reinforcement learning experiments:\\n\",\n \"1. **Training**: Collecting interaction data for policy optimization\\n\",\n \"2. **Evaluation**: Assessing policy performance without learning\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"id\": \"Z6XKbj28u8Ze\"\n },\n \"source\": [\n \"### Policy Evaluation\\n\",\n \"\\n\",\n \"Periodic policy evaluation is essential in deep reinforcement learning (DRL) experiments to monitor training progress and assess generalization. The Collector provides a standardized interface for this purpose.\\n\",\n \"\\n\",\n \"**Setup**: A Collector requires two components:\\n\",\n \"1. An environment (or vectorized environment for parallelization)\\n\",\n \"2. A policy instance to evaluate\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"metadata\": {\n \"editable\": true,\n \"id\": \"w8t9ubO7u69J\",\n \"slideshow\": {\n \"slide_type\": \"\"\n },\n \"tags\": [\n \"hide-cell\",\n \"remove-output\"\n ],\n \"ExecuteTime\": {\n \"end_time\": \"2025-10-26T21:59:25.914405Z\",\n \"start_time\": \"2025-10-26T21:59:22.196044Z\"\n }\n },\n \"source\": [\n \"import gymnasium as gym\\n\",\n \"import torch\\n\",\n \"\\n\",\n \"from tianshou.algorithm.modelfree.reinforce import ProbabilisticActorPolicy\"\n ],\n \"outputs\": [],\n \"execution_count\": 1\n },\n {\n \"cell_type\": \"code\",\n \"metadata\": {\n \"ExecuteTime\": {\n \"end_time\": \"2025-10-26T21:59:30.621207Z\",\n \"start_time\": \"2025-10-26T21:59:25.922401Z\"\n }\n },\n \"source\": [\n \"from tianshou.data import Collector, CollectStats, VectorReplayBuffer\\n\",\n \"from tianshou.env import DummyVectorEnv\\n\",\n \"from tianshou.utils.net.common import Net\\n\",\n \"from tianshou.utils.net.discrete import DiscreteActor\\n\",\n \"\\n\",\n \"# Initialize single environment for configuration\\n\",\n \"env = gym.make(\\\"CartPole-v1\\\")\\n\",\n \"\\n\",\n \"# Create vectorized test environments (2 parallel environments)\\n\",\n \"test_envs = DummyVectorEnv([lambda: gym.make(\\\"CartPole-v1\\\") for _ in range(2)])\\n\",\n \"\\n\",\n \"# Configure neural network architecture\\n\",\n \"assert env.observation_space.shape is not None # for mypy\\n\",\n \"preprocess_net = Net(\\n\",\n \" state_shape=env.observation_space.shape,\\n\",\n \" hidden_sizes=[\\n\",\n \" 16,\\n\",\n \" ],\\n\",\n \")\\n\",\n \"\\n\",\n \"# Initialize discrete action actor network\\n\",\n \"assert isinstance(env.action_space, gym.spaces.Discrete) # for mypy\\n\",\n \"actor = DiscreteActor(preprocess_net=preprocess_net, action_shape=env.action_space.n)\\n\",\n \"\\n\",\n \"# Create policy with categorical action distribution\\n\",\n \"policy = ProbabilisticActorPolicy(\\n\",\n \" actor=actor,\\n\",\n \" dist_fn=torch.distributions.Categorical,\\n\",\n \" action_space=env.action_space,\\n\",\n \" action_scaling=False,\\n\",\n \")\\n\",\n \"\\n\",\n \"# Initialize collector for evaluation\\n\",\n \"test_collector = Collector[CollectStats](policy, test_envs)\"\n ],\n \"outputs\": [],\n \"execution_count\": 2\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"id\": \"wmt8vuwpzQdR\"\n },\n \"source\": [\n \"### Evaluating Untrained Policy Performance\\n\",\n \"\\n\",\n \"We now evaluate the randomly initialized policy across 9 episodes to establish a baseline performance metric:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"metadata\": {\n \"colab\": {\n \"base_uri\": \"https://localhost:8080/\"\n },\n \"id\": \"9SuT6MClyjyH\",\n \"outputId\": \"1e48f13b-c1fe-4fc2-ca1b-669485efdcae\",\n \"ExecuteTime\": {\n \"end_time\": \"2025-10-26T21:59:31.362074Z\",\n \"start_time\": \"2025-10-26T21:59:30.752198Z\"\n }\n },\n \"source\": [\n \"# Collect 9 complete episodes with environment reset\\n\",\n \"collect_result = test_collector.collect(reset_before_collect=True, n_episode=9)\\n\",\n \"\\n\",\n \"collect_result.pprint_asdict()\"\n ],\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"CollectStats\\n\",\n \"----------------------------------------\\n\",\n \"{ 'collect_speed': 288.36823267420584,\\n\",\n \" 'collect_time': 0.5860562324523926,\\n\",\n \" 'lens': array([15, 22, 29, 22, 8, 16, 28, 10, 19]),\\n\",\n \" 'lens_stat': { 'max': 29.0,\\n\",\n \" 'mean': 18.77777777777778,\\n\",\n \" 'min': 8.0,\\n\",\n \" 'std': 6.876332643007022},\\n\",\n \" 'n_collected_episodes': 9,\\n\",\n \" 'n_collected_steps': 169,\\n\",\n \" 'pred_dist_std_array': array([[0.49482444],\\n\",\n \" [0.49513358],\\n\",\n \" [0.491721 ],\\n\",\n \" [0.49804375],\\n\",\n \" [0.48936436],\\n\",\n \" [0.49519676],\\n\",\n \" [0.49186328],\\n\",\n \" [0.4981152 ],\\n\",\n \" [0.49512368],\\n\",\n \" [0.49527684],\\n\",\n \" [0.49800068],\\n\",\n \" [0.4982014 ],\\n\",\n \" [0.49516457],\\n\",\n \" [0.4953748 ],\\n\",\n \" [0.49805665],\\n\",\n \" [0.49269873],\\n\",\n \" [0.49946678],\\n\",\n \" [0.49553478],\\n\",\n \" [0.49997097],\\n\",\n \" [0.49289048],\\n\",\n \" [0.4998387 ],\\n\",\n \" [0.4957225 ],\\n\",\n \" [0.49908724],\\n\",\n \" [0.4930759 ],\\n\",\n \" [0.49776313],\\n\",\n \" [0.49590224],\\n\",\n \" [0.49913287],\\n\",\n \" [0.4986142 ],\\n\",\n \" [0.4998805 ],\\n\",\n \" [0.49605882],\\n\",\n \" [0.49581137],\\n\",\n \" [0.49861884],\\n\",\n \" [0.4922438 ],\\n\",\n \" [0.4962572 ],\\n\",\n \" [0.49569792],\\n\",\n \" [0.49863097],\\n\",\n \" [0.4982123 ],\\n\",\n \" [0.49961406],\\n\",\n \" [0.49553847],\\n\",\n \" [0.4999985 ],\\n\",\n \" [0.49808985],\\n\",\n \" [0.4997094 ],\\n\",\n \" [0.49964666],\\n\",\n \" [0.4987858 ],\\n\",\n \" [0.49796286],\\n\",\n \" [0.4948797 ],\\n\",\n \" [0.49960598],\\n\",\n \" [0.4916098 ],\\n\",\n \" [0.4999896 ],\\n\",\n \" [0.49003887],\\n\",\n \" [0.4997966 ],\\n\",\n \" [0.48927104],\\n\",\n \" [0.4999768 ],\\n\",\n \" [0.4899478 ],\\n\",\n \" [0.49948972],\\n\",\n \" [0.49140957],\\n\",\n \" [0.4978501 ],\\n\",\n \" [0.49466696],\\n\",\n \" [0.49509352],\\n\",\n \" [0.49118617],\\n\",\n \" [0.49797186],\\n\",\n \" [0.49447665],\\n\",\n \" [0.49950802],\\n\",\n \" [0.49740306],\\n\",\n \" [0.498081 ],\\n\",\n \" [0.49935713],\\n\",\n \" [0.49534237],\\n\",\n \" [0.49994358],\\n\",\n \" [0.49823537],\\n\",\n \" [0.499905 ],\\n\",\n \" [0.4955164 ],\\n\",\n \" [0.49991024],\\n\",\n \" [0.49839276],\\n\",\n \" [0.4999328 ],\\n\",\n \" [0.49570385],\\n\",\n \" [0.4993451 ],\\n\",\n \" [0.49855497],\\n\",\n \" [0.49995714],\\n\",\n \" [0.4995841 ],\\n\",\n \" [0.49939576],\\n\",\n \" [0.49999622],\\n\",\n \" [0.49824795],\\n\",\n \" [0.49972966],\\n\",\n \" [0.49653304],\\n\",\n \" [0.49880832],\\n\",\n \" [0.49425703],\\n\",\n \" [0.49974373],\\n\",\n \" [0.49662435],\\n\",\n \" [0.49473634],\\n\",\n \" [0.49472553],\\n\",\n \" [0.49773476],\\n\",\n \" [0.49140546],\\n\",\n \" [0.49935693],\\n\",\n \" [0.48954245],\\n\",\n \" [0.4999408 ],\\n\",\n \" [0.491403 ],\\n\",\n \" [0.49988943],\\n\",\n \" [0.49483237],\\n\",\n \" [0.49920663],\\n\",\n \" [0.49134007],\\n\",\n \" [0.49793968],\\n\",\n \" [0.4894678 ],\\n\",\n \" [0.49924025],\\n\",\n \" [0.4912263 ],\\n\",\n \" [0.4945435 ],\\n\",\n \" [0.49469063],\\n\",\n \" [0.49759832],\\n\",\n \" [0.49754107],\\n\",\n \" [0.49466005],\\n\",\n \" [0.49943802],\\n\",\n \" [0.4977191 ],\\n\",\n \" [0.49995443],\\n\",\n \" [0.49479046],\\n\",\n \" [0.49937534],\\n\",\n \" [0.49785116],\\n\",\n \" [0.49731255],\\n\",\n \" [0.49934685],\\n\",\n \" [0.4993554 ],\\n\",\n \" [0.49798217],\\n\",\n \" [0.4999266 ],\\n\",\n \" [0.4993439 ],\\n\",\n \" [0.49931702],\\n\",\n \" [0.49815634],\\n\",\n \" [0.49991363],\\n\",\n \" [0.4993506 ],\\n\",\n \" [0.49928144],\\n\",\n \" [0.49821213],\\n\",\n \" [0.4973895 ],\\n\",\n \" [0.49938264],\\n\",\n \" [0.4992856 ],\\n\",\n \" [0.4999623 ],\\n\",\n \" [0.49991205],\\n\",\n \" [0.49940434],\\n\",\n \" [0.49991933],\\n\",\n \" [0.49825713],\\n\",\n \" [0.49990463],\\n\",\n \" [0.49554875],\\n\",\n \" [0.49924377],\\n\",\n \" [0.49196848],\\n\",\n \" [0.49991465],\\n\",\n \" [0.48965713],\\n\",\n \" [0.49991086],\\n\",\n \" [0.4888782 ],\\n\",\n \" [0.49921995],\\n\",\n \" [0.48808664],\\n\",\n \" [0.49516302],\\n\",\n \" [0.48725367],\\n\",\n \" [0.49179506],\\n\",\n \" [0.4879356 ],\\n\",\n \" [0.4952572 ],\\n\",\n \" [0.48861024],\\n\",\n \" [0.49187768],\\n\",\n \" [0.48927858],\\n\",\n \" [0.4953288 ],\\n\",\n \" [0.48839873],\\n\",\n \" [0.49193203],\\n\",\n \" [0.49538046],\\n\",\n \" [0.49808696],\\n\",\n \" [0.49537748],\\n\",\n \" [0.49810043],\\n\",\n \" [0.4953903 ],\\n\",\n \" [0.4981276 ],\\n\",\n \" [0.49956635],\\n\",\n \" [0.49998853],\\n\",\n \" [0.49978945],\\n\",\n \" [0.49897715],\\n\",\n \" [0.4975953 ],\\n\",\n \" [0.49903452],\\n\",\n \" [0.49765074]], dtype=float32),\\n\",\n \" 'pred_dist_std_array_stat': { 0: { 'max': 0.4999985098838806,\\n\",\n \" 'mean': 0.4965951144695282,\\n\",\n \" 'min': 0.48725366592407227,\\n\",\n \" 'std': 0.003376598935574293}},\\n\",\n \" 'returns': array([15., 22., 29., 22., 8., 16., 28., 10., 19.]),\\n\",\n \" 'returns_stat': { 'max': 29.0,\\n\",\n \" 'mean': 18.77777777777778,\\n\",\n \" 'min': 8.0,\\n\",\n \" 'std': 6.876332643007022}}\\n\"\n ]\n }\n ],\n \"execution_count\": 3\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"id\": \"zX9AQY0M0R3C\"\n },\n \"source\": [\n \"### Baseline Comparison: Random Policy\\n\",\n \"\\n\",\n \"To contextualize the initialized policy's performance, we establish a random action baseline:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"metadata\": {\n \"colab\": {\n \"base_uri\": \"https://localhost:8080/\"\n },\n \"id\": \"UEcs8P8P0RLt\",\n \"outputId\": \"85f02f9d-b79b-48b2-99c6-36a1602f0884\",\n \"ExecuteTime\": {\n \"end_time\": \"2025-10-26T21:59:31.431099Z\",\n \"start_time\": \"2025-10-26T21:59:31.371074Z\"\n }\n },\n \"source\": [\n \"# Evaluate random policy by sampling actions uniformly from action space\\n\",\n \"collect_result = test_collector.collect(reset_before_collect=True, n_episode=9, random=True)\\n\",\n \"\\n\",\n \"collect_result.pprint_asdict()\"\n ],\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"CollectStats\\n\",\n \"----------------------------------------\\n\",\n \"{ 'collect_speed': 4407.5322624798,\\n\",\n \" 'collect_time': 0.053998470306396484,\\n\",\n \" 'lens': array([11, 13, 15, 29, 15, 12, 15, 30, 98]),\\n\",\n \" 'lens_stat': { 'max': 98.0,\\n\",\n \" 'mean': 26.444444444444443,\\n\",\n \" 'min': 11.0,\\n\",\n \" 'std': 26.16236105175657},\\n\",\n \" 'n_collected_episodes': 9,\\n\",\n \" 'n_collected_steps': 238,\\n\",\n \" 'pred_dist_std_array': None,\\n\",\n \" 'pred_dist_std_array_stat': None,\\n\",\n \" 'returns': array([11., 13., 15., 29., 15., 12., 15., 30., 98.]),\\n\",\n \" 'returns_stat': { 'max': 98.0,\\n\",\n \" 'mean': 26.444444444444443,\\n\",\n \" 'min': 11.0,\\n\",\n \" 'std': 26.16236105175657}}\\n\"\n ]\n }\n ],\n \"execution_count\": 4\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"id\": \"sKQRTiG10ljU\"\n },\n \"source\": [\n \"**Observation**: The randomly initialized policy performs comparably to (or worse than) uniform random actions prior to training. This is expected behavior, as the network weights lack task-specific optimization.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"id\": \"8RKmHIoG1A1k\"\n },\n \"source\": [\n \"### Training Data Collection\\n\",\n \"\\n\",\n \"During the training phase, the Collector manages experience gathering and automatic storage in a replay buffer. This enables the experience replay mechanism fundamental to off-policy algorithms.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"metadata\": {\n \"editable\": true,\n \"id\": \"CB9XB9bF1YPC\",\n \"slideshow\": {\n \"slide_type\": \"\"\n },\n \"tags\": [],\n \"ExecuteTime\": {\n \"end_time\": \"2025-10-26T21:59:31.452144Z\",\n \"start_time\": \"2025-10-26T21:59:31.444096Z\"\n }\n },\n \"source\": [\n \"# Configuration for parallel training data collection\\n\",\n \"train_env_num = 4\\n\",\n \"buffer_size = 100\\n\",\n \"\\n\",\n \"# Initialize vectorized training environments\\n\",\n \"train_envs = DummyVectorEnv([lambda: gym.make(\\\"CartPole-v1\\\") for _ in range(train_env_num)])\\n\",\n \"\\n\",\n \"# Create replay buffer compatible with vectorized environments\\n\",\n \"replayBuffer = VectorReplayBuffer(buffer_size, train_env_num)\\n\",\n \"\\n\",\n \"# Initialize training collector with buffer integration\\n\",\n \"training_collector = Collector[CollectStats](policy, train_envs, replayBuffer)\"\n ],\n \"outputs\": [],\n \"execution_count\": 5\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"id\": \"rWKDazA42IUQ\"\n },\n \"source\": [\n \"### Step-Based Collection\\n\",\n \"\\n\",\n \"The Collector supports both step-based and episode-based collection modes. Here we demonstrate step-based collection, which is commonly used in training loops with fixed update frequencies.\\n\",\n \"\\n\",\n \"**Note**: When using vectorized environments, the actual number of collected steps may exceed the requested amount to maintain synchronization across parallel environments.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"metadata\": {\n \"colab\": {\n \"base_uri\": \"https://localhost:8080/\"\n },\n \"id\": \"-fUtQOnM2Yi1\",\n \"outputId\": \"dceee987-433e-4b75-ed9e-823c20a9e1c2\",\n \"ExecuteTime\": {\n \"end_time\": \"2025-10-26T21:59:31.501487Z\",\n \"start_time\": \"2025-10-26T21:59:31.459140Z\"\n }\n },\n \"source\": [\n \"# Reset collector and buffer to clean state\\n\",\n \"training_collector.reset()\\n\",\n \"replayBuffer.reset()\\n\",\n \"\\n\",\n \"print(f\\\"Replay buffer before collecting is empty, and has length={len(replayBuffer)} \\\\n\\\")\\n\",\n \"\\n\",\n \"# Collect 50 environment steps\\n\",\n \"n_step = 50\\n\",\n \"collect_result = training_collector.collect(n_step=n_step)\\n\",\n \"\\n\",\n \"print(\\n\",\n \" f\\\"Replay buffer after collecting {n_step} steps has length={len(replayBuffer)}.\\\\n\\\"\\n\",\n \" f\\\"The actual count may exceed n_step when it is not a multiple of train_env_num \\\\n\\\"\\n\",\n \" f\\\"due to vectorization synchronization requirements.\\\\n\\\",\\n\",\n \")\\n\",\n \"\\n\",\n \"collect_result.pprint_asdict()\"\n ],\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"Replay buffer before collecting is empty, and has length=0 \\n\",\n \"\\n\",\n \"Replay buffer after collecting 50 steps has length=52.\\n\",\n \"The actual count may exceed n_step when it is not a multiple of train_env_num \\n\",\n \"due to vectorization synchronization requirements.\\n\",\n \"\\n\",\n \"CollectStats\\n\",\n \"----------------------------------------\\n\",\n \"{ 'collect_speed': 1529.5011711244197,\\n\",\n \" 'collect_time': 0.03399801254272461,\\n\",\n \" 'lens': array([], dtype=int32),\\n\",\n \" 'lens_stat': None,\\n\",\n \" 'n_collected_episodes': 0,\\n\",\n \" 'n_collected_steps': 52,\\n\",\n \" 'pred_dist_std_array': array([[0.4944575 ],\\n\",\n \" [0.49571753],\\n\",\n \" [0.49482644],\\n\",\n \" [0.49571693],\\n\",\n \" [0.49746 ],\\n\",\n \" [0.49228 ],\\n\",\n \" [0.491648 ],\\n\",\n \" [0.49237084],\\n\",\n \" [0.49931562],\\n\",\n \" [0.48953396],\\n\",\n \" [0.4949102 ],\\n\",\n \" [0.49022076],\\n\",\n \" [0.49992043],\\n\",\n \" [0.4921799 ],\\n\",\n \" [0.49171764],\\n\",\n \" [0.4894729 ],\\n\",\n \" [0.4992769 ],\\n\",\n \" [0.48948848],\\n\",\n \" [0.49497682],\\n\",\n \" [0.48870105],\\n\",\n \" [0.49763048],\\n\",\n \" [0.49201292],\\n\",\n \" [0.49787888],\\n\",\n \" [0.4893877 ],\\n\",\n \" [0.49927947],\\n\",\n \" [0.4955971 ],\\n\",\n \" [0.49943653],\\n\",\n \" [0.49005648],\\n\",\n \" [0.49780723],\\n\",\n \" [0.49179533],\\n\",\n \" [0.49995926],\\n\",\n \" [0.49153325],\\n\",\n \" [0.49928913],\\n\",\n \" [0.48941523],\\n\",\n \" [0.49986592],\\n\",\n \" [0.49499276],\\n\",\n \" [0.4999287 ],\\n\",\n \" [0.49152908],\\n\",\n \" [0.4991583 ],\\n\",\n \" [0.49093276],\\n\",\n \" [0.4998997 ],\\n\",\n \" [0.48936346],\\n\",\n \" [0.4978821 ],\\n\",\n \" [0.49442956],\\n\",\n \" [0.49992698],\\n\",\n \" [0.49117777],\\n\",\n \" [0.49921465],\\n\",\n \" [0.49751103],\\n\",\n \" [0.4992887 ],\\n\",\n \" [0.4893143 ],\\n\",\n \" [0.49991187],\\n\",\n \" [0.4992216 ]], dtype=float32),\\n\",\n \" 'pred_dist_std_array_stat': { 0: { 'max': 0.499959260225296,\\n\",\n \" 'mean': 0.49497732520103455,\\n\",\n \" 'min': 0.4887010455131531,\\n\",\n \" 'std': 0.003929081838577986}},\\n\",\n \" 'returns': array([], dtype=float64),\\n\",\n \" 'returns_stat': None}\\n\"\n ]\n },\n {\n \"name\": \"stderr\",\n \"output_type\": \"stream\",\n \"text\": [\n \"F:\\\\Users\\\\Dominik Jain\\\\Dev\\\\AI\\\\tianshou\\\\tianshou\\\\data\\\\collector.py:537: UserWarning: n_step=50 is not a multiple of (self.env_num=4), which may cause extra transitions being collected into the buffer.\\n\",\n \" warnings.warn(\\n\"\n ]\n }\n ],\n \"execution_count\": 6\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Buffer Sampling Verification\\n\",\n \"\\n\",\n \"Verify that collected experiences are properly stored and can be sampled for training:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"metadata\": {\n \"ExecuteTime\": {\n \"end_time\": \"2025-10-26T21:59:31.517583Z\",\n \"start_time\": \"2025-10-26T21:59:31.509483Z\"\n }\n },\n \"source\": [\n \"# Sample mini-batch of 10 transitions from buffer\\n\",\n \"replayBuffer.sample(10)\"\n ],\n \"outputs\": [\n {\n \"data\": {\n \"text/plain\": [\n \"(Batch(\\n\",\n \" obs: array([[-7.59119692e-04, -3.54404569e-01, 8.15278068e-02,\\n\",\n \" 6.34967446e-01],\\n\",\n \" [ 2.03953441e-02, -5.46947002e-01, 4.59121428e-02,\\n\",\n \" 8.69558692e-01],\\n\",\n \" [-5.53812869e-02, -3.63834441e-01, 1.84285983e-01,\\n\",\n \" 8.54350269e-01],\\n\",\n \" [ 5.94463721e-02, -3.39802876e-02, -5.61027192e-02,\\n\",\n \" -2.05838066e-02],\\n\",\n \" [ 1.70439295e-02, -3.58715117e-01, 2.22064722e-02,\\n\",\n \" 6.39448643e-01],\\n\",\n \" [ 1.51256351e-02, 2.27344140e-01, 1.95531528e-02,\\n\",\n \" -2.54039675e-01],\\n\",\n \" [-7.69001395e-02, -7.54580617e-01, 1.79230303e-01,\\n\",\n \" 1.36748278e+00],\\n\",\n \" [-3.51171643e-02, -1.14145672e+00, 1.09657384e-01,\\n\",\n \" 1.86768615e+00],\\n\",\n \" [ 2.10114848e-02, 3.47817928e-01, -1.05900057e-01,\\n\",\n \" -6.93330288e-01],\\n\",\n \" [-1.53460149e-02, 5.40259123e-01, -4.36654910e-02,\\n\",\n \" -9.24050748e-01]], dtype=float32),\\n\",\n \" act: array([1, 1, 0, 1, 0, 0, 1, 1, 0, 1], dtype=int64),\\n\",\n \" rew: array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1.]),\\n\",\n \" terminated: array([False, False, False, False, False, False, False, False, False,\\n\",\n \" False]),\\n\",\n \" truncated: array([False, False, False, False, False, False, False, False, False,\\n\",\n \" False]),\\n\",\n \" done: array([False, False, False, False, False, False, False, False, False,\\n\",\n \" False]),\\n\",\n \" obs_next: array([[-0.00784721, -0.16050874, 0.09422715, 0.36903235],\\n\",\n \" [ 0.0094564 , -0.3524787 , 0.06330331, 0.59165704],\\n\",\n \" [-0.06265797, -0.5609251 , 0.20137298, 1.1988543 ],\\n\",\n \" [ 0.05876676, 0.16189948, -0.05651439, -0.33042672],\\n\",\n \" [ 0.00986963, -0.5541395 , 0.03499544, 0.93904114],\\n\",\n \" [ 0.01967252, 0.03194853, 0.01447236, 0.04474595],\\n\",\n \" [-0.09199175, -0.5620968 , 0.20657995, 1.135794 ],\\n\",\n \" [-0.05794629, -0.94769216, 0.1470111 , 1.6109598 ],\\n\",\n \" [ 0.02796784, 0.15431203, -0.11976666, -0.435774 ],\\n\",\n \" [-0.00454083, 0.73594284, -0.06214651, -1.2301302 ]],\\n\",\n \" dtype=float32),\\n\",\n \" info: Batch(\\n\",\n \" env_id: array([0, 0, 0, 1, 2, 2, 2, 2, 3, 3]),\\n\",\n \" ),\\n\",\n \" policy: Batch(),\\n\",\n \" ),\\n\",\n \" array([ 6, 3, 12, 31, 56, 53, 62, 60, 81, 78]))\"\n ]\n },\n \"execution_count\": 7,\n \"metadata\": {},\n \"output_type\": \"execute_result\"\n }\n ],\n \"execution_count\": 7\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"id\": \"8NP7lOBU3-VS\"\n },\n \"source\": [\n \"## Advanced Topics\\n\",\n \"\\n\",\n \"### Asynchronous Collection\\n\",\n \"\\n\",\n \"The standard `Collector` implementation may collect more steps than requested when using vectorized environments. In the example above, requesting 50 steps resulted in 52 steps (the smallest multiple of 4 that is ≥50).\\n\",\n \"\\n\",\n \"For scenarios requiring precise step counts, Tianshou provides the `AsyncCollector`, which enables exact step collection at the cost of additional implementation complexity. This is particularly relevant for:\\n\",\n \"- Strict reproducibility requirements\\n\",\n \"- Algorithms sensitive to exact batch sizes\\n\",\n \"- Fine-grained control over data collection\\n\",\n \"\\n\",\n \"Consult the [AsyncCollector documentation](https://tianshou.org/en/master/03_api/data/collector.html#tianshou.data.collector.AsyncCollector) for implementation details and usage patterns.\"\n ]\n }\n ],\n \"metadata\": {\n \"colab\": {\n \"provenance\": []\n },\n \"kernelspec\": {\n \"display_name\": \"Python 3 (ipykernel)\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.11.4\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 4\n}\n" }, { "path": "docs/02_deep_dives/L6_MARL.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Multi-Agent Reinforcement Learning (MARL)\\n\",\n \"\\n\",\n \"This tutorial demonstrates how to use Tianshou for multi-agent reinforcement learning scenarios. We'll explore different MARL paradigms and implement a practical example using the Tic-Tac-Toe game.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## MARL Paradigms\\n\",\n \"\\n\",\n \"Tianshou supports three fundamental types of multi-agent reinforcement learning paradigms:\\n\",\n \"\\n\",\n \"1. **Simultaneous move**: All agents take their actions at each timestep simultaneously (e.g., MOBA games)\\n\",\n \"2. **Cyclic move**: Agents take actions sequentially in turns (e.g., Go)\\n\",\n \"3. **Conditional move**: The environment conditionally selects which agent acts at each timestep (e.g., [Pig Game](https://en.wikipedia.org/wiki/Pig_(dice_game)))\\n\",\n \"\\n\",\n \"Our approach addresses these multi-agent RL problems by converting them into traditional single-agent RL formulations.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Converting MARL to Single-Agent RL\\n\",\n \"\\n\",\n \"### Simultaneous Move\\n\",\n \"\\n\",\n \"For simultaneous-move scenarios, the solution is straightforward: we add an extra `num_agents` dimension to the state, action, and reward tensors. No other modifications are necessary.\\n\",\n \"\\n\",\n \"### Cyclic and Conditional Move\\n\",\n \"\\n\",\n \"Both cyclic and conditional move scenarios can be unified into a single framework. At each timestep, the environment selects an agent identified by `agent_id` to act. Since multiple agents are typically wrapped into a single object (the \\\"abstract agent\\\"), we pass the `agent_id` to this abstract agent, which then delegates the action to the appropriate specific agent.\\n\",\n \"\\n\",\n \"Additionally, in multi-agent RL, the set of legal actions often varies across timesteps (as in Go). Therefore, the environment must also provide a legal action mask to the abstract agent. This mask is a boolean array where `True` indicates available actions and `False` indicates illegal actions at the current timestep.\\n\",\n \"\\n\",\n \"
\\n\",\n \"
\\n\",\n \"The abstract agent framework for multi-agent RL\\n\",\n \"
\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Unified Formulation\\n\",\n \"\\n\",\n \"This architecture leads to the following formulation of multi-agent RL:\\n\",\n \"\\n\",\n \"```python\\n\",\n \"act = policy(state, agent_id, mask)\\n\",\n \"(next_state, next_agent_id, next_mask), reward = env.step(act)\\n\",\n \"```\\n\",\n \"\\n\",\n \"By constructing an augmented state `state_ = (state, agent_id, mask)`, we can reduce this to the standard single-agent RL formulation:\\n\",\n \"\\n\",\n \"```python\\n\",\n \"act = policy(state_)\\n\",\n \"next_state_, reward = env.step(act)\\n\",\n \"```\\n\",\n \"\\n\",\n \"Following this principle, we'll implement a Q-learning algorithm to play [Tic-Tac-Toe](https://en.wikipedia.org/wiki/Tic-tac-toe) against a random opponent.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## PettingZoo Integration\\n\",\n \"\\n\",\n \"Tianshou is fully compatible with [PettingZoo](https://pettingzoo.farama.org/) environments for multi-agent RL. While Tianshou doesn't directly provide specialized MARL facilities, it offers a flexible framework that can be adapted to various MARL scenarios.\\n\",\n \"\\n\",\n \"For comprehensive tutorials on using Tianshou with PettingZoo, refer to:\\n\",\n \"\\n\",\n \"* [Beginner Tutorial](https://pettingzoo.farama.org/tutorials/tianshou/beginner/)\\n\",\n \"* [Intermediate Tutorial](https://pettingzoo.farama.org/tutorials/tianshou/intermediate/)\\n\",\n \"* [Advanced Tutorial](https://pettingzoo.farama.org/tutorials/tianshou/advanced/)\\n\",\n \"\\n\",\n \"In this tutorial, we'll demonstrate how to use Tianshou in a multi-agent setting where only one agent is trained while the other uses a fixed random policy. You can then use this as a blueprint to replace the random policy with another trainable agent.\\n\",\n \"\\n\",\n \"Specifically, we'll train an agent to play Tic-Tac-Toe against a random opponent:\\n\",\n \"\\n\",\n \"
\\n\",\n \"
\\n\",\n \"Tic-Tac-Toe game board\\n\",\n \"
\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Exploring the Tic-Tac-Toe Environment\\n\",\n \"\\n\",\n \"The complete scripts are located in `test/pettingzoo/`. Tianshou provides the `PettingZooEnv` wrapper class that can wrap any PettingZoo environment. Let's explore the 3×3 Tic-Tac-Toe environment provided by PettingZoo.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"from pettingzoo.classic import tictactoe_v3 # the Tic-Tac-Toe environment\\n\",\n \"\\n\",\n \"from tianshou.env import PettingZooEnv # wrapper for PettingZoo environments\\n\",\n \"\\n\",\n \"# Initialize the environment\\n\",\n \"# The board has 3 rows and 3 columns (9 positions total)\\n\",\n \"# Players place 'X' and 'O' alternately on the board\\n\",\n \"# The first player to get 3 consecutive marks wins\\n\",\n \"env = PettingZooEnv(tictactoe_v3.env(render_mode=\\\"human\\\"))\\n\",\n \"obs = env.reset()\\n\",\n \"env.render() # render the empty board\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"The output shows an empty 3×3 board:\\n\",\n \"\\n\",\n \"```\\n\",\n \"board (step 0):\\n\",\n \" | |\\n\",\n \" - | - | -\\n\",\n \"_____|_____|_____\\n\",\n \" | |\\n\",\n \" - | - | -\\n\",\n \"_____|_____|_____\\n\",\n \" | |\\n\",\n \" - | - | -\\n\",\n \" | |\\n\",\n \"```\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Examine the observation structure\\n\",\n \"print(obs)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Understanding the Observation Space\\n\",\n \"\\n\",\n \"The observation returned by the environment is a dictionary with three keys:\\n\",\n \"\\n\",\n \"- **`agent_id`**: The identifier of the currently acting agent (e.g., `'player_1'` or `'player_2'`)\\n\",\n \"\\n\",\n \"- **`obs`**: The actual environment observation. For Tic-Tac-Toe, this is a numpy array with shape `(3, 3, 2)`:\\n\",\n \" - For `player_1`: The first 3×3 plane represents X placements, the second plane represents O placements\\n\",\n \" - For `player_2`: The planes are swapped (O in first plane, X in second)\\n\",\n \" - Each cell contains either 0 (empty/not placed) or 1 (mark placed)\\n\",\n \"\\n\",\n \"- **`mask`**: A boolean array indicating legal actions at the current timestep. For Tic-Tac-Toe, index `i` corresponds to position `(i // 3, i % 3)` on the board. If `mask[i] == True`, the player can place their mark at that position. Initially, all positions are available, so all mask values are `True`.\\n\",\n \"\\n\",\n \"> **Note**: The mask representation is flexible and works for both discrete and continuous action spaces. While we use a boolean array here, you could also use action spaces like `gymnasium.spaces.Discrete` or `gymnasium.spaces.Box` to represent available actions.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Playing a Few Steps\\n\",\n \"\\n\",\n \"Let's play a couple of moves to understand the environment dynamics better.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import numpy as np\\n\",\n \"\\n\",\n \"# Take an action (place mark at position 0 - top-left corner)\\n\",\n \"action = 0 # action can be an integer or a numpy array with one element\\n\",\n \"obs, reward, done, truncated, info = env.step(action) # follows the Gymnasium API\\n\",\n \"\\n\",\n \"print(\\\"Observation after first move:\\\")\\n\",\n \"print(obs)\\n\",\n \"\\n\",\n \"# Examine the reward structure\\n\",\n \"# Reward has two items (one for each player): 1 for win, -1 for loss, 0 otherwise\\n\",\n \"print(f\\\"\\\\nReward: {reward}\\\")\\n\",\n \"\\n\",\n \"# Check if the game is over\\n\",\n \"print(f\\\"Done: {done}\\\")\\n\",\n \"\\n\",\n \"# Info is typically an empty dict in Tic-Tac-Toe but may contain useful information in other environments\\n\",\n \"print(f\\\"Info: {info}\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"Notice that after the first move:\\n\",\n \"- The `agent_id` switches to `'player_2'`\\n\",\n \"- The observation array shows the X placement in the first position\\n\",\n \"- The mask now has `False` at index 0 (that position is occupied)\\n\",\n \"- The reward is `[0, 0]` (no winner yet)\\n\",\n \"- The game continues (`done = False`)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": \"Note: If we continue playing, the game terminates when only one empty position remains, rather than when the board is completely full. This is because a player with only one available position has no meaningful choice.\"\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Random Agents\\n\",\n \"\\n\",\n \"Now that we understand the environment, let's start by watching two random agents play against each other.\\n\",\n \"\\n\",\n \"Tianshou provides built-in classes for multi-agent learning. The key components are:\\n\",\n \"\\n\",\n \"- **`RandomPolicy`**: A policy that randomly selects actions\\n\",\n \"- **`MultiAgentPolicyManager`**: Manages multiple agent policies and delegates actions to the appropriate agent based on `agent_id`\\n\",\n \"\\n\",\n \"
\\n\",\n \"
\\n\",\n \"The relationship between MultiAgentPolicyManager and individual agent policies\\n\",\n \"
\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"from tianshou.algorithm.multiagent.marl import MultiAgentOffPolicyAlgorithm\\n\",\n \"from tianshou.algorithm.random import MARLRandomDiscreteMaskedOffPolicyAlgorithm\\n\",\n \"from tianshou.data import Collector\\n\",\n \"from tianshou.env import DummyVectorEnv\\n\",\n \"\\n\",\n \"# Create a multi-agent algorithm with two random agents\\n\",\n \"policy = MultiAgentOffPolicyAlgorithm(\\n\",\n \" algorithms=[\\n\",\n \" MARLRandomDiscreteMaskedOffPolicyAlgorithm(action_space=env.action_space),\\n\",\n \" MARLRandomDiscreteMaskedOffPolicyAlgorithm(action_space=env.action_space),\\n\",\n \" ],\\n\",\n \" env=env,\\n\",\n \")\\n\",\n \"\\n\",\n \"# Vectorize the environment for the collector\\n\",\n \"env = DummyVectorEnv([lambda: env])\\n\",\n \"\\n\",\n \"# Create a collector to gather trajectories\\n\",\n \"collector = Collector(policy, env)\\n\",\n \"\\n\",\n \"# Collect and visualize one episode\\n\",\n \"result = collector.collect(n_episode=1, render=0.1, reset_before_collect=True)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"You'll see the game progress step by step. Here's an example of the final moves:\\n\",\n \"\\n\",\n \"```\\n\",\n \" | |\\n\",\n \" X | X | -\\n\",\n \"_____|_____|_____\\n\",\n \" | |\\n\",\n \" X | O | -\\n\",\n \"_____|_____|_____\\n\",\n \" | |\\n\",\n \" O | - | -\\n\",\n \" | |\\n\",\n \"```\\n\",\n \"\\n\",\n \"```\\n\",\n \" | |\\n\",\n \" X | X | -\\n\",\n \"_____|_____|_____\\n\",\n \" | |\\n\",\n \" X | O | -\\n\",\n \"_____|_____|_____\\n\",\n \" | |\\n\",\n \" O | - | O\\n\",\n \" | |\\n\",\n \"```\\n\",\n \"\\n\",\n \"```\\n\",\n \" | |\\n\",\n \" X | X | X\\n\",\n \"_____|_____|_____\\n\",\n \" | |\\n\",\n \" X | O | -\\n\",\n \"_____|_____|_____\\n\",\n \" | |\\n\",\n \" O | - | O\\n\",\n \" | |\\n\",\n \"```\\n\",\n \"\\n\",\n \"Random agents perform poorly. In the game above, although agent 2 eventually wins, a smart agent 1 would have won immediately by placing an X at position (1, 1) (center of middle row).\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Training an Agent Against a Random Opponent\\n\",\n \"\\n\",\n \"Now let's train an intelligent agent! We'll use Deep Q-Network (DQN) to learn optimal play against a random opponent.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Imports and Setup\\n\",\n \"\\n\",\n \"First, let's import all necessary modules:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import os\\n\",\n \"from copy import deepcopy\\n\",\n \"from functools import partial\\n\",\n \"\\n\",\n \"import gymnasium\\n\",\n \"import torch\\n\",\n \"from pettingzoo.classic import tictactoe_v3\\n\",\n \"from torch.utils.tensorboard import SummaryWriter\\n\",\n \"\\n\",\n \"from tianshou.algorithm import (\\n\",\n \" DQN,\\n\",\n \" Algorithm,\\n\",\n \" MARLRandomDiscreteMaskedOffPolicyAlgorithm,\\n\",\n \" MultiAgentOffPolicyAlgorithm,\\n\",\n \")\\n\",\n \"from tianshou.algorithm.algorithm_base import OffPolicyAlgorithm\\n\",\n \"from tianshou.algorithm.modelfree.dqn import DiscreteQLearningPolicy\\n\",\n \"from tianshou.algorithm.optim import AdamOptimizerFactory, OptimizerFactory\\n\",\n \"from tianshou.data import Collector, CollectStats, VectorReplayBuffer\\n\",\n \"from tianshou.data.stats import InfoStats\\n\",\n \"from tianshou.env import DummyVectorEnv\\n\",\n \"from tianshou.env.pettingzoo_env import PettingZooEnv\\n\",\n \"from tianshou.trainer import OffPolicyTrainerParams\\n\",\n \"from tianshou.utils import TensorboardLogger\\n\",\n \"from tianshou.utils.net.common import Net\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Hyperparameters\\n\",\n \"\\n\",\n \"Let's define the hyperparameters for our training experiment directly (no argparse needed in notebooks!):\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Define hyperparameters\\n\",\n \"class Args:\\n\",\n \" seed = 1626\\n\",\n \" eps_test = 0.05\\n\",\n \" eps_train = 0.1\\n\",\n \" buffer_size = 20000\\n\",\n \" lr = 1e-4\\n\",\n \" gamma = 0.9 # A smaller gamma favors earlier wins\\n\",\n \" n_step = 3\\n\",\n \" target_update_freq = 320\\n\",\n \" epoch = 50\\n\",\n \" epoch_num_steps = 1000\\n\",\n \" collection_step_num_env_steps = 10\\n\",\n \" update_per_step = 0.1\\n\",\n \" batch_size = 64\\n\",\n \" hidden_sizes = [128, 128, 128, 128] # noqa: RUF012\\n\",\n \" num_train_envs = 10\\n\",\n \" num_test_envs = 10\\n\",\n \" logdir = \\\"log\\\"\\n\",\n \" render = 0.1\\n\",\n \" win_rate = 0.6 # Target winning rate (optimal policy can get ~0.7)\\n\",\n \" watch = False # Set to True to skip training and watch pre-trained models\\n\",\n \" agent_id = 2 # The learned agent plays as player 2\\n\",\n \" resume_path = \\\"\\\" # Path to pre-trained agent .pth file\\n\",\n \" opponent_path = \\\"\\\" # Path to pre-trained opponent .pth file\\n\",\n \" device = \\\"cuda\\\" if torch.cuda.is_available() else \\\"cpu\\\"\\n\",\n \" model_save_path = None # Will be set in save_best_fn\\n\",\n \"\\n\",\n \"\\n\",\n \"args = Args()\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Agent Setup\\n\",\n \"\\n\",\n \"The `get_agents` function creates and configures our agents:\\n\",\n \"\\n\",\n \"- **Neural Network**: We use `Net`, a multi-layer perceptron with ReLU activations\\n\",\n \"- **Learning Algorithm**: A `DiscreteQLearningPolicy` combined with `DQN` for Q-learning updates\\n\",\n \"- **Opponent**: Either a `MARLRandomDiscreteMaskedOffPolicyAlgorithm` that randomly chooses legal actions, or a pre-trained agent for self-play\\n\",\n \"\\n\",\n \"Both agents are managed by `MultiAgentOffPolicyAlgorithm`, which:\\n\",\n \"- Calls the correct agent based on `agent_id` in the observation\\n\",\n \"- Dispatches data to each agent according to their `agent_id`\\n\",\n \"- Makes each agent perceive the environment as a single-agent problem\\n\",\n \"\\n\",\n \"
\\n\",\n \"
\\n\",\n \"How MultiAgentOffPolicyAlgorithm coordinates agent algorithms\\n\",\n \"
\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"def get_env(render_mode: str | None = None) -> PettingZooEnv:\\n\",\n \" return PettingZooEnv(tictactoe_v3.env(render_mode=render_mode))\\n\",\n \"\\n\",\n \"\\n\",\n \"def get_agents(\\n\",\n \" args,\\n\",\n \" agent_learn: OffPolicyAlgorithm | None = None,\\n\",\n \" agent_opponent: OffPolicyAlgorithm | None = None,\\n\",\n \" optim: OptimizerFactory | None = None,\\n\",\n \") -> tuple[MultiAgentOffPolicyAlgorithm, torch.optim.Optimizer | None, list]:\\n\",\n \" \\\"\\\"\\\"Create or load agents for training.\\\"\\\"\\\"\\n\",\n \" env = get_env()\\n\",\n \" observation_space = (\\n\",\n \" env.observation_space.spaces[\\\"observation\\\"]\\n\",\n \" if isinstance(env.observation_space, gymnasium.spaces.Dict)\\n\",\n \" else env.observation_space\\n\",\n \" )\\n\",\n \" args.state_shape = observation_space.shape or int(observation_space.n)\\n\",\n \" args.action_shape = env.action_space.shape or int(env.action_space.n)\\n\",\n \"\\n\",\n \" if agent_learn is None:\\n\",\n \" # Create the neural network model\\n\",\n \" net = Net(\\n\",\n \" state_shape=args.state_shape,\\n\",\n \" action_shape=args.action_shape,\\n\",\n \" hidden_sizes=args.hidden_sizes,\\n\",\n \" ).to(args.device)\\n\",\n \"\\n\",\n \" if optim is None:\\n\",\n \" optim = AdamOptimizerFactory(lr=args.lr)\\n\",\n \"\\n\",\n \" # Create Q-learning policy for the learning agent\\n\",\n \" algorithm = DiscreteQLearningPolicy(\\n\",\n \" model=net,\\n\",\n \" action_space=env.action_space,\\n\",\n \" eps_training=args.eps_train,\\n\",\n \" eps_inference=args.eps_test,\\n\",\n \" )\\n\",\n \"\\n\",\n \" # Wrap in DQN algorithm\\n\",\n \" agent_learn = DQN(\\n\",\n \" policy=algorithm,\\n\",\n \" optim=optim,\\n\",\n \" n_step_return_horizon=args.n_step,\\n\",\n \" gamma=args.gamma,\\n\",\n \" target_update_freq=args.target_update_freq,\\n\",\n \" )\\n\",\n \"\\n\",\n \" if args.resume_path:\\n\",\n \" agent_learn.load_state_dict(torch.load(args.resume_path))\\n\",\n \"\\n\",\n \" if agent_opponent is None:\\n\",\n \" if args.opponent_path:\\n\",\n \" # Load a pre-trained opponent for self-play\\n\",\n \" agent_opponent = deepcopy(agent_learn)\\n\",\n \" agent_opponent.load_state_dict(torch.load(args.opponent_path))\\n\",\n \" else:\\n\",\n \" # Use a random opponent\\n\",\n \" agent_opponent = MARLRandomDiscreteMaskedOffPolicyAlgorithm(\\n\",\n \" action_space=env.action_space\\n\",\n \" )\\n\",\n \"\\n\",\n \" # Arrange agents based on which player position the learning agent takes\\n\",\n \" if args.agent_id == 1:\\n\",\n \" agents = [agent_learn, agent_opponent]\\n\",\n \" else:\\n\",\n \" agents = [agent_opponent, agent_learn]\\n\",\n \"\\n\",\n \" ma_algorithm = MultiAgentOffPolicyAlgorithm(algorithms=agents, env=env)\\n\",\n \" return ma_algorithm, optim, env.agents\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Training Loop\\n\",\n \"\\n\",\n \"The training procedure follows the standard Tianshou workflow, similar to single-agent DQN training:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"def train_agent(\\n\",\n \" args,\\n\",\n \" agent_learn: OffPolicyAlgorithm | None = None,\\n\",\n \" agent_opponent: OffPolicyAlgorithm | None = None,\\n\",\n \" optim: OptimizerFactory | None = None,\\n\",\n \") -> tuple[InfoStats, OffPolicyAlgorithm]:\\n\",\n \" \\\"\\\"\\\"Train the agent using DQN.\\\"\\\"\\\"\\n\",\n \" # ======== Environment Setup =========\\n\",\n \" train_envs = DummyVectorEnv([get_env for _ in range(args.num_train_envs)])\\n\",\n \" test_envs = DummyVectorEnv([get_env for _ in range(args.num_test_envs)])\\n\",\n \"\\n\",\n \" # Set random seeds for reproducibility\\n\",\n \" np.random.seed(args.seed)\\n\",\n \" torch.manual_seed(args.seed)\\n\",\n \" train_envs.seed(args.seed)\\n\",\n \" test_envs.seed(args.seed)\\n\",\n \"\\n\",\n \" # ======== Agent Setup =========\\n\",\n \" marl_algorithm, optim, agents = get_agents(\\n\",\n \" args,\\n\",\n \" agent_learn=agent_learn,\\n\",\n \" agent_opponent=agent_opponent,\\n\",\n \" optim=optim,\\n\",\n \" )\\n\",\n \"\\n\",\n \" # ======== Collector Setup =========\\n\",\n \" training_collector = Collector[CollectStats](\\n\",\n \" marl_algorithm,\\n\",\n \" train_envs,\\n\",\n \" VectorReplayBuffer(args.buffer_size, len(train_envs)),\\n\",\n \" exploration_noise=True,\\n\",\n \" )\\n\",\n \" test_collector = Collector[CollectStats](marl_algorithm, test_envs, exploration_noise=True)\\n\",\n \"\\n\",\n \" # Collect initial random samples\\n\",\n \" training_collector.reset()\\n\",\n \" training_collector.collect(n_step=args.batch_size * args.num_train_envs)\\n\",\n \"\\n\",\n \" # ======== Logging Setup =========\\n\",\n \" log_path = os.path.join(args.logdir, \\\"tic_tac_toe\\\", \\\"dqn\\\")\\n\",\n \" writer = SummaryWriter(log_path)\\n\",\n \" writer.add_text(\\\"args\\\", str(args))\\n\",\n \" logger = TensorboardLogger(writer)\\n\",\n \"\\n\",\n \" player_agent_id = agents[args.agent_id - 1]\\n\",\n \"\\n\",\n \" # ======== Callback Functions =========\\n\",\n \" def save_best_fn(policy: Algorithm) -> None:\\n\",\n \" \\\"\\\"\\\"Save the best performing policy.\\\"\\\"\\\"\\n\",\n \" if hasattr(args, \\\"model_save_path\\\") and args.model_save_path:\\n\",\n \" model_save_path = args.model_save_path\\n\",\n \" else:\\n\",\n \" model_save_path = os.path.join(args.logdir, \\\"tic_tac_toe\\\", \\\"dqn\\\", \\\"policy.pth\\\")\\n\",\n \" torch.save(policy.get_algorithm(player_agent_id).state_dict(), model_save_path)\\n\",\n \"\\n\",\n \" def stop_fn(mean_rewards: float) -> bool:\\n\",\n \" \\\"\\\"\\\"Stop training when target win rate is achieved.\\\"\\\"\\\"\\n\",\n \" return mean_rewards >= args.win_rate\\n\",\n \"\\n\",\n \" def reward_metric(rews: np.ndarray) -> np.ndarray:\\n\",\n \" \\\"\\\"\\\"Extract the reward for our learning agent.\\\"\\\"\\\"\\n\",\n \" return rews[:, args.agent_id - 1]\\n\",\n \"\\n\",\n \" # ======== Trainer =========\\n\",\n \" result = marl_algorithm.run_training(\\n\",\n \" OffPolicyTrainerParams(\\n\",\n \" training_collector=training_collector,\\n\",\n \" test_collector=test_collector,\\n\",\n \" max_epochs=args.epoch,\\n\",\n \" epoch_num_steps=args.epoch_num_steps,\\n\",\n \" collection_step_num_env_steps=args.collection_step_num_env_steps,\\n\",\n \" test_step_num_episodes=args.num_test_envs,\\n\",\n \" batch_size=args.batch_size,\\n\",\n \" stop_fn=stop_fn,\\n\",\n \" save_best_fn=save_best_fn,\\n\",\n \" update_step_num_gradient_steps_per_sample=args.update_per_step,\\n\",\n \" logger=logger,\\n\",\n \" test_in_training=False,\\n\",\n \" multi_agent_return_reduction=reward_metric,\\n\",\n \" show_progress=False,\\n\",\n \" )\\n\",\n \" )\\n\",\n \"\\n\",\n \" return result, marl_algorithm.get_algorithm(player_agent_id)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Evaluation Function\\n\",\n \"\\n\",\n \"This function allows us to watch a trained agent play:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"def watch(\\n\",\n \" args,\\n\",\n \" agent_learn: OffPolicyAlgorithm | None = None,\\n\",\n \" agent_opponent: OffPolicyAlgorithm | None = None,\\n\",\n \") -> None:\\n\",\n \" \\\"\\\"\\\"Watch a pre-trained agent play.\\\"\\\"\\\"\\n\",\n \" env = DummyVectorEnv([partial(get_env, render_mode=\\\"human\\\")])\\n\",\n \" policy, optim, agents = get_agents(args, agent_learn=agent_learn, agent_opponent=agent_opponent)\\n\",\n \" collector = Collector[CollectStats](policy, env, exploration_noise=True)\\n\",\n \" result = collector.collect(n_episode=1, render=args.render, reset_before_collect=True)\\n\",\n \" result.pprint_asdict()\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Running the Training\\n\",\n \"\\n\",\n \"Now let's train the agent and watch it play!\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Train the agent\\n\",\n \"result, agent = train_agent(args)\\n\",\n \"\\n\",\n \"# Watch the trained agent play\\n\",\n \"watch(args, agent)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Training Results\\n\",\n \"\\n\",\n \"After training for less than a minute, you'll see the agent play against the random opponent. Here's an example game:\\n\",\n \"\\n\",\n \"
\\n\",\n \"Example: Trained Agent vs Random Opponent\\n\",\n \"\\n\",\n \"```\\n\",\n \" | |\\n\",\n \" - | - | -\\n\",\n \"_____|_____|_____\\n\",\n \" | |\\n\",\n \" - | - | X\\n\",\n \"_____|_____|_____\\n\",\n \" | |\\n\",\n \" - | - | -\\n\",\n \" | |\\n\",\n \"```\\n\",\n \"\\n\",\n \"```\\n\",\n \" | |\\n\",\n \" - | - | -\\n\",\n \"_____|_____|_____\\n\",\n \" | |\\n\",\n \" - | O | X\\n\",\n \"_____|_____|_____\\n\",\n \" | |\\n\",\n \" - | - | -\\n\",\n \" | |\\n\",\n \"```\\n\",\n \"\\n\",\n \"```\\n\",\n \" | |\\n\",\n \" - | - | -\\n\",\n \"_____|_____|_____\\n\",\n \" | |\\n\",\n \" X | O | X\\n\",\n \"_____|_____|_____\\n\",\n \" | |\\n\",\n \" - | - | -\\n\",\n \" | |\\n\",\n \"```\\n\",\n \"\\n\",\n \"```\\n\",\n \" | |\\n\",\n \" - | O | -\\n\",\n \"_____|_____|_____\\n\",\n \" | |\\n\",\n \" X | O | X\\n\",\n \"_____|_____|_____\\n\",\n \" | |\\n\",\n \" - | - | -\\n\",\n \" | |\\n\",\n \"```\\n\",\n \"\\n\",\n \"```\\n\",\n \" | |\\n\",\n \" - | O | -\\n\",\n \"_____|_____|_____\\n\",\n \" | |\\n\",\n \" X | O | X\\n\",\n \"_____|_____|_____\\n\",\n \" | |\\n\",\n \" - | X | -\\n\",\n \" | |\\n\",\n \"```\\n\",\n \"\\n\",\n \"```\\n\",\n \" | |\\n\",\n \" O | O | -\\n\",\n \"_____|_____|_____\\n\",\n \" | |\\n\",\n \" X | O | X\\n\",\n \"_____|_____|_____\\n\",\n \" | |\\n\",\n \" - | X | -\\n\",\n \" | |\\n\",\n \"```\\n\",\n \"\\n\",\n \"```\\n\",\n \" | |\\n\",\n \" O | O | X\\n\",\n \"_____|_____|_____\\n\",\n \" | |\\n\",\n \" X | O | X\\n\",\n \"_____|_____|_____\\n\",\n \" | |\\n\",\n \" - | X | -\\n\",\n \" | |\\n\",\n \"```\\n\",\n \"\\n\",\n \"```\\n\",\n \" | |\\n\",\n \" O | O | X\\n\",\n \"_____|_____|_____\\n\",\n \" | |\\n\",\n \" X | O | X\\n\",\n \"_____|_____|_____\\n\",\n \" | |\\n\",\n \" - | X | O\\n\",\n \" | |\\n\",\n \"```\\n\",\n \"\\n\",\n \"Final reward: 1.0, length: 8.0\\n\",\n \"\\n\",\n \"
\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"Notice that our trained agent plays as player 2 (O) and wins! The agent has learned the game rules through trial and error, understanding that three consecutive O marks lead to victory.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": \"It is easily possible to make the trained agent play against itself. Try this as an exercise!\"\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"While the trained agent plays well against a random opponent, it's still far from perfect play. The next step would be to implement self-play training, similar to AlphaZero, where the agent continuously improves by playing against increasingly stronger versions of itself.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Summary\\n\",\n \"\\n\",\n \"In this tutorial, we demonstrated how to use Tianshou for training a single agent in a multi-agent reinforcement learning setting. Key takeaways:\\n\",\n \"\\n\",\n \"1. **MARL Paradigms**: Tianshou supports simultaneous, cyclic, and conditional move scenarios\\n\",\n \"2. **Abstraction**: Multi-agent problems can be converted to single-agent RL through clever state augmentation\\n\",\n \"3. **PettingZoo Integration**: Seamless compatibility with PettingZoo environments via `PettingZooEnv`\\n\",\n \"4. **Algorithm Management**: `MultiAgentOffPolicyAlgorithm` handles agent coordination and data distribution\\n\",\n \"5. **Flexible Framework**: Easy to extend from single-agent training to more complex multi-agent scenarios\\n\",\n \"\\n\",\n \"Tianshou provides a flexible and intuitive framework for reinforcement learning. Experiment with different architectures, training regimes, and opponent strategies to build even more capable agents!\"\n ]\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.8.0\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 4\n}\n" }, { "path": "docs/04_benchmarks/benchmarks.rst", "content": "Benchmarks\n==========\n\nTianshou's algorithm implementations lead to state-of-the-art results on standard benchmarks.\n\nAn efficient parallel implementation for\nevaluating algorithms on mujoco or atari is in `benchmark/run_benchmark.py`. It can easily be adapted for custom\nbenchmarks as well. The reported results are thus completely reproducible.\n\nThe evaluation code uses Tianshou's integration with the `rliable `_ framework,\nwhich supports best practices for trustworthy RL evaluation.\nEach experiment is conducted under 5 random seeds, we report the interquartile mean (IQM) and 95% confidence intervals over these seeds.\n\n.. raw:: html\n\n
\n \n
\n
\n
\n
\n" }, { "path": "docs/05_developer_guide/developer_guide.md", "content": "# Developer Guide\n\nThe section addresses developers of Tianshou, providing information for \nboth casual contributors and maintainers alike.\n\n\n## Python Virtual Environment\n\nTianshou is built and managed by [poetry](https://python-poetry.org/). \n\nThe development environment uses Python 3.11.\n\nTo install all relevant requirements (as well as Tianshou itself in editable mode)\nyou can simply call\n\n poetry install --with dev\n\n```{important}\nDepending on your setup, you may need to create and activate an empty virtual environment\nusing the right Python version beforehand. For instance, to do this with conda, use:\n\n conda create -n tianshou python=3.11\n conda activate tianshou\n```\n\n\n## Code Style and Auto-Formatting\n\nWhen editing code in Tianshou, strive for **local consistency**, i.e.\nadhere to the style already present in the codebase.\n\nTianshou uses an auto-formatting for consistency.\nTo apply it, call\n\n poe format\n\nTo check whether your formatting is compliant without applying the\nauto-formatter, call\n\n poe lint\n\n\n## Type Checking\n\nWe use [mypy](https://github.com/python/mypy/) to perform static type analysis. \nTo check typing, run\n\n poe type-check\n\n\n## Tests\n\n### Running the Test Suite Locally\n\nTianshou uses pytest. Tests are located in `./test`.\n\nTo run the full set of tests locally, run\n\n poe test\n\n### Determinism Tests\n\nWe implemented **determinism tests** for Tianshou's algorithms, which allow us to determine\nwhether algorithms still compute exactly the same results even after large refactorings.\nThese tests are applied by\n\n 1. creating a behavior snapshot in the old code branch before the changes and then\n 2. running the respective determinism test in the new branch to ensure that the behavior is the same.\n\nUnfortunately, full determinism is difficult to achieve across different platforms and even different\nmachines using the same platform an Python environment.\nTherefore, these tests are not carried out in the CI pipeline.\nInstead, it is up to the developer to run them locally and check the results whenever a change\nis made to the codebase that could affect algorithm behavior.\n\nTechnically, the two steps are handled by setting static flags in class `AlgorithmDeterminismTest` and then\nrunning either the full test suite or a specific determinism test (`test_*_determinism`, e.g. `test_ddpg_determinism`)\nin the two branches to be compared.\n\n 1. On the old branch: (Temporarily) set `ENABLED=True` and `FORCE_SNAPSHOT_UPDATE=True` and run the test(s).\n 2. On the new branch: (Temporarily) set `ENABLED=True` and `FORCE_SNAPSHOT_UPDATE=False` and run the test(s).\n 3. Inspect the test results; find a summary in `determinism_tests.log`\n\n### Tests in CI (GitHub Actions)\n\nCI tests will extensively test Tianshou's functionality in multiple environments.\n\nIn particular, we test\n * on Ubuntu (full functionality tested)\n * **py_pinned**: using the pinned development environment (Python 3.11, known versions of all dependencies)\n * **py_latest**: using a more recent Python version with the newest set of compatible dependencies (automatically resolved)\n * on Windows and macOS (core functionality tested)\n\n\n#### Principle of Maximum Compatibility\n\nThe idea behind testing with dynamically resolved dependencies is that we want to maximize the applicability \nof Tianshou: For important dependencies that could conflict with environments used by our users, **we do not restrict the version of a dependency unless there is a known incompatibility.**\n\nIf incompatibilities should arise (e.g. by the \"py_latest\" test failing), we either \n * resolve them by making the code compatible with both old and new versions OR\n * add an upper bound to our dependency declarations (excluding the incompatible versions) and release a new \n version of Tianshou to make these exclusions explicit.\n\n\n## High-Level API\n\nThe high-level API provides a declarative, user-friendly interface for setting up reinforcement learning experiments. From a library developer's perspective, it is important that this API be clearly structured and maintainable. This section explains the architectural principles and how to extend the API to support new algorithms.\n\n### Core Abstractions\n\nThe high-level API is built around a clear separation of concerns:\n\n**Parameter Classes** are dataclasses (inheriting from `Params`) that represent algorithm-specific configuration. \nThey capture hyperparameters in a high-level, user-friendly form. \nBecause the high-level interface must abstract away from low-level details, parameters may need transformation before being passed to policy classes. \nThis is handled via `ParamTransformer` instances, which successively transform the parameter dictionary representation. \nTo maintain clarity and reduce coupling, parameter transformers are co-located with the parameters they affect. \nThe system uses inheritance and mixins extensively to reduce duplication while maintaining flexibility.\n\n**Factories** embody the principle of declarative configuration. \nBecause object creation may depend on other objects that don't yet exist at configuration time (e.g., neural networks depend on environment properties), \nthe API transitions from objects to factories. \nKey factory types include:\n- `EnvFactory` for creating training, test, and watch environments\n- `AgentFactory` as the central factory that creates policies, trainers, and collectors\n- Various specialized factories for optimizers, actors, critics, noise, distributions, learning rate schedulers, and policy wrappers\n\n**Algorithm Factories** (subclasses of `AlgorithmFactory`) are the core components responsible for orchestrating the creation of all algorithm-specific objects. \nThey handle the creation of neural network architectures, apply parameter transformations, instantiate policies, and create trainers with appropriate collectors. \nTo support a new algorithm, this is the primary extension point.\n\n**Experiment Builders** (subclasses of `ExperimentBuilder`) provide the user-facing interface following the builder pattern. \nThey contain sensible defaults while allowing customization through fluent `with_*` methods. \nBuilder mixins provide composable functionality for common patterns (e.g., actor/critic configuration), avoiding code duplication across algorithm-specific implementations.\n\n### Supporting a New Algorithm\n\nExtending the high-level API to support a new algorithm involves creating three main components:\n\n**Parameter Class**: Define a dataclass in `tianshou/highlevel/params/algorithm_params.py` that inherits from appropriate base classes and mixins. \nThe choice of base class depends on the algorithm's architecture (actor-critic, single network, etc.) and learning paradigm (on-policy, off-policy). \nOverride `_get_param_transformers()` to specify how high-level parameters should be transformed for the low-level policy API.\nCommon transformers handle optimizer creation, noise instantiation, and environment-dependent parameter resolution.\n\n**Algorithm Factory**: Implement a subclass of `AlgorithmFactory` in `tianshou/highlevel/algorithm.py`. \nIn most cases, inherit from existing base factories like `ActorCriticOnPolicyAlgorithmFactory`, `ActorCriticOffPolicyAlgorithmFactory`, \nor `DiscreteCriticOnlyOffPolicyAlgorithmFactory`, which handle common creation patterns. \nThe primary requirement is implementing `_get_algorithm_class()` to return the appropriate algorithm class. \nFor algorithms with non-standard requirements, override `_create_algorithm()`, `_create_kwargs()`, etc. to customize the instantiation logic.\n\n**Experiment Builder**: Add a builder class in `tianshou/highlevel/experiment.py` that inherits from `OnPolicyExperimentBuilder` or `OffPolicyExperimentBuilder` \nalong with appropriate mixins. The mixins provide standard functionality for configuring actors and critics \n(single critic, dual critics, critic ensembles, parameter sharing patterns, etc.). \nThe main responsibility is implementing `_create_algorithm_factory()` to instantiate the algorithm factory with appropriate parameters and network factories. \nOptionally provide `with_*` methods for algorithm-specific configuration.\n\nExport the new classes in `tianshou/highlevel/__init__.py` to make them available to users.\n\n### Design Principles\n\nThe architecture follows several key principles:\n\n**Separation of Concerns**: Configuration is cleanly separated from implementation. \nThe transformation system bridges these layers while maintaining independence.\n\n**Declarative Configuration**: Factories enable a declarative style where experiments are defined by what should be created rather than imperative steps. \nThis makes experiments easily serializable and reproducible.\n\n**Composition and Inheritance**: Mixins and inheritance reduce code duplication. \nCommon functionality is factored into reusable components while maintaining flexibility for algorithm-specific requirements.\n\n**Progressive Disclosure**: The API provides sensible defaults for simple use cases while allowing deep customization when needed. \nUsers can progress from simple configurations to advanced setups without fighting the abstractions.\n\n**Co-location**: Related code is kept together. Parameter transformers are defined near the parameters they transform, \nmaintaining clarity about dependencies and making the codebase easier to navigate.\n\n**Type Safety**: Extensive use of generics and type hints ensures that type checkers can catch configuration errors at development time rather than runtime.\n\n\n## Documentation\n\nDocumentation is in the `docs/` directory, using Markdown (`.md`), ReStructuredText (`.rst`) and notebook files. \n`index.rst` is the main page. \n\nAPI References are automatically generated by [Sphinx](http://www.sphinx-doc.org/en/stable/) according to the outlines under `docs/api/` and should be modified when any code changes.\n\nTo compile documentation into webpage, run\n\n poe doc-build\n\nThe generated webpages can subsequently be found in `docs/_build` and can be viewed with any browser.\n\n### Verifications\n\nWe have several automated verification methods for documentation:\n\n1. pydocstyle (as part of ruff): tests all docstring under `tianshou/`;\n\n2. doc8 (as part of ruff): tests ReStructuredText format;\n\n3. sphinx spelling and test: test if there is any error/warning when generating front-end html documentation.\n\n\n## Creating a Release\n\nTo release a new version on PyPI,\n\n * set the version to be released in `tianshou/__init__.py` and in `pyproject.toml`, creating a commit\n * tag the commit with the version (using the format `v1.2.3`)\n * push the commit (`git push`) and the tag (`git push --tags`)\n * create a new release on GitHub based on the tag; this will trigger the release job for PyPI.\n\nIn the past, we provided releases to conda-forge as well, but this is currently not maintained.\n" }, { "path": "docs/06_contributors/contributors.rst", "content": "Contributors\n============\n\nWe always welcome contributions to help make Tianshou better!\nTianshou was originally created by the `THU-ML Group `_ at Tsinghua University.\n\nToday, it is backed by the `appliedAI Institute for Europe `_,\na non-profit organization committed to making Tianshou the go-to resource for reinforcement learning research and development,\nguaranteeing its long-term maintenance and support.\n\nThe original creator Jiayi Weng (`Trinkle23897 `_) continues\nto be involved in Tianshou development.\n\nThe current core developers, who are behind the v1.0 and v2.0 releases of Tianshou, are:\n\n* Dominik Jain (`opcode81 `_)\n* Michael Panchenko (`MischaPanch `_)\n\nAn incomplete list of early contributors is:\n\n* Alexis Duburcq (`duburcqa `_)\n* Kaichao You (`youkaichao `_)\n* Huayu Chen (`ChenDRAG `_)\n* Yi Su (`nuance1979 `_)\n\nYou can find more information about contributors `here `_.\n" }, { "path": "docs/_config.yml", "content": "# Book settings\n# Learn more at https://jupyterbook.org/customize/config.html\n\n#######################################################################################\n# A default configuration that will be loaded for all jupyter books\n# Users are expected to override these values in their own `_config.yml` file.\n# This is also the \"master list\" of all allowed keys and values.\n\n#######################################################################################\n# Book settings\ntitle : Tianshou Documentation # The title of the book. Will be placed in the left navbar.\nauthor : Tianshou contributors # The author of the book\ncopyright : \"2020, Tianshou contributors.\" # Copyright year to be placed in the footer\nlogo : _static/images/tianshou-logo.png # A path to the book logo\n# Patterns to skip when building the book. Can be glob-style (e.g. \"*skip.ipynb\")\nexclude_patterns : ['**.ipynb_checkpoints', '.DS_Store', 'Thumbs.db', '_build', 'jupyter_execute', '.jupyter_cache', '.pytest_cache', 'docs/autogen_rst.py', 'docs/create_toc.py']\n# Auto-exclude files not in the toc\nonly_build_toc_files : false\n\n#######################################################################################\n# Execution settings\nexecute:\n execute_notebooks : cache # Whether to execute notebooks at build time. Must be one of (\"auto\", \"force\", \"cache\", \"off\")\n cache : \"\" # A path to the jupyter cache that will be used to store execution artifacts. Defaults to `_build/.jupyter_cache/`\n exclude_patterns : [] # A list of patterns to *skip* in execution (e.g. a notebook that takes a really long time)\n timeout : -1 # The maximum time (in seconds) each notebook cell is allowed to run.\n run_in_temp : false # If `True`, then a temporary directory will be created and used as the command working directory (cwd),\n # otherwise the notebook's parent directory will be the cwd.\n allow_errors : false # If `False`, when a code cell raises an error the execution is stopped, otherwise all cells are always run.\n stderr_output : show # One of 'show', 'remove', 'remove-warn', 'warn', 'error', 'severe'\n\n#######################################################################################\n# Parse and render settings\nparse:\n myst_enable_extensions: # default extensions to enable in the myst parser. See https://myst-parser.readthedocs.io/en/latest/using/syntax-optional.html\n - amsmath\n - colon_fence\n # - deflist\n - dollarmath\n - html_admonition\n # - html_image\n - linkify\n # - replacements\n # - smartquotes\n - substitution\n - tasklist\n myst_url_schemes: [ mailto, http, https ] # URI schemes that will be recognised as external URLs in Markdown links\n myst_dmath_double_inline: true # Allow display math ($$) within an inline context\n\n#######################################################################################\n# HTML-specific settings\nhtml:\n favicon : \"_static/images/tianshou-favicon.png\" # A path to a favicon image\n use_edit_page_button : false # Whether to add an \"edit this page\" button to pages. If `true`, repository information in repository: must be filled in\n use_repository_button : false # Whether to add a link to your repository button\n use_issues_button : false # Whether to add an \"open an issue\" button\n use_multitoc_numbering : true # Continuous numbering across parts/chapters\n extra_footer : \"\"\n google_analytics_id : \"\" # A GA id that can be used to track book views.\n home_page_in_navbar : true # Whether to include your home page in the left Navigation Bar\n baseurl : \"https://tianshou.readthedocs.io/en/master/\"\n analytics:\n\n comments:\n hypothesis : false\n utterances : false\n announcement : \"\" # A banner announcement at the top of the site.\n\n#######################################################################################\n# LaTeX-specific settings\nlatex:\n latex_engine : pdflatex # one of 'pdflatex', 'xelatex' (recommended for unicode), 'luatex', 'platex', 'uplatex'\n use_jupyterbook_latex : true # use sphinx-jupyterbook-latex for pdf builds as default\n targetname : book.tex\n# Add a bibtex file so that we can create citations\nbibtex_bibfiles:\n - refs.bib\n\n#######################################################################################\n# Launch button settings\nlaunch_buttons:\n notebook_interface : classic # The interface interactive links will activate [\"classic\", \"jupyterlab\"]\n binderhub_url : \"\" # The URL of the BinderHub (e.g., https://mybinder.org)\n jupyterhub_url : \"\" # The URL of the JupyterHub (e.g., https://datahub.berkeley.edu)\n thebe : false # Add a thebe button to pages (requires the repository to run on Binder)\n colab_url : \"https://colab.research.google.com\"\n\nrepository:\n url : https://github.com/thu-ml/tianshou # The URL to your book's repository\n path_to_book : docs # A path to your book's folder, relative to the repository root.\n branch : master # Which branch of the repository should be used when creating links\n\n#######################################################################################\n# Advanced and power-user settings\nsphinx:\n extra_extensions :\n - sphinx.ext.autodoc\n - sphinx.ext.viewcode\n - sphinx_toolbox.more_autodoc.sourcelink\n - sphinxcontrib.spelling\n - sphinxcontrib.mermaid\n local_extensions : # A list of local extensions to load by sphinx specified by \"name: path\" items\n recursive_update : false # A boolean indicating whether to overwrite the Sphinx config (true) or recursively update (false)\n config : # key-value pairs to directly over-ride the Sphinx configuration\n autodoc_typehints_format: \"short\"\n autodoc_member_order: \"bysource\"\n autodoc_mock_imports:\n # mock imports for optional dependencies (e.g. dependencies of atari/atari_wrapper)\n - cv2\n autoclass_content: \"both\"\n autodoc_default_options:\n show-inheritance: True\n html_js_files:\n# We have to list them explicitly because they need to be loaded in a specific order\n - js/vega@5.js\n - js/vega-lite@5.js\n - js/vega-embed@5.js\n autodoc_show_sourcelink: True\n add_module_names: False\n github_username: thu-ml\n github_repository: tianshou\n python_use_unqualified_type_names: True\n nb_mime_priority_overrides: [\n [ 'html', 'application/vnd.jupyter.widget-view+json', 10 ],\n [ 'html', 'application/javascript', 20 ],\n [ 'html', 'text/html', 30 ],\n [ 'html', 'text/latex', 40 ],\n [ 'html', 'image/svg+xml', 50 ],\n [ 'html', 'image/png', 60 ],\n [ 'html', 'image/jpeg', 70 ],\n [ 'html', 'text/markdown', 80 ],\n [ 'html', 'text/plain', 90 ],\n [ 'spelling', 'application/vnd.jupyter.widget-view+json', 10 ],\n [ 'spelling', 'application/javascript', 20 ],\n [ 'spelling', 'text/html', 30 ],\n [ 'spelling', 'text/latex', 40 ],\n [ 'spelling', 'image/svg+xml', 50 ],\n [ 'spelling', 'image/png', 60 ],\n [ 'spelling', 'image/jpeg', 70 ],\n [ 'spelling', 'text/markdown', 80 ],\n [ 'spelling', 'text/plain', 90 ],\n ]\n mathjax_path: https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js\n myst_fence_as_directive: [\"mermaid\"]\n mathjax3_config:\n loader: { load: [ '[tex]/configmacros' ] }\n tex:\n packages: { '[+]': [ 'configmacros' ] }\n macros:\n vect: [\"{\\\\mathbf{\\\\boldsymbol{#1}} }\", 1]\n E: \"{\\\\mathbb{E}}\"\n P: \"{\\\\mathbb{P}}\"\n R: \"{\\\\mathbb{R}}\"\n abs: [\"{\\\\left| #1 \\\\right|}\", 1]\n simpl: [\"{\\\\Delta^{#1} }\", 1]\n amax: \"{\\\\text{argmax}}\"\n" }, { "path": "docs/_static/css/style.css", "content": "body {\n font-family: \"Lato\",\"proxima-nova\",\"Helvetica Neue\",Arial,sans-serif;\n}\n\n/* Default header fonts are ugly */\nh1, h2, .rst-content .toctree-wrapper p.caption, h3, h4, h5, h6, legend, p.caption {\n font-family: \"Lato\",\"proxima-nova\",\"Helvetica Neue\",Arial,sans-serif;\n}\n\n/* Use white for docs background */\n.wy-side-nav-search {\n background-color: #fff;\n}\n\n.wy-nav-content {\n max-width: 1200px !important;\n}\n\n.wy-nav-content-wrap, .wy-menu li.current > a {\n background-color: #fff;\n}\n\n.wy-side-nav-search>a img.logo {\n width: 80%;\n margin-top: 10px;\n}\n\n@media screen and (min-width: 1400px) {\n .wy-nav-content-wrap {\n background-color: #fff;\n }\n\n .wy-nav-content {\n background-color: #fff;\n }\n}\n\n/* Fixes for mobile */\n.wy-nav-top {\n background-color: #fff;\n /*background-image: url('../images/tianshou-logo.png');*/\n background-repeat: no-repeat;\n background-position: center;\n padding: 0;\n margin: 0.4045em 0.809em;\n color: #333;\n}\n\n.wy-nav-top > a {\n display: none;\n}\n\n@media (min-width: 960px) {\n .bd-page-width {\n max-width: none !important;\n }\n}\n\n\n@media screen and (max-width: 768px) {\n .wy-side-nav-search>a img.logo {\n height: 60px;\n }\n}\n\n/* This is needed to ensure that logo above search scales properly */\n.wy-side-nav-search a {\n display: block;\n}\n\n/* This ensures that multiple constructors will remain in separate lines. */\n.rst-content dl:not(.docutils) dt {\n display: table;\n}\n\n/* Use our red for literals (it's very similar to the original color) */\n.rst-content tt.literal, .rst-content tt.literal, .rst-content code.literal {\n color: #4692BC;\n}\n\n.rst-content tt.xref, a .rst-content tt, .rst-content tt.xref,\n.rst-content code.xref, a .rst-content tt, a .rst-content code {\n color: #404040;\n}\n\n/* Change link colors (except for the menu) */\n\na {\n color: #4692BC;\n}\n\na:hover {\n color: #4692BC;\n}\n\n\na:visited {\n color: #4692BC;\n}\n\n.wy-menu a {\n color: #b3b3b3;\n}\n\n.wy-menu a:hover {\n color: #b3b3b3;\n}\n\n/* Default footer text is quite big */\nfooter {\n font-size: 80%;\n}\n\nfooter .rst-footer-buttons {\n font-size: 125%; /* revert footer settings - 1/80% = 125% */\n}\n\nfooter p {\n font-size: 100%;\n}\n\n.ethical-rtd {\n display: none;\n}\n\n.ethical-fixedfooter {\n display: none;\n}\n\n.ethical-content {\n display: none;\n}\n\n/* For hidden headers that appear in TOC tree */\n/* see http://stackoverflow.com/a/32363545/3343043 */\n.rst-content .hidden-section {\n display: none;\n}\n\nnav .hidden-section {\n display: inherit;\n}\n\n.wy-side-nav-search>div.version {\n color: #000;\n}\n" }, { "path": "docs/_static/js/benchmark.js", "content": "var mujoco_envs = [\n \"Ant-v4\",\n];\n\nvar atari_envs = [\n // \"PongNoFrameskip-v4\",\n // \"BreakoutNoFrameskip-v4\",\n // \"EnduroNoFrameskip-v4\",\n // \"QbertNoFrameskip-v4\",\n // \"MsPacmanNoFrameskip-v4\",\n // \"SeaquestNoFrameskip-v4\",\n // \"SpaceInvadersNoFrameskip-v4\",\n];\n\nfunction getDataSource(selectEnv, dirName) {\n return {\n // Paths are relative to the only file using this script, which is docs/04_benchmarks/benchmarks.rst\n $schema: \"https://vega.github.io/schema/vega-lite/v5.json\",\n data: {\n url: \"../_static/js/\" + dirName + \"/benchmark/\" + selectEnv + \"/results.json\"\n },\n mark: \"line\",\n height: 400,\n width: 800,\n params: [{name: \"Range\", value: 1000000, bind: {input: \"range\", min: 10000, max: 10000000}}],\n transform: [\n {calculate: \"datum.iqm_confidence_interval[0]\", as: \"iqm_confidence_lower\"},\n {calculate: \"datum.iqm_confidence_interval[1]\", as: \"iqm_confidence_upper\"},\n {calculate: \"datum.iqm\", as: \"tooltip_str\"},\n {filter: \"datum.env_step <= Range\"},\n ],\n encoding: {\n color: {\"field\": \"agent\", \"type\": \"nominal\"},\n x: {field: \"env_step\", type: \"quantitative\", title: \"Env step\"},\n },\n layer: [{\n \"encoding\": {\n \"opacity\": {\"value\": 0.3},\n \"y\": {\n \"field\": \"iqm_confidence_lower\",\n \"type\": \"quantitative\",\n },\n \"y2\": {\"field\": \"iqm_confidence_upper\"},\n tooltip: [\n {field: \"env_step\", type: \"quantitative\", title: \"Env step\"},\n {field: \"agent\", type: \"nominal\"},\n {field: \"tooltip_str\", type: \"nominal\", title: \"Return\"},\n ]\n },\n \"mark\": \"area\"\n }, {\n \"encoding\": {\n \"y\": {\n \"title\": \"Return Interquartile Mean (5 seeds)\",\n \"field\": \"iqm\",\n \"type\": \"quantitative\"\n }\n },\n \"mark\": \"line\"\n }]\n };\n}\n\nfunction showMujocoResults(elem) {\n const selectEnv = elem.value || mujoco_envs[0];\n const dataSource = getDataSource(selectEnv, \"mujoco\");\n vegaEmbed(\"#vis-mujoco\", dataSource);\n}\n\nfunction showAtariResults(elem) {\n const selectEnv = elem.value || atari_envs[0];\n const dataSource = getDataSource(selectEnv, \"atari\");\n vegaEmbed(\"#vis-atari\", dataSource);\n}\n\n\n\ndocument.addEventListener('DOMContentLoaded', function() {\n var envMujocoSelect = $(\"#env-mujoco\");\n if (envMujocoSelect.length) {\n $.each(mujoco_envs, function(idx, env) {envMujocoSelect.append($(\"\").val(env).html(env));})\n showMujocoResults(envMujocoSelect);\n }\n var envAtariSelect = $(\"#env-atari\");\n if (envAtariSelect.length) {\n $.each(atari_envs, function(idx, env) {envAtariSelect.append($(\"\").val(env).html(env));})\n showAtariResults(envAtariSelect);\n }\n});\n" }, { "path": "docs/_static/js/copybutton.js", "content": "document.addEventListener('DOMContentLoaded', function() {\n /* Add a [>>>] button on the top-right corner of code samples to hide\n * the >>> and ... prompts and the output and thus make the code\n * copyable. */\n var div = $('.highlight-python .highlight,' +\n '.highlight-python3 .highlight,' +\n '.highlight-pycon .highlight,' +\n '.highlight-default .highlight');\n var pre = div.find('pre');\n\n // get the styles from the current theme\n pre.parent().parent().css('position', 'relative');\n var hide_text = 'Hide the prompts and output';\n var show_text = 'Show the prompts and output';\n var border_width = pre.css('border-top-width');\n var border_style = pre.css('border-top-style');\n var border_color = pre.css('border-top-color');\n var button_styles = {\n 'cursor':'pointer', 'position': 'absolute', 'top': '0', 'right': '0',\n 'border-color': border_color, 'border-style': border_style,\n 'border-width': border_width, 'color': border_color, 'text-size': '75%',\n 'font-family': 'monospace', 'padding-left': '0.2em', 'padding-right': '0.2em',\n 'border-radius': '0 3px 0 0'\n }\n\n // create and add the button to all the code blocks that contain >>>\n div.each(function(index) {\n var jthis = $(this);\n if (jthis.find('.gp').length > 0) {\n var button = $('>>>');\n button.css(button_styles)\n button.attr('title', hide_text);\n button.data('hidden', 'false');\n jthis.prepend(button);\n }\n // tracebacks (.gt) contain bare text elements that need to be\n // wrapped in a span to work with .nextUntil() (see later)\n jthis.find('pre:has(.gt)').contents().filter(function() {\n return ((this.nodeType == 3) && (this.data.trim().length > 0));\n }).wrap('');\n });\n\n // define the behavior of the button when it's clicked\n $('.copybutton').click(function(e){\n e.preventDefault();\n var button = $(this);\n if (button.data('hidden') === 'false') {\n // hide the code output\n button.parent().find('.go, .gp, .gt').hide();\n button.next('pre').find('.gt').nextUntil('.gp, .go').css('visibility', 'hidden');\n button.css('text-decoration', 'line-through');\n button.attr('title', show_text);\n button.data('hidden', 'true');\n } else {\n // show the code output\n button.parent().find('.go, .gp, .gt').show();\n button.next('pre').find('.gt').nextUntil('.gp, .go').css('visibility', 'visible');\n button.css('text-decoration', 'none');\n button.attr('title', hide_text);\n button.data('hidden', 'false');\n }\n });\n});\n\n" }, { "path": "docs/_static/js/mujoco/benchmark/Ant-v4/results.json", "content": "[{\"env_step\":0.0,\"rew\":-42.10716438293457,\"rew_std\":24.039944270941408,\"iqm\":-40.97881825764974,\"iqm_confidence_interval\":[-70.32092030843098,-16.246893564860027],\"agent\":\"NPG\"},{\"env_step\":30720.0,\"rew\":-28.9559645652771,\"rew_std\":24.61339003090938,\"iqm\":-31.551718711853027,\"iqm_confidence_interval\":[-56.042564392089844,0.8885383605957031],\"agent\":\"NPG\"},{\"env_step\":61440.0,\"rew\":-11.55083270072937,\"rew_std\":6.110236122564615,\"iqm\":-12.476377169291178,\"iqm_confidence_interval\":[-17.99208927154541,-3.9522757530212402],\"agent\":\"NPG\"},{\"env_step\":92160.0,\"rew\":18.439939069747926,\"rew_std\":10.787288878835925,\"iqm\":18.934740384419758,\"iqm_confidence_interval\":[5.5553921063741045,30.87356185913086],\"agent\":\"NPG\"},{\"env_step\":122880.0,\"rew\":45.153709411621094,\"rew_std\":10.71529835147344,\"iqm\":42.32269287109375,\"iqm_confidence_interval\":[35.65855153401693,59.28111139933268],\"agent\":\"NPG\"},{\"env_step\":153600.0,\"rew\":88.77803573608398,\"rew_std\":23.054480042346643,\"iqm\":89.32733408610027,\"iqm_confidence_interval\":[60.482452392578125,112.37182362874348],\"agent\":\"NPG\"},{\"env_step\":184320.0,\"rew\":121.12091217041015,\"rew_std\":30.036537018727188,\"iqm\":120.6772689819336,\"iqm_confidence_interval\":[85.26046752929688,155.5569814046224],\"agent\":\"NPG\"},{\"env_step\":215040.0,\"rew\":165.61034240722657,\"rew_std\":32.24095152542376,\"iqm\":168.1877187093099,\"iqm_confidence_interval\":[126.84097290039062,202.26611836751303],\"agent\":\"NPG\"},{\"env_step\":245760.0,\"rew\":264.8183166503906,\"rew_std\":68.57585730650109,\"iqm\":258.05474853515625,\"iqm_confidence_interval\":[192.57537333170572,347.3933512369792],\"agent\":\"NPG\"},{\"env_step\":276480.0,\"rew\":316.14655151367185,\"rew_std\":61.54133584779444,\"iqm\":329.4031473795573,\"iqm_confidence_interval\":[237.4843953450521,373.4185078938802],\"agent\":\"NPG\"},{\"env_step\":307200.0,\"rew\":394.7198486328125,\"rew_std\":37.27962477000651,\"iqm\":394.5103759765625,\"iqm_confidence_interval\":[352.14459228515625,439.00913492838544],\"agent\":\"NPG\"},{\"env_step\":337920.0,\"rew\":526.7607849121093,\"rew_std\":89.47713614019793,\"iqm\":527.4289042154948,\"iqm_confidence_interval\":[423.1936747233073,624.7489217122396],\"agent\":\"NPG\"},{\"env_step\":368640.0,\"rew\":579.3040954589844,\"rew_std\":115.02891711430003,\"iqm\":588.0213419596354,\"iqm_confidence_interval\":[440.58685302734375,703.8764444986979],\"agent\":\"NPG\"},{\"env_step\":399360.0,\"rew\":724.5651000976562,\"rew_std\":105.69306907584757,\"iqm\":722.1583048502604,\"iqm_confidence_interval\":[604.6950073242188,845.4350992838541],\"agent\":\"NPG\"},{\"env_step\":430080.0,\"rew\":863.718896484375,\"rew_std\":188.1599330780623,\"iqm\":876.3805948893229,\"iqm_confidence_interval\":[638.7650349934896,1069.964619954427],\"agent\":\"NPG\"},{\"env_step\":460800.0,\"rew\":1041.2549926757813,\"rew_std\":166.71506155109884,\"iqm\":1064.1121215820312,\"iqm_confidence_interval\":[832.3571980794271,1219.2620849609375],\"agent\":\"NPG\"},{\"env_step\":491520.0,\"rew\":1216.867138671875,\"rew_std\":161.53753506344896,\"iqm\":1205.9603678385417,\"iqm_confidence_interval\":[1039.2998860677083,1407.0541178385417],\"agent\":\"NPG\"},{\"env_step\":522240.0,\"rew\":1292.862158203125,\"rew_std\":136.2897089330508,\"iqm\":1266.5582682291667,\"iqm_confidence_interval\":[1160.0815022786458,1464.893798828125],\"agent\":\"NPG\"},{\"env_step\":552960.0,\"rew\":1349.3367919921875,\"rew_std\":216.29374512926137,\"iqm\":1342.1337076822917,\"iqm_confidence_interval\":[1095.9574788411458,1596.9417724609375],\"agent\":\"NPG\"},{\"env_step\":583680.0,\"rew\":1286.1487548828125,\"rew_std\":325.0510298184137,\"iqm\":1309.6995442708333,\"iqm_confidence_interval\":[923.5931396484375,1622.830078125],\"agent\":\"NPG\"},{\"env_step\":614400.0,\"rew\":1501.306494140625,\"rew_std\":135.64654587687215,\"iqm\":1486.9634602864583,\"iqm_confidence_interval\":[1367.4110107421875,1674.4379069010417],\"agent\":\"NPG\"},{\"env_step\":645120.0,\"rew\":1566.505615234375,\"rew_std\":207.3775273975989,\"iqm\":1595.12890625,\"iqm_confidence_interval\":[1309.4273681640625,1787.2928873697917],\"agent\":\"NPG\"},{\"env_step\":675840.0,\"rew\":1648.5074951171875,\"rew_std\":167.47017729280728,\"iqm\":1674.1391194661458,\"iqm_confidence_interval\":[1451.5147705078125,1829.447021484375],\"agent\":\"NPG\"},{\"env_step\":706560.0,\"rew\":1826.64931640625,\"rew_std\":210.24840626670206,\"iqm\":1826.7261962890625,\"iqm_confidence_interval\":[1598.8306477864583,2082.1640625],\"agent\":\"NPG\"},{\"env_step\":737280.0,\"rew\":1870.0795654296876,\"rew_std\":116.74630498193775,\"iqm\":1893.3362630208333,\"iqm_confidence_interval\":[1723.8335367838542,1989.8927408854167],\"agent\":\"NPG\"},{\"env_step\":768000.0,\"rew\":1872.2861328125,\"rew_std\":85.56317841655876,\"iqm\":1872.4123942057292,\"iqm_confidence_interval\":[1775.9134521484375,1975.1356201171875],\"agent\":\"NPG\"},{\"env_step\":798720.0,\"rew\":2006.606005859375,\"rew_std\":258.3459636539935,\"iqm\":1976.9451904296875,\"iqm_confidence_interval\":[1724.650146484375,2327.4354654947915],\"agent\":\"NPG\"},{\"env_step\":829440.0,\"rew\":1947.3549560546876,\"rew_std\":218.90451416506937,\"iqm\":1959.3906656901042,\"iqm_confidence_interval\":[1693.5713704427083,2196.7334798177085],\"agent\":\"NPG\"},{\"env_step\":860160.0,\"rew\":2106.194873046875,\"rew_std\":117.60013614027503,\"iqm\":2067.5992838541665,\"iqm_confidence_interval\":[2008.2084147135417,2256.8001302083335],\"agent\":\"NPG\"},{\"env_step\":890880.0,\"rew\":2071.628271484375,\"rew_std\":242.5118564930354,\"iqm\":2149.12744140625,\"iqm_confidence_interval\":[1755.46728515625,2272.0733235677085],\"agent\":\"NPG\"},{\"env_step\":921600.0,\"rew\":2366.916162109375,\"rew_std\":289.17324371255546,\"iqm\":2325.9986979166665,\"iqm_confidence_interval\":[2074.3031412760415,2734.9005533854165],\"agent\":\"NPG\"},{\"env_step\":952320.0,\"rew\":2205.8991455078126,\"rew_std\":422.5754930755271,\"iqm\":2282.3590494791665,\"iqm_confidence_interval\":[1670.6670735677083,2625.806884765625],\"agent\":\"NPG\"},{\"env_step\":983040.0,\"rew\":2261.9572265625,\"rew_std\":509.6706066309041,\"iqm\":2190.5692138671875,\"iqm_confidence_interval\":[1760.44482421875,2919.6507161458335],\"agent\":\"NPG\"},{\"env_step\":1013760.0,\"rew\":2433.71572265625,\"rew_std\":493.58342825899086,\"iqm\":2341.9806315104165,\"iqm_confidence_interval\":[1942.987060546875,3055.232666015625],\"agent\":\"NPG\"},{\"env_step\":1044480.0,\"rew\":2279.4240234375,\"rew_std\":646.6016713206892,\"iqm\":2132.0210774739585,\"iqm_confidence_interval\":[1656.076904296875,3117.5598958333335],\"agent\":\"NPG\"},{\"env_step\":1075200.0,\"rew\":2621.333935546875,\"rew_std\":588.2199189530081,\"iqm\":2415.3409016927085,\"iqm_confidence_interval\":[2146.8382161458335,3389.3638509114585],\"agent\":\"NPG\"},{\"env_step\":1105920.0,\"rew\":2538.0572998046873,\"rew_std\":516.4605893509663,\"iqm\":2412.5494791666665,\"iqm_confidence_interval\":[2085.3553059895835,3158.64501953125],\"agent\":\"NPG\"},{\"env_step\":1136640.0,\"rew\":2696.96259765625,\"rew_std\":536.0556880955188,\"iqm\":2612.5350748697915,\"iqm_confidence_interval\":[2147.63671875,3383.4264322916665],\"agent\":\"NPG\"},{\"env_step\":1167360.0,\"rew\":2600.9685546875,\"rew_std\":502.4029487892295,\"iqm\":2644.7255859375,\"iqm_confidence_interval\":[2001.6358235677083,3170.20849609375],\"agent\":\"NPG\"},{\"env_step\":1198080.0,\"rew\":2765.19111328125,\"rew_std\":577.5845895357381,\"iqm\":2693.3756510416665,\"iqm_confidence_interval\":[2225.9715169270835,3483.7542317708335],\"agent\":\"NPG\"},{\"env_step\":1228800.0,\"rew\":2881.88994140625,\"rew_std\":527.5027862353074,\"iqm\":2920.961669921875,\"iqm_confidence_interval\":[2219.39794921875,3442.47705078125],\"agent\":\"NPG\"},{\"env_step\":1259520.0,\"rew\":2948.0620849609377,\"rew_std\":594.5163611818551,\"iqm\":3007.4317220052085,\"iqm_confidence_interval\":[2209.598876953125,3598.4187825520835],\"agent\":\"NPG\"},{\"env_step\":1290240.0,\"rew\":3064.065625,\"rew_std\":440.83851262024234,\"iqm\":2960.6610514322915,\"iqm_confidence_interval\":[2634.0765787760415,3625.16015625],\"agent\":\"NPG\"},{\"env_step\":1320960.0,\"rew\":3280.083984375,\"rew_std\":707.1134563719127,\"iqm\":3348.0303548177085,\"iqm_confidence_interval\":[2432.7224934895835,4057.8509114583335],\"agent\":\"NPG\"},{\"env_step\":1351680.0,\"rew\":3205.251513671875,\"rew_std\":316.46324853306436,\"iqm\":3163.8677571614585,\"iqm_confidence_interval\":[2869.4271647135415,3602.5740559895835],\"agent\":\"NPG\"},{\"env_step\":1382400.0,\"rew\":3444.421728515625,\"rew_std\":436.2355752828923,\"iqm\":3369.95654296875,\"iqm_confidence_interval\":[3007.6968587239585,4004.7176920572915],\"agent\":\"NPG\"},{\"env_step\":1413120.0,\"rew\":3011.940234375,\"rew_std\":634.0937754760221,\"iqm\":3051.2601725260415,\"iqm_confidence_interval\":[2281.888427734375,3751.8732096354165],\"agent\":\"NPG\"},{\"env_step\":1443840.0,\"rew\":3466.91298828125,\"rew_std\":572.6590330179324,\"iqm\":3594.772216796875,\"iqm_confidence_interval\":[2714.4599609375,3963.66162109375],\"agent\":\"NPG\"},{\"env_step\":1474560.0,\"rew\":3302.862255859375,\"rew_std\":762.3598545058242,\"iqm\":3239.7962239583335,\"iqm_confidence_interval\":[2556.1107584635415,4242.556315104167],\"agent\":\"NPG\"},{\"env_step\":1505280.0,\"rew\":3478.742822265625,\"rew_std\":557.1591110802749,\"iqm\":3495.2234700520835,\"iqm_confidence_interval\":[2808.22705078125,4099.184000651042],\"agent\":\"NPG\"},{\"env_step\":1536000.0,\"rew\":3490.6912109375,\"rew_std\":444.31353563177555,\"iqm\":3609.8375651041665,\"iqm_confidence_interval\":[2909.4974772135415,3896.072998046875],\"agent\":\"NPG\"},{\"env_step\":1566720.0,\"rew\":3646.065234375,\"rew_std\":773.6771731564615,\"iqm\":3672.367431640625,\"iqm_confidence_interval\":[2757.34033203125,4560.120768229167],\"agent\":\"NPG\"},{\"env_step\":1597440.0,\"rew\":3487.64560546875,\"rew_std\":466.1504258357491,\"iqm\":3650.9379069010415,\"iqm_confidence_interval\":[2892.069580078125,3872.5625],\"agent\":\"NPG\"},{\"env_step\":1628160.0,\"rew\":3813.238232421875,\"rew_std\":195.45548449408994,\"iqm\":3848.3933919270835,\"iqm_confidence_interval\":[3557.7486979166665,3992.6292317708335],\"agent\":\"NPG\"},{\"env_step\":1658880.0,\"rew\":3634.64384765625,\"rew_std\":357.5938916670007,\"iqm\":3626.664306640625,\"iqm_confidence_interval\":[3249.5238444010415,4047.3922526041665],\"agent\":\"NPG\"},{\"env_step\":1689600.0,\"rew\":3831.650048828125,\"rew_std\":373.6238249300375,\"iqm\":3942.2557779947915,\"iqm_confidence_interval\":[3367.9471842447915,4165.78466796875],\"agent\":\"NPG\"},{\"env_step\":1720320.0,\"rew\":3441.63681640625,\"rew_std\":107.20122493250447,\"iqm\":3469.4847005208335,\"iqm_confidence_interval\":[3299.667724609375,3530.85693359375],\"agent\":\"NPG\"},{\"env_step\":1751040.0,\"rew\":4085.60537109375,\"rew_std\":380.79098883303703,\"iqm\":4027.6840006510415,\"iqm_confidence_interval\":[3706.8218587239585,4541.901041666667],\"agent\":\"NPG\"},{\"env_step\":1781760.0,\"rew\":3908.0486328125,\"rew_std\":531.1436678021352,\"iqm\":3868.1513671875,\"iqm_confidence_interval\":[3365.0298665364585,4572.987141927083],\"agent\":\"NPG\"},{\"env_step\":1812480.0,\"rew\":3977.9677734375,\"rew_std\":334.31211529476235,\"iqm\":3977.7456868489585,\"iqm_confidence_interval\":[3583.3868001302085,4351.19580078125],\"agent\":\"NPG\"},{\"env_step\":1843200.0,\"rew\":3542.310986328125,\"rew_std\":385.2524129298876,\"iqm\":3677.203857421875,\"iqm_confidence_interval\":[3076.446533203125,3828.6275227864585],\"agent\":\"NPG\"},{\"env_step\":1873920.0,\"rew\":4163.43583984375,\"rew_std\":492.598368484635,\"iqm\":4214.400065104167,\"iqm_confidence_interval\":[3540.4664713541665,4677.500813802083],\"agent\":\"NPG\"},{\"env_step\":1904640.0,\"rew\":4181.5763671875,\"rew_std\":554.0300039532967,\"iqm\":4131.946533203125,\"iqm_confidence_interval\":[3620.850341796875,4883.752278645833],\"agent\":\"NPG\"},{\"env_step\":1935360.0,\"rew\":4026.918310546875,\"rew_std\":583.5711637398964,\"iqm\":4088.448486328125,\"iqm_confidence_interval\":[3314.7400716145835,4676.281412760417],\"agent\":\"NPG\"},{\"env_step\":1966080.0,\"rew\":4025.4564453125,\"rew_std\":454.36050405051066,\"iqm\":3980.6463216145835,\"iqm_confidence_interval\":[3539.2442220052085,4587.694498697917],\"agent\":\"NPG\"},{\"env_step\":1996800.0,\"rew\":4082.299462890625,\"rew_std\":567.8325714042335,\"iqm\":4076.89306640625,\"iqm_confidence_interval\":[3463.0868326822915,4747.28076171875],\"agent\":\"NPG\"},{\"env_step\":2027520.0,\"rew\":3936.75859375,\"rew_std\":526.8825948319501,\"iqm\":3821.395263671875,\"iqm_confidence_interval\":[3427.847412109375,4580.795654296875],\"agent\":\"NPG\"},{\"env_step\":2058240.0,\"rew\":4397.854345703125,\"rew_std\":587.2665622699866,\"iqm\":4586.83935546875,\"iqm_confidence_interval\":[3680.1486002604165,4879.48291015625],\"agent\":\"NPG\"},{\"env_step\":2088960.0,\"rew\":4056.78583984375,\"rew_std\":715.0343288525298,\"iqm\":4241.515950520833,\"iqm_confidence_interval\":[3110.4615885416665,4684.402180989583],\"agent\":\"NPG\"},{\"env_step\":2119680.0,\"rew\":4201.72041015625,\"rew_std\":434.0396009798354,\"iqm\":4308.550944010417,\"iqm_confidence_interval\":[3637.28857421875,4594.300618489583],\"agent\":\"NPG\"},{\"env_step\":2150400.0,\"rew\":3781.59541015625,\"rew_std\":346.4414147044312,\"iqm\":3839.78369140625,\"iqm_confidence_interval\":[3346.6486002604165,4140.158121744792],\"agent\":\"NPG\"},{\"env_step\":2181120.0,\"rew\":3943.3373046875,\"rew_std\":610.6881082446964,\"iqm\":3962.8224283854165,\"iqm_confidence_interval\":[3243.5960286458335,4605.189534505208],\"agent\":\"NPG\"},{\"env_step\":2211840.0,\"rew\":4266.37197265625,\"rew_std\":681.0246834918399,\"iqm\":4187.938720703125,\"iqm_confidence_interval\":[3565.35400390625,5115.400065104167],\"agent\":\"NPG\"},{\"env_step\":2242560.0,\"rew\":4126.19501953125,\"rew_std\":487.938527027499,\"iqm\":4053.7718098958335,\"iqm_confidence_interval\":[3646.608642578125,4739.072591145833],\"agent\":\"NPG\"},{\"env_step\":2273280.0,\"rew\":4228.608203125,\"rew_std\":549.7404756899181,\"iqm\":4230.651936848958,\"iqm_confidence_interval\":[3596.397216796875,4873.546712239583],\"agent\":\"NPG\"},{\"env_step\":2304000.0,\"rew\":4057.852783203125,\"rew_std\":602.2027305602329,\"iqm\":4093.2787272135415,\"iqm_confidence_interval\":[3350.7344563802085,4752.5224609375],\"agent\":\"NPG\"},{\"env_step\":2334720.0,\"rew\":4452.448046875,\"rew_std\":531.1544726008665,\"iqm\":4385.053548177083,\"iqm_confidence_interval\":[3901.7164713541665,5128.13818359375],\"agent\":\"NPG\"},{\"env_step\":2365440.0,\"rew\":4271.966796875,\"rew_std\":437.37979559110437,\"iqm\":4285.06005859375,\"iqm_confidence_interval\":[3740.1287434895835,4744.523600260417],\"agent\":\"NPG\"},{\"env_step\":2396160.0,\"rew\":4347.164404296875,\"rew_std\":223.5063369913802,\"iqm\":4431.816080729167,\"iqm_confidence_interval\":[4060.6329752604165,4509.202473958333],\"agent\":\"NPG\"},{\"env_step\":2426880.0,\"rew\":4481.2849609375,\"rew_std\":346.00707012552084,\"iqm\":4513.728515625,\"iqm_confidence_interval\":[4048.8810221354165,4857.212076822917],\"agent\":\"NPG\"},{\"env_step\":2457600.0,\"rew\":4564.82314453125,\"rew_std\":637.0825921129947,\"iqm\":4590.755533854167,\"iqm_confidence_interval\":[3819.2635091145835,5305.782552083333],\"agent\":\"NPG\"},{\"env_step\":2488320.0,\"rew\":4520.0427734375,\"rew_std\":436.4336347118552,\"iqm\":4590.888834635417,\"iqm_confidence_interval\":[3962.0810546875,4967.877278645833],\"agent\":\"NPG\"},{\"env_step\":2519040.0,\"rew\":4359.85390625,\"rew_std\":287.77420296772794,\"iqm\":4469.613932291667,\"iqm_confidence_interval\":[4007.3859049479165,4579.832682291667],\"agent\":\"NPG\"},{\"env_step\":2549760.0,\"rew\":4693.39501953125,\"rew_std\":396.4874171852096,\"iqm\":4781.868326822917,\"iqm_confidence_interval\":[4193.547037760417,5072.042643229167],\"agent\":\"NPG\"},{\"env_step\":2580480.0,\"rew\":4582.3765625,\"rew_std\":421.0727810737856,\"iqm\":4619.725748697917,\"iqm_confidence_interval\":[4058.74072265625,5022.57373046875],\"agent\":\"NPG\"},{\"env_step\":2611200.0,\"rew\":4703.40751953125,\"rew_std\":210.00483804774578,\"iqm\":4754.280924479167,\"iqm_confidence_interval\":[4431.056315104167,4899.427571614583],\"agent\":\"NPG\"},{\"env_step\":2641920.0,\"rew\":4900.25234375,\"rew_std\":290.4812495566195,\"iqm\":4872.10693359375,\"iqm_confidence_interval\":[4618.145182291667,5264.572916666667],\"agent\":\"NPG\"},{\"env_step\":2672640.0,\"rew\":4474.5326171875,\"rew_std\":269.2334315633679,\"iqm\":4443.0693359375,\"iqm_confidence_interval\":[4191.284016927083,4815.879231770833],\"agent\":\"NPG\"},{\"env_step\":2703360.0,\"rew\":4658.04375,\"rew_std\":336.6230138689072,\"iqm\":4545.173502604167,\"iqm_confidence_interval\":[4387.645182291667,5095.66064453125],\"agent\":\"NPG\"},{\"env_step\":2734080.0,\"rew\":4878.24658203125,\"rew_std\":334.79437839842825,\"iqm\":4920.015950520833,\"iqm_confidence_interval\":[4472.2958984375,5213.7880859375],\"agent\":\"NPG\"},{\"env_step\":2764800.0,\"rew\":4760.90791015625,\"rew_std\":214.13738081026182,\"iqm\":4727.073567708333,\"iqm_confidence_interval\":[4549.66845703125,5038.178548177083],\"agent\":\"NPG\"},{\"env_step\":2795520.0,\"rew\":4684.60634765625,\"rew_std\":77.41656898240524,\"iqm\":4706.542317708333,\"iqm_confidence_interval\":[4591.1083984375,4749.69140625],\"agent\":\"NPG\"},{\"env_step\":2826240.0,\"rew\":4977.3126953125,\"rew_std\":182.15891965403532,\"iqm\":4958.58740234375,\"iqm_confidence_interval\":[4791.443684895833,5209.349609375],\"agent\":\"NPG\"},{\"env_step\":2856960.0,\"rew\":4734.32392578125,\"rew_std\":102.18784300235029,\"iqm\":4715.5224609375,\"iqm_confidence_interval\":[4632.307942708333,4864.966959635417],\"agent\":\"NPG\"},{\"env_step\":2887680.0,\"rew\":5007.94814453125,\"rew_std\":206.2397802762669,\"iqm\":5018.766438802083,\"iqm_confidence_interval\":[4766.834309895833,5246.975911458333],\"agent\":\"NPG\"},{\"env_step\":2918400.0,\"rew\":5097.96357421875,\"rew_std\":163.5823288450074,\"iqm\":5032.53125,\"iqm_confidence_interval\":[4977.635091145833,5299.5048828125],\"agent\":\"NPG\"},{\"env_step\":2949120.0,\"rew\":4788.7689453125,\"rew_std\":376.3747806878339,\"iqm\":4787.894368489583,\"iqm_confidence_interval\":[4337.253580729167,5212.905110677083],\"agent\":\"NPG\"},{\"env_step\":2979840.0,\"rew\":4699.702587890625,\"rew_std\":583.2143349211142,\"iqm\":4777.147786458333,\"iqm_confidence_interval\":[3975.2060546875,5330.618489583333],\"agent\":\"NPG\"},{\"env_step\":3010560.0,\"rew\":4728.6564453125,\"rew_std\":402.81542872103023,\"iqm\":4677.973795572917,\"iqm_confidence_interval\":[4308.068196614583,5241.167154947917],\"agent\":\"NPG\"},{\"env_step\":3041280.0,\"rew\":4987.98134765625,\"rew_std\":409.05004072355825,\"iqm\":4909.922526041667,\"iqm_confidence_interval\":[4619.532063802083,5518.312662760417],\"agent\":\"NPG\"},{\"env_step\":3072000.0,\"rew\":4764.5708984375,\"rew_std\":448.03125410300396,\"iqm\":4817.55908203125,\"iqm_confidence_interval\":[4203.057291666667,5225.935709635417],\"agent\":\"NPG\"},{\"env_step\":0.0,\"rew\":-42.10716438293457,\"rew_std\":24.039944270941408,\"iqm\":-40.97881825764974,\"iqm_confidence_interval\":[-70.32092030843098,-16.246893564860027],\"agent\":\"PPO\"},{\"env_step\":30720.0,\"rew\":-37.372220039367676,\"rew_std\":19.689550303988273,\"iqm\":-35.171875,\"iqm_confidence_interval\":[-62.41522725423177,-17.062697728474934],\"agent\":\"PPO\"},{\"env_step\":61440.0,\"rew\":-21.24895796775818,\"rew_std\":14.76330539847627,\"iqm\":-22.682526270548504,\"iqm_confidence_interval\":[-36.67624537150065,-2.578148365020752],\"agent\":\"PPO\"},{\"env_step\":92160.0,\"rew\":-13.753775215148925,\"rew_std\":14.13574584193426,\"iqm\":-19.474328994750977,\"iqm_confidence_interval\":[-23.108850479125977,3.3546009063720703],\"agent\":\"PPO\"},{\"env_step\":122880.0,\"rew\":-12.556952285766602,\"rew_std\":13.641631192489111,\"iqm\":-8.645663897196451,\"iqm_confidence_interval\":[-29.560285568237305,-0.892268180847168],\"agent\":\"PPO\"},{\"env_step\":153600.0,\"rew\":5.548850393295288,\"rew_std\":5.419250191615881,\"iqm\":6.114233175913493,\"iqm_confidence_interval\":[-0.7145605087280273,10.98777691523234],\"agent\":\"PPO\"},{\"env_step\":184320.0,\"rew\":4.924838566780091,\"rew_std\":8.117651806112379,\"iqm\":6.176069458325704,\"iqm_confidence_interval\":[-5.165180961290996,13.583230336507162],\"agent\":\"PPO\"},{\"env_step\":215040.0,\"rew\":17.765496063232423,\"rew_std\":13.144212017376775,\"iqm\":20.197986602783203,\"iqm_confidence_interval\":[1.0707556406656902,30.401254653930664],\"agent\":\"PPO\"},{\"env_step\":245760.0,\"rew\":31.95562801361084,\"rew_std\":12.791377589350782,\"iqm\":35.37232271830241,\"iqm_confidence_interval\":[15.888467152913412,43.269569396972656],\"agent\":\"PPO\"},{\"env_step\":276480.0,\"rew\":63.5789077758789,\"rew_std\":14.202631661044354,\"iqm\":66.58460362752278,\"iqm_confidence_interval\":[44.90095901489258,76.55831654866536],\"agent\":\"PPO\"},{\"env_step\":307200.0,\"rew\":86.53251419067382,\"rew_std\":29.695438915186116,\"iqm\":80.42616017659505,\"iqm_confidence_interval\":[55.82790883382162,122.85542551676433],\"agent\":\"PPO\"},{\"env_step\":337920.0,\"rew\":102.9419448852539,\"rew_std\":36.432540278237525,\"iqm\":93.47763570149739,\"iqm_confidence_interval\":[70.79665120442708,150.19486490885416],\"agent\":\"PPO\"},{\"env_step\":368640.0,\"rew\":138.24014587402343,\"rew_std\":17.957499408815412,\"iqm\":137.02452596028647,\"iqm_confidence_interval\":[118.35025533040364,158.88599141438803],\"agent\":\"PPO\"},{\"env_step\":399360.0,\"rew\":166.96195373535156,\"rew_std\":10.492143797386662,\"iqm\":165.95062255859375,\"iqm_confidence_interval\":[156.25365702311197,179.8980712890625],\"agent\":\"PPO\"},{\"env_step\":430080.0,\"rew\":188.4481964111328,\"rew_std\":21.446438325420985,\"iqm\":185.77357482910156,\"iqm_confidence_interval\":[165.51288350423178,215.12000528971353],\"agent\":\"PPO\"},{\"env_step\":460800.0,\"rew\":254.69054565429687,\"rew_std\":40.37535841417855,\"iqm\":246.8503214518229,\"iqm_confidence_interval\":[216.7704823811849,307.5547281901042],\"agent\":\"PPO\"},{\"env_step\":491520.0,\"rew\":313.9736083984375,\"rew_std\":13.84002768360938,\"iqm\":313.45168050130206,\"iqm_confidence_interval\":[298.9747823079427,330.33074951171875],\"agent\":\"PPO\"},{\"env_step\":522240.0,\"rew\":321.7760498046875,\"rew_std\":27.74167703701409,\"iqm\":319.7954610188802,\"iqm_confidence_interval\":[292.18919881184894,356.52332560221356],\"agent\":\"PPO\"},{\"env_step\":552960.0,\"rew\":377.2718872070312,\"rew_std\":82.12267171002435,\"iqm\":350.90550740559894,\"iqm_confidence_interval\":[306.2198486328125,479.6610412597656],\"agent\":\"PPO\"},{\"env_step\":583680.0,\"rew\":390.71156616210936,\"rew_std\":35.88038655768206,\"iqm\":389.2766825358073,\"iqm_confidence_interval\":[350.6089782714844,434.0635986328125],\"agent\":\"PPO\"},{\"env_step\":614400.0,\"rew\":470.87006225585935,\"rew_std\":100.31065042455326,\"iqm\":447.6538899739583,\"iqm_confidence_interval\":[374.72276814778644,600.7935587565104],\"agent\":\"PPO\"},{\"env_step\":645120.0,\"rew\":522.5170715332031,\"rew_std\":101.98108030878707,\"iqm\":549.3511149088541,\"iqm_confidence_interval\":[398.22825113932294,611.7529296875],\"agent\":\"PPO\"},{\"env_step\":675840.0,\"rew\":539.6492065429687,\"rew_std\":127.94532386643006,\"iqm\":551.1155395507812,\"iqm_confidence_interval\":[388.29457600911456,673.1277262369791],\"agent\":\"PPO\"},{\"env_step\":706560.0,\"rew\":600.2794616699218,\"rew_std\":82.50651436358652,\"iqm\":606.5103149414062,\"iqm_confidence_interval\":[508.68638102213544,695.8967692057291],\"agent\":\"PPO\"},{\"env_step\":737280.0,\"rew\":742.03623046875,\"rew_std\":90.94995649924844,\"iqm\":761.3958740234375,\"iqm_confidence_interval\":[631.2799479166666,832.5506184895834],\"agent\":\"PPO\"},{\"env_step\":768000.0,\"rew\":615.0272155761719,\"rew_std\":58.96221259019708,\"iqm\":632.1763916015625,\"iqm_confidence_interval\":[541.9233805338541,666.8138631184896],\"agent\":\"PPO\"},{\"env_step\":798720.0,\"rew\":657.0923034667969,\"rew_std\":205.12095729630033,\"iqm\":611.0579833984375,\"iqm_confidence_interval\":[448.13185628255206,900.5207722981771],\"agent\":\"PPO\"},{\"env_step\":829440.0,\"rew\":808.9402954101563,\"rew_std\":93.11438452829933,\"iqm\":812.4151611328125,\"iqm_confidence_interval\":[706.8053385416666,906.13330078125],\"agent\":\"PPO\"},{\"env_step\":860160.0,\"rew\":690.5447570800782,\"rew_std\":178.52351903516816,\"iqm\":656.3587036132812,\"iqm_confidence_interval\":[508.01845296223956,916.5430094401041],\"agent\":\"PPO\"},{\"env_step\":890880.0,\"rew\":824.253515625,\"rew_std\":147.86348202648495,\"iqm\":773.6602986653646,\"iqm_confidence_interval\":[698.3820393880209,1007.6150512695312],\"agent\":\"PPO\"},{\"env_step\":921600.0,\"rew\":1026.9066162109375,\"rew_std\":139.86281711892363,\"iqm\":971.0055745442709,\"iqm_confidence_interval\":[931.3086954752604,1202.254659016927],\"agent\":\"PPO\"},{\"env_step\":952320.0,\"rew\":983.5424194335938,\"rew_std\":139.64380326887348,\"iqm\":989.9229329427084,\"iqm_confidence_interval\":[820.3156941731771,1147.0453287760417],\"agent\":\"PPO\"},{\"env_step\":983040.0,\"rew\":1006.61904296875,\"rew_std\":130.47452131573982,\"iqm\":983.8915812174479,\"iqm_confidence_interval\":[880.1797892252604,1169.6405843098958],\"agent\":\"PPO\"},{\"env_step\":1013760.0,\"rew\":1023.1817016601562,\"rew_std\":229.86015328346903,\"iqm\":970.7048950195312,\"iqm_confidence_interval\":[800.6198527018229,1319.8238118489583],\"agent\":\"PPO\"},{\"env_step\":1044480.0,\"rew\":1235.4329956054687,\"rew_std\":308.5856035365616,\"iqm\":1193.8307698567708,\"iqm_confidence_interval\":[917.0994059244791,1627.1339111328125],\"agent\":\"PPO\"},{\"env_step\":1075200.0,\"rew\":1283.767919921875,\"rew_std\":369.94054944625407,\"iqm\":1235.2249755859375,\"iqm_confidence_interval\":[910.7933349609375,1758.7515462239583],\"agent\":\"PPO\"},{\"env_step\":1105920.0,\"rew\":1306.4467895507812,\"rew_std\":304.9427535917461,\"iqm\":1280.5807291666667,\"iqm_confidence_interval\":[970.6359049479166,1681.99755859375],\"agent\":\"PPO\"},{\"env_step\":1136640.0,\"rew\":1421.9716064453125,\"rew_std\":408.54132443351,\"iqm\":1296.5053304036458,\"iqm_confidence_interval\":[1087.0143229166667,1953.9322102864583],\"agent\":\"PPO\"},{\"env_step\":1167360.0,\"rew\":1590.911376953125,\"rew_std\":294.2090689637845,\"iqm\":1670.9098307291667,\"iqm_confidence_interval\":[1203.5606689453125,1852.9765218098958],\"agent\":\"PPO\"},{\"env_step\":1198080.0,\"rew\":1585.925048828125,\"rew_std\":281.95354699539524,\"iqm\":1636.9428304036458,\"iqm_confidence_interval\":[1220.1016438802083,1863.3033447265625],\"agent\":\"PPO\"},{\"env_step\":1228800.0,\"rew\":1488.6505126953125,\"rew_std\":242.07114369351723,\"iqm\":1458.1148681640625,\"iqm_confidence_interval\":[1236.4732259114583,1792.5924886067708],\"agent\":\"PPO\"},{\"env_step\":1259520.0,\"rew\":1674.939990234375,\"rew_std\":158.53917724624395,\"iqm\":1687.1951090494792,\"iqm_confidence_interval\":[1475.0849202473958,1840.28564453125],\"agent\":\"PPO\"},{\"env_step\":1290240.0,\"rew\":1613.6635986328124,\"rew_std\":226.93365135714689,\"iqm\":1641.7682291666667,\"iqm_confidence_interval\":[1333.0029296875,1854.9034016927083],\"agent\":\"PPO\"},{\"env_step\":1320960.0,\"rew\":1690.7473876953125,\"rew_std\":228.711704729816,\"iqm\":1671.047119140625,\"iqm_confidence_interval\":[1444.6800537109375,1975.0709228515625],\"agent\":\"PPO\"},{\"env_step\":1351680.0,\"rew\":1637.27607421875,\"rew_std\":326.39842414211853,\"iqm\":1689.7603759765625,\"iqm_confidence_interval\":[1214.1621500651042,1938.1739501953125],\"agent\":\"PPO\"},{\"env_step\":1382400.0,\"rew\":1875.2035888671876,\"rew_std\":117.24078150507262,\"iqm\":1870.533447265625,\"iqm_confidence_interval\":[1749.0623779296875,2015.3394368489583],\"agent\":\"PPO\"},{\"env_step\":1413120.0,\"rew\":1630.64677734375,\"rew_std\":278.33546365831046,\"iqm\":1666.410888671875,\"iqm_confidence_interval\":[1284.4798583984375,1922.7447102864583],\"agent\":\"PPO\"},{\"env_step\":1443840.0,\"rew\":2303.1189697265627,\"rew_std\":332.04121880276756,\"iqm\":2275.5123697916665,\"iqm_confidence_interval\":[1921.7642415364583,2683.3470052083335],\"agent\":\"PPO\"},{\"env_step\":1474560.0,\"rew\":1948.187353515625,\"rew_std\":450.3484385458742,\"iqm\":1980.5458984375,\"iqm_confidence_interval\":[1415.1756998697917,2445.75146484375],\"agent\":\"PPO\"},{\"env_step\":1505280.0,\"rew\":1660.3700439453125,\"rew_std\":81.58748049080477,\"iqm\":1657.2394612630208,\"iqm_confidence_interval\":[1576.0192057291667,1758.386474609375],\"agent\":\"PPO\"},{\"env_step\":1536000.0,\"rew\":2080.3249755859374,\"rew_std\":516.867492908338,\"iqm\":2198.355753580729,\"iqm_confidence_interval\":[1411.3249918619792,2566.95361328125],\"agent\":\"PPO\"},{\"env_step\":1566720.0,\"rew\":1932.98095703125,\"rew_std\":497.55494819068974,\"iqm\":1932.7194010416667,\"iqm_confidence_interval\":[1333.7371419270833,2499.1190592447915],\"agent\":\"PPO\"},{\"env_step\":1597440.0,\"rew\":2143.2340576171873,\"rew_std\":475.77974848071915,\"iqm\":2227.9281412760415,\"iqm_confidence_interval\":[1549.1127115885417,2640.3284505208335],\"agent\":\"PPO\"},{\"env_step\":1628160.0,\"rew\":2226.6865478515624,\"rew_std\":246.70248568804817,\"iqm\":2290.8931477864585,\"iqm_confidence_interval\":[1912.80712890625,2454.1283365885415],\"agent\":\"PPO\"},{\"env_step\":1658880.0,\"rew\":2434.57939453125,\"rew_std\":281.12032818486637,\"iqm\":2458.473388671875,\"iqm_confidence_interval\":[2097.693359375,2749.7991536458335],\"agent\":\"PPO\"},{\"env_step\":1689600.0,\"rew\":2254.104052734375,\"rew_std\":364.90063328970285,\"iqm\":2279.374267578125,\"iqm_confidence_interval\":[1812.7180989583333,2615.252197265625],\"agent\":\"PPO\"},{\"env_step\":1720320.0,\"rew\":2224.58935546875,\"rew_std\":301.9480649936515,\"iqm\":2230.579793294271,\"iqm_confidence_interval\":[1857.5064697265625,2564.9047037760415],\"agent\":\"PPO\"},{\"env_step\":1751040.0,\"rew\":2474.04462890625,\"rew_std\":413.29056303724695,\"iqm\":2463.1443684895835,\"iqm_confidence_interval\":[1996.7662760416667,2942.8408203125],\"agent\":\"PPO\"},{\"env_step\":1781760.0,\"rew\":2450.4150634765624,\"rew_std\":406.0064709999227,\"iqm\":2600.332763671875,\"iqm_confidence_interval\":[1916.1021321614583,2758.1326497395835],\"agent\":\"PPO\"},{\"env_step\":1812480.0,\"rew\":2615.113134765625,\"rew_std\":379.6628530641846,\"iqm\":2674.5680338541665,\"iqm_confidence_interval\":[2129.9462076822915,2991.6184895833335],\"agent\":\"PPO\"},{\"env_step\":1843200.0,\"rew\":2482.0716552734375,\"rew_std\":328.13810380085454,\"iqm\":2535.2766927083335,\"iqm_confidence_interval\":[2060.6763509114585,2815.9703776041665],\"agent\":\"PPO\"},{\"env_step\":1873920.0,\"rew\":2287.769189453125,\"rew_std\":273.181875960619,\"iqm\":2222.917724609375,\"iqm_confidence_interval\":[2027.02685546875,2637.3173014322915],\"agent\":\"PPO\"},{\"env_step\":1904640.0,\"rew\":2599.824365234375,\"rew_std\":228.66302607578032,\"iqm\":2573.5817057291665,\"iqm_confidence_interval\":[2358.0563151041665,2879.1910807291665],\"agent\":\"PPO\"},{\"env_step\":1935360.0,\"rew\":2541.87373046875,\"rew_std\":289.5021642360021,\"iqm\":2539.570556640625,\"iqm_confidence_interval\":[2208.8688151041665,2887.46533203125],\"agent\":\"PPO\"},{\"env_step\":1966080.0,\"rew\":2933.87001953125,\"rew_std\":365.7239536954544,\"iqm\":2887.910400390625,\"iqm_confidence_interval\":[2533.5679524739585,3381.1993001302085],\"agent\":\"PPO\"},{\"env_step\":1996800.0,\"rew\":2579.30927734375,\"rew_std\":454.20133503110765,\"iqm\":2520.7469889322915,\"iqm_confidence_interval\":[2141.8518880208335,3161.8784993489585],\"agent\":\"PPO\"},{\"env_step\":2027520.0,\"rew\":3025.467724609375,\"rew_std\":270.24624348081664,\"iqm\":3012.11279296875,\"iqm_confidence_interval\":[2737.0088704427085,3361.9505208333335],\"agent\":\"PPO\"},{\"env_step\":2058240.0,\"rew\":2684.42509765625,\"rew_std\":306.8236635672918,\"iqm\":2601.1487630208335,\"iqm_confidence_interval\":[2417.588134765625,3090.6845703125],\"agent\":\"PPO\"},{\"env_step\":2088960.0,\"rew\":2737.178173828125,\"rew_std\":260.7824921193103,\"iqm\":2758.1848958333335,\"iqm_confidence_interval\":[2414.929931640625,3024.1651204427085],\"agent\":\"PPO\"},{\"env_step\":2119680.0,\"rew\":2952.967919921875,\"rew_std\":372.3186731081476,\"iqm\":2970.3858235677085,\"iqm_confidence_interval\":[2496.8993326822915,3367.85791015625],\"agent\":\"PPO\"},{\"env_step\":2150400.0,\"rew\":2704.452392578125,\"rew_std\":463.98607495935727,\"iqm\":2612.8490397135415,\"iqm_confidence_interval\":[2269.6512858072915,3307.7020670572915],\"agent\":\"PPO\"},{\"env_step\":2181120.0,\"rew\":2952.671826171875,\"rew_std\":263.0708852230181,\"iqm\":3056.8150227864585,\"iqm_confidence_interval\":[2625.5342610677085,3149.9842122395835],\"agent\":\"PPO\"},{\"env_step\":2211840.0,\"rew\":2808.459326171875,\"rew_std\":238.9752354834229,\"iqm\":2726.9951985677085,\"iqm_confidence_interval\":[2604.6825358072915,3101.40673828125],\"agent\":\"PPO\"},{\"env_step\":2242560.0,\"rew\":3009.982470703125,\"rew_std\":74.9151516668644,\"iqm\":3012.4922688802085,\"iqm_confidence_interval\":[2919.9193522135415,3096.140625],\"agent\":\"PPO\"},{\"env_step\":2273280.0,\"rew\":2761.7705078125,\"rew_std\":360.90498272036405,\"iqm\":2695.267822265625,\"iqm_confidence_interval\":[2459.7607421875,3223.03955078125],\"agent\":\"PPO\"},{\"env_step\":2304000.0,\"rew\":3065.429833984375,\"rew_std\":208.58126876046154,\"iqm\":3072.5851236979165,\"iqm_confidence_interval\":[2825.865478515625,3308.800048828125],\"agent\":\"PPO\"},{\"env_step\":2334720.0,\"rew\":2497.6119140625,\"rew_std\":168.72569472467632,\"iqm\":2471.8560384114585,\"iqm_confidence_interval\":[2331.022705078125,2695.2312825520835],\"agent\":\"PPO\"},{\"env_step\":2365440.0,\"rew\":3119.83740234375,\"rew_std\":394.03295535098914,\"iqm\":3144.8118489583335,\"iqm_confidence_interval\":[2627.7822265625,3542.5327962239585],\"agent\":\"PPO\"},{\"env_step\":2396160.0,\"rew\":3456.946533203125,\"rew_std\":292.95038801934356,\"iqm\":3519.7478841145835,\"iqm_confidence_interval\":[3081.8028157552085,3747.7261555989585],\"agent\":\"PPO\"},{\"env_step\":2426880.0,\"rew\":2753.8327392578126,\"rew_std\":506.1009013588715,\"iqm\":2723.019775390625,\"iqm_confidence_interval\":[2205.009765625,3349.3614908854165],\"agent\":\"PPO\"},{\"env_step\":2457600.0,\"rew\":2759.991162109375,\"rew_std\":499.19339471250754,\"iqm\":2571.3480631510415,\"iqm_confidence_interval\":[2386.9571126302085,3371.4786783854165],\"agent\":\"PPO\"},{\"env_step\":2488320.0,\"rew\":3062.648388671875,\"rew_std\":378.59710060804554,\"iqm\":3073.1571451822915,\"iqm_confidence_interval\":[2607.640625,3467.515869140625],\"agent\":\"PPO\"},{\"env_step\":2519040.0,\"rew\":3111.23828125,\"rew_std\":287.9575490617171,\"iqm\":3045.4600423177085,\"iqm_confidence_interval\":[2838.7545572916665,3485.8863118489585],\"agent\":\"PPO\"},{\"env_step\":2549760.0,\"rew\":3244.82412109375,\"rew_std\":206.83376168878198,\"iqm\":3249.402587890625,\"iqm_confidence_interval\":[2996.6829427083335,3480.8203938802085],\"agent\":\"PPO\"},{\"env_step\":2580480.0,\"rew\":3003.4182861328127,\"rew_std\":664.6998400999946,\"iqm\":3071.9596354166665,\"iqm_confidence_interval\":[2204.2228190104165,3750.7945149739585],\"agent\":\"PPO\"},{\"env_step\":2611200.0,\"rew\":3196.064501953125,\"rew_std\":377.4709011433584,\"iqm\":3297.418212890625,\"iqm_confidence_interval\":[2721.66455078125,3556.1273600260415],\"agent\":\"PPO\"},{\"env_step\":2641920.0,\"rew\":2605.3752197265626,\"rew_std\":588.9940724955435,\"iqm\":2536.89013671875,\"iqm_confidence_interval\":[2026.8653157552083,3360.588623046875],\"agent\":\"PPO\"},{\"env_step\":2672640.0,\"rew\":2879.83369140625,\"rew_std\":273.4504838936323,\"iqm\":2911.3448079427085,\"iqm_confidence_interval\":[2535.531494140625,3139.8009440104165],\"agent\":\"PPO\"},{\"env_step\":2703360.0,\"rew\":3016.5560546875,\"rew_std\":229.28307374375265,\"iqm\":3019.5062662760415,\"iqm_confidence_interval\":[2774.4791666666665,3292.1170247395835],\"agent\":\"PPO\"},{\"env_step\":2734080.0,\"rew\":3156.365966796875,\"rew_std\":366.55058529979067,\"iqm\":3045.2474772135415,\"iqm_confidence_interval\":[2837.220947265625,3637.2925618489585],\"agent\":\"PPO\"},{\"env_step\":2764800.0,\"rew\":3243.116015625,\"rew_std\":318.76341846175626,\"iqm\":3214.4501953125,\"iqm_confidence_interval\":[2888.8876139322915,3635.6490071614585],\"agent\":\"PPO\"},{\"env_step\":2795520.0,\"rew\":3162.737255859375,\"rew_std\":421.73512918702005,\"iqm\":3265.7046712239585,\"iqm_confidence_interval\":[2640.34912109375,3538.4329427083335],\"agent\":\"PPO\"},{\"env_step\":2826240.0,\"rew\":2813.2316650390626,\"rew_std\":538.8788688188539,\"iqm\":2993.232421875,\"iqm_confidence_interval\":[2138.80712890625,3238.02294921875],\"agent\":\"PPO\"},{\"env_step\":2856960.0,\"rew\":2723.7482421875,\"rew_std\":395.313112852787,\"iqm\":2772.0569661458335,\"iqm_confidence_interval\":[2232.9578450520835,3151.04736328125],\"agent\":\"PPO\"},{\"env_step\":2887680.0,\"rew\":3243.996533203125,\"rew_std\":438.84849169788566,\"iqm\":3380.9866536458335,\"iqm_confidence_interval\":[2680.8663736979165,3632.7811686197915],\"agent\":\"PPO\"},{\"env_step\":2918400.0,\"rew\":3352.18662109375,\"rew_std\":178.99601980908236,\"iqm\":3343.7360026041665,\"iqm_confidence_interval\":[3150.2766927083335,3570.91748046875],\"agent\":\"PPO\"},{\"env_step\":2949120.0,\"rew\":3044.200390625,\"rew_std\":366.54644565146214,\"iqm\":2977.0995279947915,\"iqm_confidence_interval\":[2705.872314453125,3523.64599609375],\"agent\":\"PPO\"},{\"env_step\":2979840.0,\"rew\":3209.932763671875,\"rew_std\":189.78177916093884,\"iqm\":3224.5018717447915,\"iqm_confidence_interval\":[2990.1775716145835,3427.7244466145835],\"agent\":\"PPO\"},{\"env_step\":3010560.0,\"rew\":3192.97626953125,\"rew_std\":673.4347294627223,\"iqm\":3317.2999674479165,\"iqm_confidence_interval\":[2343.0367024739585,3888.066650390625],\"agent\":\"PPO\"},{\"env_step\":3041280.0,\"rew\":2891.0755859375,\"rew_std\":264.96420492417445,\"iqm\":2896.5139973958335,\"iqm_confidence_interval\":[2567.085693359375,3168.7027180989585],\"agent\":\"PPO\"},{\"env_step\":3072000.0,\"rew\":3297.96611328125,\"rew_std\":252.4458971635053,\"iqm\":3263.8701985677085,\"iqm_confidence_interval\":[3042.9195149739585,3622.0187174479165],\"agent\":\"PPO\"},{\"env_step\":0.0,\"rew\":19.122140550613402,\"rew_std\":10.828158316117733,\"iqm\":19.053614298502605,\"iqm_confidence_interval\":[6.762896378835042,31.858115514119465],\"agent\":\"REDQ\"},{\"env_step\":5000.0,\"rew\":-0.3874577522277832,\"rew_std\":20.51044985790059,\"iqm\":-3.4084525108337402,\"iqm_confidence_interval\":[-21.0011043548584,25.558574040730793],\"agent\":\"REDQ\"},{\"env_step\":10000.0,\"rew\":15.17558171749115,\"rew_std\":19.118557777633733,\"iqm\":7.209461688995361,\"iqm_confidence_interval\":[2.4170307318369546,39.32273292541504],\"agent\":\"REDQ\"},{\"env_step\":15000.0,\"rew\":18.561113548278808,\"rew_std\":26.933683712547285,\"iqm\":16.118902524312336,\"iqm_confidence_interval\":[-11.86785856882731,50.54333623250326],\"agent\":\"REDQ\"},{\"env_step\":20000.0,\"rew\":22.58060312271118,\"rew_std\":18.478246441131926,\"iqm\":22.68181037902832,\"iqm_confidence_interval\":[0.4586966832478841,42.824031829833984],\"agent\":\"REDQ\"},{\"env_step\":25000.0,\"rew\":26.83237566947937,\"rew_std\":22.788315952093132,\"iqm\":23.187583605448406,\"iqm_confidence_interval\":[2.362187226613363,53.76578903198242],\"agent\":\"REDQ\"},{\"env_step\":30000.0,\"rew\":26.75415654182434,\"rew_std\":14.504257444432223,\"iqm\":29.219275156656902,\"iqm_confidence_interval\":[8.868590831756592,42.14922841389974],\"agent\":\"REDQ\"},{\"env_step\":35000.0,\"rew\":40.10686206817627,\"rew_std\":28.254898689421985,\"iqm\":33.00629107157389,\"iqm_confidence_interval\":[14.200682322184244,75.1447016398112],\"agent\":\"REDQ\"},{\"env_step\":40000.0,\"rew\":29.391694402694704,\"rew_std\":26.29284086929632,\"iqm\":27.753348350524902,\"iqm_confidence_interval\":[2.39839760462443,62.38681411743164],\"agent\":\"REDQ\"},{\"env_step\":45000.0,\"rew\":34.10289249420166,\"rew_std\":14.799463210084305,\"iqm\":31.986361821492512,\"iqm_confidence_interval\":[19.76604715983073,51.56613032023112],\"agent\":\"REDQ\"},{\"env_step\":50000.0,\"rew\":36.32660884857178,\"rew_std\":15.576514826259867,\"iqm\":36.95574633280436,\"iqm_confidence_interval\":[19.14916229248047,54.83012898763021],\"agent\":\"REDQ\"},{\"env_step\":55000.0,\"rew\":42.851513671875,\"rew_std\":13.609421520237806,\"iqm\":43.765868504842125,\"iqm_confidence_interval\":[25.845146814982098,57.435323079427086],\"agent\":\"REDQ\"},{\"env_step\":60000.0,\"rew\":41.53638954162598,\"rew_std\":19.636276908871174,\"iqm\":41.28042538960775,\"iqm_confidence_interval\":[17.94518979390462,62.46479288736979],\"agent\":\"REDQ\"},{\"env_step\":65000.0,\"rew\":50.33994483947754,\"rew_std\":24.545305592826463,\"iqm\":49.94168599446615,\"iqm_confidence_interval\":[22.127840677897137,79.39129384358723],\"agent\":\"REDQ\"},{\"env_step\":70000.0,\"rew\":44.47986755371094,\"rew_std\":4.600869240164357,\"iqm\":45.67780685424805,\"iqm_confidence_interval\":[38.634847005208336,48.49294408162435],\"agent\":\"REDQ\"},{\"env_step\":75000.0,\"rew\":48.29617462158203,\"rew_std\":16.110098364439374,\"iqm\":48.11051813761393,\"iqm_confidence_interval\":[30.79833221435547,66.8612912495931],\"agent\":\"REDQ\"},{\"env_step\":80000.0,\"rew\":55.402624893188474,\"rew_std\":24.430260082246257,\"iqm\":51.397787729899086,\"iqm_confidence_interval\":[29.97335433959961,86.28074391682942],\"agent\":\"REDQ\"},{\"env_step\":85000.0,\"rew\":44.108120346069335,\"rew_std\":11.068213006744026,\"iqm\":44.7865244547526,\"iqm_confidence_interval\":[30.729891459147137,56.39498519897461],\"agent\":\"REDQ\"},{\"env_step\":90000.0,\"rew\":42.49583158493042,\"rew_std\":31.699443718350526,\"iqm\":41.404434521993004,\"iqm_confidence_interval\":[9.195992151896158,80.47208913167317],\"agent\":\"REDQ\"},{\"env_step\":95000.0,\"rew\":65.2559928894043,\"rew_std\":19.92327549779489,\"iqm\":66.64696884155273,\"iqm_confidence_interval\":[41.3400510152181,88.01953379313152],\"agent\":\"REDQ\"},{\"env_step\":100000.0,\"rew\":58.94149398803711,\"rew_std\":12.552099052111915,\"iqm\":55.4300905863444,\"iqm_confidence_interval\":[48.69604619344076,75.6355463663737],\"agent\":\"REDQ\"},{\"env_step\":105000.0,\"rew\":47.423474884033205,\"rew_std\":18.442554244270397,\"iqm\":52.15555318196615,\"iqm_confidence_interval\":[23.097124735514324,62.573177337646484],\"agent\":\"REDQ\"},{\"env_step\":110000.0,\"rew\":46.11401519775391,\"rew_std\":17.63670714798056,\"iqm\":49.090494791666664,\"iqm_confidence_interval\":[23.976512908935547,64.5013033548991],\"agent\":\"REDQ\"},{\"env_step\":115000.0,\"rew\":48.510765838623044,\"rew_std\":14.95405674973619,\"iqm\":49.95635096232096,\"iqm_confidence_interval\":[29.695638020833332,63.88478088378906],\"agent\":\"REDQ\"},{\"env_step\":120000.0,\"rew\":46.53772888183594,\"rew_std\":7.871587588416568,\"iqm\":46.81902313232422,\"iqm_confidence_interval\":[36.88836924235026,54.4616813659668],\"agent\":\"REDQ\"},{\"env_step\":125000.0,\"rew\":39.374982833862305,\"rew_std\":12.732017198364703,\"iqm\":38.69713592529297,\"iqm_confidence_interval\":[26.014727274576824,55.24446360270182],\"agent\":\"REDQ\"},{\"env_step\":130000.0,\"rew\":33.05560150146484,\"rew_std\":22.92158223741304,\"iqm\":33.4929567972819,\"iqm_confidence_interval\":[5.686606089274089,59.452101389567055],\"agent\":\"REDQ\"},{\"env_step\":135000.0,\"rew\":63.105552673339844,\"rew_std\":16.518051063313987,\"iqm\":66.08929824829102,\"iqm_confidence_interval\":[41.579349517822266,78.98898824055989],\"agent\":\"REDQ\"},{\"env_step\":140000.0,\"rew\":46.85098991394043,\"rew_std\":18.930184847734054,\"iqm\":47.70589701334635,\"iqm_confidence_interval\":[25.453510284423828,69.16705067952473],\"agent\":\"REDQ\"},{\"env_step\":145000.0,\"rew\":52.60828285217285,\"rew_std\":20.0156007925802,\"iqm\":50.19569396972656,\"iqm_confidence_interval\":[32.482496897379555,77.17677942911784],\"agent\":\"REDQ\"},{\"env_step\":150000.0,\"rew\":32.87973213195801,\"rew_std\":5.752220982221004,\"iqm\":33.12152926127116,\"iqm_confidence_interval\":[25.864628473917644,38.533990224202476],\"agent\":\"REDQ\"},{\"env_step\":155000.0,\"rew\":44.07918701171875,\"rew_std\":8.609497115980329,\"iqm\":44.441888173421226,\"iqm_confidence_interval\":[33.86382802327474,54.07351048787435],\"agent\":\"REDQ\"},{\"env_step\":160000.0,\"rew\":30.693387031555176,\"rew_std\":10.843262637384841,\"iqm\":33.20183245340983,\"iqm_confidence_interval\":[17.02465057373047,41.24501164754232],\"agent\":\"REDQ\"},{\"env_step\":165000.0,\"rew\":33.355489349365236,\"rew_std\":5.196249955937854,\"iqm\":33.20615895589193,\"iqm_confidence_interval\":[27.692097981770832,39.44086456298828],\"agent\":\"REDQ\"},{\"env_step\":170000.0,\"rew\":35.94187335968017,\"rew_std\":15.523436180672805,\"iqm\":38.41239356994629,\"iqm_confidence_interval\":[16.758402506510418,52.45171864827474],\"agent\":\"REDQ\"},{\"env_step\":175000.0,\"rew\":41.43852634429932,\"rew_std\":27.498704792345396,\"iqm\":35.69605954488119,\"iqm_confidence_interval\":[14.622135162353516,74.53206888834636],\"agent\":\"REDQ\"},{\"env_step\":180000.0,\"rew\":34.13255910873413,\"rew_std\":25.307062223373727,\"iqm\":28.26945209503174,\"iqm_confidence_interval\":[8.89621353149414,65.08337910970052],\"agent\":\"REDQ\"},{\"env_step\":185000.0,\"rew\":39.39708557128906,\"rew_std\":31.013538301188465,\"iqm\":43.62894503275553,\"iqm_confidence_interval\":[1.4370594024658203,72.83478291829427],\"agent\":\"REDQ\"},{\"env_step\":190000.0,\"rew\":25.197536277770997,\"rew_std\":14.176854535443127,\"iqm\":23.52732563018799,\"iqm_confidence_interval\":[9.304034868876139,41.48284022013346],\"agent\":\"REDQ\"},{\"env_step\":195000.0,\"rew\":19.246288204193114,\"rew_std\":20.540901260792012,\"iqm\":9.52669350306193,\"iqm_confidence_interval\":[7.575188159942627,44.38832473754883],\"agent\":\"REDQ\"},{\"env_step\":200000.0,\"rew\":26.64177188873291,\"rew_std\":10.990681141501083,\"iqm\":24.55473454793294,\"iqm_confidence_interval\":[15.542388280232748,40.352203369140625],\"agent\":\"REDQ\"},{\"env_step\":205000.0,\"rew\":22.146988439559937,\"rew_std\":12.838547894626247,\"iqm\":22.134310404459637,\"iqm_confidence_interval\":[7.028153578440349,36.973257064819336],\"agent\":\"REDQ\"},{\"env_step\":210000.0,\"rew\":20.998230075836183,\"rew_std\":13.356703236839795,\"iqm\":24.047775904337566,\"iqm_confidence_interval\":[5.1614993413289385,32.8866761525472],\"agent\":\"REDQ\"},{\"env_step\":215000.0,\"rew\":27.793178844451905,\"rew_std\":13.918359181461092,\"iqm\":28.164426803588867,\"iqm_confidence_interval\":[12.030838330586752,43.87256622314453],\"agent\":\"REDQ\"},{\"env_step\":220000.0,\"rew\":30.735445976257324,\"rew_std\":14.063949149444934,\"iqm\":28.017695744832356,\"iqm_confidence_interval\":[16.682162602742512,48.69248708089193],\"agent\":\"REDQ\"},{\"env_step\":225000.0,\"rew\":38.60630149841309,\"rew_std\":31.994663408504838,\"iqm\":33.90487003326416,\"iqm_confidence_interval\":[7.282408714294434,79.19370778401692],\"agent\":\"REDQ\"},{\"env_step\":230000.0,\"rew\":19.250569534301757,\"rew_std\":24.54294644655565,\"iqm\":11.479660749435425,\"iqm_confidence_interval\":[-1.0587985515594482,51.295522689819336],\"agent\":\"REDQ\"},{\"env_step\":235000.0,\"rew\":30.77835578918457,\"rew_std\":12.581581321709292,\"iqm\":26.17158381144206,\"iqm_confidence_interval\":[20.701234181722004,46.75053914388021],\"agent\":\"REDQ\"},{\"env_step\":240000.0,\"rew\":15.26291389465332,\"rew_std\":9.251941738607126,\"iqm\":16.373140652974445,\"iqm_confidence_interval\":[4.278467178344727,25.60233434041341],\"agent\":\"REDQ\"},{\"env_step\":245000.0,\"rew\":29.499822044372557,\"rew_std\":23.60619473210108,\"iqm\":23.36173439025879,\"iqm_confidence_interval\":[8.617469787597656,59.65530014038086],\"agent\":\"REDQ\"},{\"env_step\":250000.0,\"rew\":36.33521785736084,\"rew_std\":21.546205271752644,\"iqm\":34.795641581217446,\"iqm_confidence_interval\":[16.443824768066406,62.70384216308594],\"agent\":\"REDQ\"},{\"env_step\":255000.0,\"rew\":18.074655723571777,\"rew_std\":9.11244349956379,\"iqm\":14.337118784586588,\"iqm_confidence_interval\":[11.932367960611979,29.727651596069336],\"agent\":\"REDQ\"},{\"env_step\":260000.0,\"rew\":27.723381662368773,\"rew_std\":37.81439982632864,\"iqm\":12.885271072387695,\"iqm_confidence_interval\":[1.7803092002868652,73.44462776184082],\"agent\":\"REDQ\"},{\"env_step\":265000.0,\"rew\":25.311579477787017,\"rew_std\":21.841700020220447,\"iqm\":20.46875,\"iqm_confidence_interval\":[5.30793559551239,51.78204854329427],\"agent\":\"REDQ\"},{\"env_step\":270000.0,\"rew\":16.696401643753052,\"rew_std\":16.042648501377865,\"iqm\":17.084654887517292,\"iqm_confidence_interval\":[-2.5589532057444253,34.69189961751302],\"agent\":\"REDQ\"},{\"env_step\":275000.0,\"rew\":21.177660942077637,\"rew_std\":14.303141693630103,\"iqm\":19.086255073547363,\"iqm_confidence_interval\":[5.847034772237142,38.21140734354655],\"agent\":\"REDQ\"},{\"env_step\":280000.0,\"rew\":23.45450601577759,\"rew_std\":19.752276983360957,\"iqm\":20.248744010925293,\"iqm_confidence_interval\":[5.931210835774739,48.72629928588867],\"agent\":\"REDQ\"},{\"env_step\":285000.0,\"rew\":32.70703010559082,\"rew_std\":13.501850589686523,\"iqm\":34.144697189331055,\"iqm_confidence_interval\":[16.019423802693684,47.23796463012695],\"agent\":\"REDQ\"},{\"env_step\":290000.0,\"rew\":15.175395011901855,\"rew_std\":14.741239047740697,\"iqm\":19.14916197458903,\"iqm_confidence_interval\":[-3.7742255528767905,28.7479674021403],\"agent\":\"REDQ\"},{\"env_step\":295000.0,\"rew\":26.005640029907227,\"rew_std\":38.27730697784234,\"iqm\":13.446321487426758,\"iqm_confidence_interval\":[-3.740265210469564,71.88369782765706],\"agent\":\"REDQ\"},{\"env_step\":300000.0,\"rew\":19.6172076523304,\"rew_std\":12.230573673359512,\"iqm\":21.172632535298664,\"iqm_confidence_interval\":[4.248912910620372,32.60005187988281],\"agent\":\"REDQ\"},{\"env_step\":305000.0,\"rew\":10.947058582305909,\"rew_std\":21.094597096111134,\"iqm\":7.029097716013591,\"iqm_confidence_interval\":[-9.622747580210367,38.37720934549967],\"agent\":\"REDQ\"},{\"env_step\":310000.0,\"rew\":17.46815927028656,\"rew_std\":10.885388360104855,\"iqm\":16.692235628763836,\"iqm_confidence_interval\":[5.680540323257446,29.90889040629069],\"agent\":\"REDQ\"},{\"env_step\":315000.0,\"rew\":5.793402194976807,\"rew_std\":9.936588845500665,\"iqm\":8.28311554590861,\"iqm_confidence_interval\":[-6.639059543609619,15.044461886088053],\"agent\":\"REDQ\"},{\"env_step\":320000.0,\"rew\":13.770581036806107,\"rew_std\":14.23274143825097,\"iqm\":10.514999677737555,\"iqm_confidence_interval\":[-0.614104300737381,31.041234334309895],\"agent\":\"REDQ\"},{\"env_step\":325000.0,\"rew\":18.799122714996336,\"rew_std\":10.376715250176975,\"iqm\":16.759232838948567,\"iqm_confidence_interval\":[8.241253852844238,31.976130803426106],\"agent\":\"REDQ\"},{\"env_step\":330000.0,\"rew\":16.406962966918947,\"rew_std\":22.157508760557633,\"iqm\":16.556129455566406,\"iqm_confidence_interval\":[-8.651859919230143,41.26849365234375],\"agent\":\"REDQ\"},{\"env_step\":335000.0,\"rew\":12.169895958900451,\"rew_std\":14.963715049441639,\"iqm\":9.281240185101828,\"iqm_confidence_interval\":[-2.3518089850743613,31.649239857991535],\"agent\":\"REDQ\"},{\"env_step\":340000.0,\"rew\":3.442046642303467,\"rew_std\":14.780112286893091,\"iqm\":-2.1294188499450684,\"iqm_confidence_interval\":[-7.394394556681315,22.499412536621094],\"agent\":\"REDQ\"},{\"env_step\":345000.0,\"rew\":4.95629916191101,\"rew_std\":13.910492011418887,\"iqm\":4.2144068876902265,\"iqm_confidence_interval\":[-11.02927009264628,21.360683759053547],\"agent\":\"REDQ\"},{\"env_step\":350000.0,\"rew\":18.853069972991943,\"rew_std\":19.73532570992836,\"iqm\":13.482548395792643,\"iqm_confidence_interval\":[1.857444127400716,43.77008628845215],\"agent\":\"REDQ\"},{\"env_step\":355000.0,\"rew\":22.750345325469972,\"rew_std\":16.966662232915105,\"iqm\":18.741106351216633,\"iqm_confidence_interval\":[7.5960343678792315,44.02186838785807],\"agent\":\"REDQ\"},{\"env_step\":360000.0,\"rew\":5.956750690937042,\"rew_std\":21.695415764944062,\"iqm\":7.111966788768768,\"iqm_confidence_interval\":[-18.873626073201496,31.441300710042317],\"agent\":\"REDQ\"},{\"env_step\":365000.0,\"rew\":20.61870470046997,\"rew_std\":21.678673031794197,\"iqm\":13.428698062896729,\"iqm_confidence_interval\":[2.7947897911071777,49.16689236958822],\"agent\":\"REDQ\"},{\"env_step\":370000.0,\"rew\":14.885484218597412,\"rew_std\":17.805360659590868,\"iqm\":12.111168702443441,\"iqm_confidence_interval\":[-4.504808266957601,35.830423990885414],\"agent\":\"REDQ\"},{\"env_step\":375000.0,\"rew\":32.646864891052246,\"rew_std\":20.373741121242226,\"iqm\":26.919445673624676,\"iqm_confidence_interval\":[14.16562016805013,58.93025588989258],\"agent\":\"REDQ\"},{\"env_step\":380000.0,\"rew\":0.800808048248291,\"rew_std\":8.66086065871918,\"iqm\":-0.5694886843363444,\"iqm_confidence_interval\":[-8.144091129302979,11.844081242879232],\"agent\":\"REDQ\"},{\"env_step\":385000.0,\"rew\":5.095739841461182,\"rew_std\":10.521219934499408,\"iqm\":7.06280533472697,\"iqm_confidence_interval\":[-7.770551681518555,15.9800812403361],\"agent\":\"REDQ\"},{\"env_step\":390000.0,\"rew\":10.66844825744629,\"rew_std\":13.29951939275367,\"iqm\":12.87478764851888,\"iqm_confidence_interval\":[-5.858527183532715,23.706661860148113],\"agent\":\"REDQ\"},{\"env_step\":395000.0,\"rew\":16.440044784545897,\"rew_std\":11.947485845534818,\"iqm\":13.981200218200684,\"iqm_confidence_interval\":[5.077644983927409,31.113087336222332],\"agent\":\"REDQ\"},{\"env_step\":400000.0,\"rew\":-0.9765789985656739,\"rew_std\":15.48268801545026,\"iqm\":-0.12583525975545248,\"iqm_confidence_interval\":[-19.707523345947266,14.964693705240885],\"agent\":\"REDQ\"},{\"env_step\":405000.0,\"rew\":6.249713802337647,\"rew_std\":7.012356161983059,\"iqm\":8.156244277954102,\"iqm_confidence_interval\":[-2.619986057281494,12.68600082397461],\"agent\":\"REDQ\"},{\"env_step\":410000.0,\"rew\":-0.9776759624481202,\"rew_std\":14.570706019678896,\"iqm\":0.0393820603688558,\"iqm_confidence_interval\":[-18.631032307942707,14.512978871663412],\"agent\":\"REDQ\"},{\"env_step\":415000.0,\"rew\":9.446916484832764,\"rew_std\":9.888179038609788,\"iqm\":10.515723069508871,\"iqm_confidence_interval\":[-2.8868762652079263,18.93733787536621],\"agent\":\"REDQ\"},{\"env_step\":420000.0,\"rew\":-0.6988149166107178,\"rew_std\":16.74480844346608,\"iqm\":-1.5075164635976155,\"iqm_confidence_interval\":[-19.361233075459797,19.76446533203125],\"agent\":\"REDQ\"},{\"env_step\":425000.0,\"rew\":-10.437113916873932,\"rew_std\":9.06114434251262,\"iqm\":-8.526923179626465,\"iqm_confidence_interval\":[-21.928112665812176,-1.3592464526494343],\"agent\":\"REDQ\"},{\"env_step\":430000.0,\"rew\":12.44153401851654,\"rew_std\":21.58998283354288,\"iqm\":6.713498870531718,\"iqm_confidence_interval\":[-6.644435087839763,40.96184539794922],\"agent\":\"REDQ\"},{\"env_step\":435000.0,\"rew\":13.494499111175537,\"rew_std\":6.296845758071655,\"iqm\":12.34780216217041,\"iqm_confidence_interval\":[7.303154309590657,21.508914947509766],\"agent\":\"REDQ\"},{\"env_step\":440000.0,\"rew\":9.318463969230653,\"rew_std\":16.37547903187382,\"iqm\":7.349913080533345,\"iqm_confidence_interval\":[-6.832603057225545,30.141359329223633],\"agent\":\"REDQ\"},{\"env_step\":445000.0,\"rew\":14.813998603820801,\"rew_std\":27.582136294485988,\"iqm\":11.343953132629395,\"iqm_confidence_interval\":[-15.433674812316895,48.29724884033203],\"agent\":\"REDQ\"},{\"env_step\":450000.0,\"rew\":4.246308040618897,\"rew_std\":10.75259828037478,\"iqm\":4.867857774098714,\"iqm_confidence_interval\":[-7.741820812225342,15.575287024180094],\"agent\":\"REDQ\"},{\"env_step\":455000.0,\"rew\":10.270092296600343,\"rew_std\":21.201130261835395,\"iqm\":11.415112018585205,\"iqm_confidence_interval\":[-13.926047801971436,33.083889961242676],\"agent\":\"REDQ\"},{\"env_step\":460000.0,\"rew\":16.007566070556642,\"rew_std\":22.588210333120543,\"iqm\":12.718087514241537,\"iqm_confidence_interval\":[-6.672385851542155,43.067083994547524],\"agent\":\"REDQ\"},{\"env_step\":465000.0,\"rew\":17.37092866897583,\"rew_std\":17.09418538925163,\"iqm\":18.513753096262615,\"iqm_confidence_interval\":[-3.585215409596761,36.4017219543457],\"agent\":\"REDQ\"},{\"env_step\":470000.0,\"rew\":7.109980523586273,\"rew_std\":16.376672204163395,\"iqm\":7.444703638553619,\"iqm_confidence_interval\":[-12.121398071448008,25.28855323791504],\"agent\":\"REDQ\"},{\"env_step\":475000.0,\"rew\":10.492794704437255,\"rew_std\":22.391037243119204,\"iqm\":16.986163934071858,\"iqm_confidence_interval\":[-19.13445170720418,29.666130701700848],\"agent\":\"REDQ\"},{\"env_step\":480000.0,\"rew\":-11.270056629180909,\"rew_std\":21.05769175036746,\"iqm\":-12.146312236785889,\"iqm_confidence_interval\":[-34.96834373474121,13.961121718088785],\"agent\":\"REDQ\"},{\"env_step\":485000.0,\"rew\":17.99180948138237,\"rew_std\":25.44042909642577,\"iqm\":11.619555821021399,\"iqm_confidence_interval\":[-6.0503911674022675,50.00814310709635],\"agent\":\"REDQ\"},{\"env_step\":490000.0,\"rew\":5.295922470092774,\"rew_std\":33.95100987171947,\"iqm\":9.257699966430664,\"iqm_confidence_interval\":[-37.17231877644857,41.746816635131836],\"agent\":\"REDQ\"},{\"env_step\":495000.0,\"rew\":3.446676015853882,\"rew_std\":7.801368735135198,\"iqm\":2.087717135747274,\"iqm_confidence_interval\":[-4.688169797261556,13.158220609029135],\"agent\":\"REDQ\"},{\"env_step\":500000.0,\"rew\":8.002798652648925,\"rew_std\":18.659321671851902,\"iqm\":12.137170473734537,\"iqm_confidence_interval\":[-14.34347407023112,25.695288976033527],\"agent\":\"REDQ\"},{\"env_step\":0.0,\"rew\":30.388103103637697,\"rew_std\":6.788395302817334,\"iqm\":29.850717544555664,\"iqm_confidence_interval\":[23.30682309468587,38.79926681518555],\"agent\":\"Reinforce\"},{\"env_step\":30720.0,\"rew\":41.393571853637695,\"rew_std\":13.971328545803654,\"iqm\":42.14703114827474,\"iqm_confidence_interval\":[24.269193013509113,56.667057037353516],\"agent\":\"Reinforce\"},{\"env_step\":61440.0,\"rew\":26.356314849853515,\"rew_std\":12.084152781955408,\"iqm\":26.866626739501953,\"iqm_confidence_interval\":[12.766315460205078,40.5825449625651],\"agent\":\"Reinforce\"},{\"env_step\":92160.0,\"rew\":31.004388427734376,\"rew_std\":17.585687276059105,\"iqm\":26.937780062357586,\"iqm_confidence_interval\":[13.437909126281738,52.94202423095703],\"agent\":\"Reinforce\"},{\"env_step\":122880.0,\"rew\":36.31029825210571,\"rew_std\":18.476436394027456,\"iqm\":40.446022033691406,\"iqm_confidence_interval\":[13.678818066914877,55.07053248087565],\"agent\":\"Reinforce\"},{\"env_step\":153600.0,\"rew\":56.68689727783203,\"rew_std\":20.55872155871622,\"iqm\":60.506306966145836,\"iqm_confidence_interval\":[29.85924784342448,76.37279001871745],\"agent\":\"Reinforce\"},{\"env_step\":184320.0,\"rew\":40.220344161987306,\"rew_std\":13.267794175764193,\"iqm\":41.17553265889486,\"iqm_confidence_interval\":[23.531400680541992,54.020825703938804],\"agent\":\"Reinforce\"},{\"env_step\":215040.0,\"rew\":62.15134239196777,\"rew_std\":18.107928204603546,\"iqm\":68.45537567138672,\"iqm_confidence_interval\":[39.44554773966471,76.57495880126953],\"agent\":\"Reinforce\"},{\"env_step\":245760.0,\"rew\":58.648136138916016,\"rew_std\":22.385100878109142,\"iqm\":55.41466522216797,\"iqm_confidence_interval\":[36.151807149251304,86.53092956542969],\"agent\":\"Reinforce\"},{\"env_step\":276480.0,\"rew\":57.12719497680664,\"rew_std\":11.675106309035622,\"iqm\":55.473453521728516,\"iqm_confidence_interval\":[44.32411575317383,71.13336181640625],\"agent\":\"Reinforce\"},{\"env_step\":307200.0,\"rew\":59.3137393951416,\"rew_std\":18.880961069375342,\"iqm\":61.800671895345054,\"iqm_confidence_interval\":[35.953857421875,79.8323237101237],\"agent\":\"Reinforce\"},{\"env_step\":337920.0,\"rew\":75.0232650756836,\"rew_std\":28.034409418640404,\"iqm\":67.36776860555013,\"iqm_confidence_interval\":[50.934304555257164,111.92232259114583],\"agent\":\"Reinforce\"},{\"env_step\":368640.0,\"rew\":70.71477661132812,\"rew_std\":7.937664767412498,\"iqm\":69.91966247558594,\"iqm_confidence_interval\":[61.88165791829427,80.43016560872395],\"agent\":\"Reinforce\"},{\"env_step\":399360.0,\"rew\":75.26746444702148,\"rew_std\":23.88399289851664,\"iqm\":75.4661038716634,\"iqm_confidence_interval\":[47.51989618937174,103.60155741373698],\"agent\":\"Reinforce\"},{\"env_step\":430080.0,\"rew\":112.8280014038086,\"rew_std\":39.87061054787958,\"iqm\":101.47241719563802,\"iqm_confidence_interval\":[79.63001251220703,165.8230438232422],\"agent\":\"Reinforce\"},{\"env_step\":460800.0,\"rew\":97.8234634399414,\"rew_std\":23.66546101387088,\"iqm\":100.41691080729167,\"iqm_confidence_interval\":[67.86837259928386,122.61161295572917],\"agent\":\"Reinforce\"},{\"env_step\":491520.0,\"rew\":89.31181411743164,\"rew_std\":43.67601321953432,\"iqm\":72.80683771769206,\"iqm_confidence_interval\":[56.229427337646484,143.82876586914062],\"agent\":\"Reinforce\"},{\"env_step\":522240.0,\"rew\":79.20274124145507,\"rew_std\":23.178516509016514,\"iqm\":74.65371195475261,\"iqm_confidence_interval\":[55.99585723876953,108.97505950927734],\"agent\":\"Reinforce\"},{\"env_step\":552960.0,\"rew\":85.39522552490234,\"rew_std\":12.139876558939596,\"iqm\":84.75838979085286,\"iqm_confidence_interval\":[72.48362986246745,100.51192982991536],\"agent\":\"Reinforce\"},{\"env_step\":583680.0,\"rew\":109.23733825683594,\"rew_std\":25.486743575543077,\"iqm\":109.1050033569336,\"iqm_confidence_interval\":[81.64951833089192,140.30201212565103],\"agent\":\"Reinforce\"},{\"env_step\":614400.0,\"rew\":138.59989166259766,\"rew_std\":33.6939212817492,\"iqm\":142.34720357259116,\"iqm_confidence_interval\":[97.67273966471355,172.8327433268229],\"agent\":\"Reinforce\"},{\"env_step\":645120.0,\"rew\":121.40247039794922,\"rew_std\":49.794980618645425,\"iqm\":122.70049794514973,\"iqm_confidence_interval\":[60.47559356689453,176.4300994873047],\"agent\":\"Reinforce\"},{\"env_step\":675840.0,\"rew\":115.31985321044922,\"rew_std\":19.179720367158627,\"iqm\":123.2266337076823,\"iqm_confidence_interval\":[91.66427357991536,128.65331013997397],\"agent\":\"Reinforce\"},{\"env_step\":706560.0,\"rew\":109.73521270751954,\"rew_std\":32.05617777423248,\"iqm\":112.18231201171875,\"iqm_confidence_interval\":[70.27650451660156,139.39081319173178],\"agent\":\"Reinforce\"},{\"env_step\":737280.0,\"rew\":138.98448944091797,\"rew_std\":31.167571793249614,\"iqm\":125.43122863769531,\"iqm_confidence_interval\":[119.23805491129558,176.8132781982422],\"agent\":\"Reinforce\"},{\"env_step\":768000.0,\"rew\":160.27299957275392,\"rew_std\":31.830329501119625,\"iqm\":162.18966166178384,\"iqm_confidence_interval\":[125.64125569661458,197.5405527750651],\"agent\":\"Reinforce\"},{\"env_step\":798720.0,\"rew\":127.82812042236328,\"rew_std\":31.185511138850533,\"iqm\":133.3681157430013,\"iqm_confidence_interval\":[90.76575978597005,161.08210245768228],\"agent\":\"Reinforce\"},{\"env_step\":829440.0,\"rew\":131.0335693359375,\"rew_std\":54.88868142273811,\"iqm\":117.62583923339844,\"iqm_confidence_interval\":[82.56664021809895,199.98969523111978],\"agent\":\"Reinforce\"},{\"env_step\":860160.0,\"rew\":174.5028549194336,\"rew_std\":45.755765517387324,\"iqm\":182.59103393554688,\"iqm_confidence_interval\":[118.81187438964844,221.41290283203125],\"agent\":\"Reinforce\"},{\"env_step\":890880.0,\"rew\":159.05300750732422,\"rew_std\":29.948995732290975,\"iqm\":165.14197794596353,\"iqm_confidence_interval\":[120.02970886230469,187.74015299479166],\"agent\":\"Reinforce\"},{\"env_step\":921600.0,\"rew\":189.1221496582031,\"rew_std\":37.90998388924396,\"iqm\":190.80534871419272,\"iqm_confidence_interval\":[142.2677256266276,230.343994140625],\"agent\":\"Reinforce\"},{\"env_step\":952320.0,\"rew\":200.0733184814453,\"rew_std\":34.72404034142182,\"iqm\":198.57618204752603,\"iqm_confidence_interval\":[158.88312276204428,238.5328572591146],\"agent\":\"Reinforce\"},{\"env_step\":983040.0,\"rew\":184.41802368164062,\"rew_std\":41.505971796954185,\"iqm\":178.31390889485678,\"iqm_confidence_interval\":[140.6153818766276,234.4051971435547],\"agent\":\"Reinforce\"},{\"env_step\":1013760.0,\"rew\":217.9223663330078,\"rew_std\":30.55818587396373,\"iqm\":212.4499053955078,\"iqm_confidence_interval\":[185.58199055989584,255.27669779459634],\"agent\":\"Reinforce\"},{\"env_step\":1044480.0,\"rew\":188.50809783935546,\"rew_std\":45.63991159019269,\"iqm\":192.9031728108724,\"iqm_confidence_interval\":[134.54771931966147,236.86406453450522],\"agent\":\"Reinforce\"},{\"env_step\":1075200.0,\"rew\":172.07819671630858,\"rew_std\":52.15507095930203,\"iqm\":170.8068389892578,\"iqm_confidence_interval\":[113.67265319824219,235.25662231445312],\"agent\":\"Reinforce\"},{\"env_step\":1105920.0,\"rew\":202.97726135253907,\"rew_std\":36.1947818092481,\"iqm\":192.60591634114584,\"iqm_confidence_interval\":[172.39493815104166,247.6568857828776],\"agent\":\"Reinforce\"},{\"env_step\":1136640.0,\"rew\":155.8261291503906,\"rew_std\":15.666026150139057,\"iqm\":157.44652303059897,\"iqm_confidence_interval\":[136.67408243815103,173.25584920247397],\"agent\":\"Reinforce\"},{\"env_step\":1167360.0,\"rew\":203.0422790527344,\"rew_std\":49.88510628853617,\"iqm\":203.64251200358072,\"iqm_confidence_interval\":[143.92074584960938,260.19286092122394],\"agent\":\"Reinforce\"},{\"env_step\":1198080.0,\"rew\":244.92487182617188,\"rew_std\":60.045671210825894,\"iqm\":231.50265502929688,\"iqm_confidence_interval\":[189.49774169921875,323.7748209635417],\"agent\":\"Reinforce\"},{\"env_step\":1228800.0,\"rew\":205.7066192626953,\"rew_std\":62.55029791054683,\"iqm\":226.20610555013022,\"iqm_confidence_interval\":[123.72357686360677,258.00376383463544],\"agent\":\"Reinforce\"},{\"env_step\":1259520.0,\"rew\":250.18730773925782,\"rew_std\":65.97905425786584,\"iqm\":240.34812927246094,\"iqm_confidence_interval\":[178.1267344156901,328.5118916829427],\"agent\":\"Reinforce\"},{\"env_step\":1290240.0,\"rew\":224.54561767578124,\"rew_std\":46.52547907892198,\"iqm\":231.78507486979166,\"iqm_confidence_interval\":[165.31780497233072,267.29746500651044],\"agent\":\"Reinforce\"},{\"env_step\":1320960.0,\"rew\":218.5148956298828,\"rew_std\":47.08101810365439,\"iqm\":221.06966654459634,\"iqm_confidence_interval\":[165.04052225748697,273.56219482421875],\"agent\":\"Reinforce\"},{\"env_step\":1351680.0,\"rew\":187.0691879272461,\"rew_std\":40.25982166153624,\"iqm\":190.78805541992188,\"iqm_confidence_interval\":[141.08477274576822,233.19166056315103],\"agent\":\"Reinforce\"},{\"env_step\":1382400.0,\"rew\":258.0915588378906,\"rew_std\":52.01694964975601,\"iqm\":247.74063618977866,\"iqm_confidence_interval\":[207.2652333577474,323.55576578776044],\"agent\":\"Reinforce\"},{\"env_step\":1413120.0,\"rew\":215.4867919921875,\"rew_std\":58.539436951587135,\"iqm\":221.1207021077474,\"iqm_confidence_interval\":[151.37629191080728,282.22046915690106],\"agent\":\"Reinforce\"},{\"env_step\":1443840.0,\"rew\":216.3734924316406,\"rew_std\":38.32389133621041,\"iqm\":212.92101033528647,\"iqm_confidence_interval\":[176.09254455566406,261.8755645751953],\"agent\":\"Reinforce\"},{\"env_step\":1474560.0,\"rew\":264.408349609375,\"rew_std\":38.30968749271509,\"iqm\":262.39134216308594,\"iqm_confidence_interval\":[221.9061533610026,311.08123779296875],\"agent\":\"Reinforce\"},{\"env_step\":1505280.0,\"rew\":283.94684143066405,\"rew_std\":104.28181017938446,\"iqm\":277.3927815755208,\"iqm_confidence_interval\":[163.11287434895834,405.19097900390625],\"agent\":\"Reinforce\"},{\"env_step\":1536000.0,\"rew\":213.6966125488281,\"rew_std\":21.125488073494697,\"iqm\":221.94438680013022,\"iqm_confidence_interval\":[187.61536153157553,229.0794881184896],\"agent\":\"Reinforce\"},{\"env_step\":1566720.0,\"rew\":240.69333190917968,\"rew_std\":43.58871731853648,\"iqm\":239.96679178873697,\"iqm_confidence_interval\":[189.83861287434897,289.31268310546875],\"agent\":\"Reinforce\"},{\"env_step\":1597440.0,\"rew\":227.7518280029297,\"rew_std\":43.6807819928471,\"iqm\":226.01578776041666,\"iqm_confidence_interval\":[178.9113566080729,280.186767578125],\"agent\":\"Reinforce\"},{\"env_step\":1628160.0,\"rew\":254.28907470703126,\"rew_std\":58.96974427894262,\"iqm\":265.43126424153644,\"iqm_confidence_interval\":[183.42256673177084,316.2568359375],\"agent\":\"Reinforce\"},{\"env_step\":1658880.0,\"rew\":279.0290191650391,\"rew_std\":29.8961086169682,\"iqm\":284.4513448079427,\"iqm_confidence_interval\":[240.81451416015625,309.4889424641927],\"agent\":\"Reinforce\"},{\"env_step\":1689600.0,\"rew\":280.4690307617187,\"rew_std\":69.44635836416715,\"iqm\":291.83249918619794,\"iqm_confidence_interval\":[195.19434611002603,353.3846028645833],\"agent\":\"Reinforce\"},{\"env_step\":1720320.0,\"rew\":285.4685363769531,\"rew_std\":45.42759739091963,\"iqm\":292.0075988769531,\"iqm_confidence_interval\":[229.6476847330729,333.6572977701823],\"agent\":\"Reinforce\"},{\"env_step\":1751040.0,\"rew\":266.4340545654297,\"rew_std\":47.66348445637528,\"iqm\":280.7711690266927,\"iqm_confidence_interval\":[205.85944620768228,306.71190388997394],\"agent\":\"Reinforce\"},{\"env_step\":1781760.0,\"rew\":234.5447265625,\"rew_std\":36.225980542194165,\"iqm\":237.1103769938151,\"iqm_confidence_interval\":[191.05297342936197,275.2825419108073],\"agent\":\"Reinforce\"},{\"env_step\":1812480.0,\"rew\":307.52418212890626,\"rew_std\":84.81374513240468,\"iqm\":275.64890543619794,\"iqm_confidence_interval\":[246.65121459960938,410.00844319661456],\"agent\":\"Reinforce\"},{\"env_step\":1843200.0,\"rew\":266.19762268066404,\"rew_std\":62.69389135691518,\"iqm\":265.62952677408856,\"iqm_confidence_interval\":[190.63823445638022,336.2323404947917],\"agent\":\"Reinforce\"},{\"env_step\":1873920.0,\"rew\":291.81089782714844,\"rew_std\":50.755500210592444,\"iqm\":272.91077677408856,\"iqm_confidence_interval\":[254.44548543294272,355.9960530598958],\"agent\":\"Reinforce\"},{\"env_step\":1904640.0,\"rew\":259.18329467773435,\"rew_std\":48.49455400892606,\"iqm\":253.11312866210938,\"iqm_confidence_interval\":[205.61703491210938,317.95094807942706],\"agent\":\"Reinforce\"},{\"env_step\":1935360.0,\"rew\":307.09999389648436,\"rew_std\":43.827630316738166,\"iqm\":313.4484151204427,\"iqm_confidence_interval\":[254.4388631184896,352.08115641276044],\"agent\":\"Reinforce\"},{\"env_step\":1966080.0,\"rew\":281.89708557128904,\"rew_std\":45.02522260976583,\"iqm\":289.0003356933594,\"iqm_confidence_interval\":[224.81385294596353,328.21592203776044],\"agent\":\"Reinforce\"},{\"env_step\":1996800.0,\"rew\":309.51084899902344,\"rew_std\":78.08004038475458,\"iqm\":303.74212646484375,\"iqm_confidence_interval\":[223.89665730794272,398.45416259765625],\"agent\":\"Reinforce\"},{\"env_step\":2027520.0,\"rew\":333.3232360839844,\"rew_std\":35.671012939999144,\"iqm\":334.38214111328125,\"iqm_confidence_interval\":[289.5745035807292,369.2607421875],\"agent\":\"Reinforce\"},{\"env_step\":2058240.0,\"rew\":271.2226257324219,\"rew_std\":14.60967271548954,\"iqm\":273.5323740641276,\"iqm_confidence_interval\":[252.78433736165366,283.69317626953125],\"agent\":\"Reinforce\"},{\"env_step\":2088960.0,\"rew\":314.7043701171875,\"rew_std\":32.77643435528697,\"iqm\":314.237060546875,\"iqm_confidence_interval\":[276.87865193684894,353.4198506673177],\"agent\":\"Reinforce\"},{\"env_step\":2119680.0,\"rew\":290.65659790039064,\"rew_std\":50.538724138399026,\"iqm\":286.47316487630206,\"iqm_confidence_interval\":[235.33682250976562,353.5723063151042],\"agent\":\"Reinforce\"},{\"env_step\":2150400.0,\"rew\":324.43235473632814,\"rew_std\":34.699153547407164,\"iqm\":327.8403015136719,\"iqm_confidence_interval\":[281.2065124511719,359.95556640625],\"agent\":\"Reinforce\"},{\"env_step\":2181120.0,\"rew\":334.29482116699216,\"rew_std\":61.61826359031619,\"iqm\":343.5783386230469,\"iqm_confidence_interval\":[261.01324462890625,395.5978291829427],\"agent\":\"Reinforce\"},{\"env_step\":2211840.0,\"rew\":301.3869171142578,\"rew_std\":54.98068490657697,\"iqm\":296.2631429036458,\"iqm_confidence_interval\":[242.42849731445312,369.9410705566406],\"agent\":\"Reinforce\"},{\"env_step\":2242560.0,\"rew\":338.5574951171875,\"rew_std\":51.59526706535765,\"iqm\":328.11199951171875,\"iqm_confidence_interval\":[286.99342854817706,404.38866170247394],\"agent\":\"Reinforce\"},{\"env_step\":2273280.0,\"rew\":301.586181640625,\"rew_std\":16.842084248418107,\"iqm\":305.23716227213544,\"iqm_confidence_interval\":[281.1344909667969,318.3143310546875],\"agent\":\"Reinforce\"},{\"env_step\":2304000.0,\"rew\":323.445556640625,\"rew_std\":61.267217037347834,\"iqm\":332.5047098795573,\"iqm_confidence_interval\":[245.55340576171875,387.0192565917969],\"agent\":\"Reinforce\"},{\"env_step\":2334720.0,\"rew\":320.56577758789064,\"rew_std\":35.492188799324026,\"iqm\":309.4940490722656,\"iqm_confidence_interval\":[289.9680887858073,367.02295939127606],\"agent\":\"Reinforce\"},{\"env_step\":2365440.0,\"rew\":335.6709899902344,\"rew_std\":66.80572709335965,\"iqm\":337.3677469889323,\"iqm_confidence_interval\":[255.53173828125,412.7243143717448],\"agent\":\"Reinforce\"},{\"env_step\":2396160.0,\"rew\":329.3257080078125,\"rew_std\":53.664796517586076,\"iqm\":331.4996032714844,\"iqm_confidence_interval\":[266.3409016927083,391.79189046223956],\"agent\":\"Reinforce\"},{\"env_step\":2426880.0,\"rew\":329.5802276611328,\"rew_std\":84.17181155441796,\"iqm\":352.0137125651042,\"iqm_confidence_interval\":[223.3980712890625,403.7108459472656],\"agent\":\"Reinforce\"},{\"env_step\":2457600.0,\"rew\":318.6984924316406,\"rew_std\":74.45687540237842,\"iqm\":315.1752421061198,\"iqm_confidence_interval\":[234.7754109700521,405.18434651692706],\"agent\":\"Reinforce\"},{\"env_step\":2488320.0,\"rew\":313.40746154785154,\"rew_std\":73.8331593572831,\"iqm\":319.9508361816406,\"iqm_confidence_interval\":[222.83462524414062,395.68932088216144],\"agent\":\"Reinforce\"},{\"env_step\":2519040.0,\"rew\":335.41685180664064,\"rew_std\":47.44474724052373,\"iqm\":317.7933044433594,\"iqm_confidence_interval\":[300.66258748372394,397.95128377278644],\"agent\":\"Reinforce\"},{\"env_step\":2549760.0,\"rew\":292.2950775146484,\"rew_std\":50.12930012184995,\"iqm\":307.0918782552083,\"iqm_confidence_interval\":[226.51502482096353,335.8219909667969],\"agent\":\"Reinforce\"},{\"env_step\":2580480.0,\"rew\":378.39520874023435,\"rew_std\":36.30654114289557,\"iqm\":374.59874471028644,\"iqm_confidence_interval\":[339.26439412434894,423.85561116536456],\"agent\":\"Reinforce\"},{\"env_step\":2611200.0,\"rew\":339.3572692871094,\"rew_std\":52.24209828973846,\"iqm\":342.9746602376302,\"iqm_confidence_interval\":[280.0288798014323,393.28118896484375],\"agent\":\"Reinforce\"},{\"env_step\":2641920.0,\"rew\":311.60890808105466,\"rew_std\":88.25142354618134,\"iqm\":342.87502034505206,\"iqm_confidence_interval\":[201.42610677083334,378.2011006673177],\"agent\":\"Reinforce\"},{\"env_step\":2672640.0,\"rew\":353.88108215332034,\"rew_std\":82.5540304412778,\"iqm\":361.1837870279948,\"iqm_confidence_interval\":[252.4522501627604,432.9479064941406],\"agent\":\"Reinforce\"},{\"env_step\":2703360.0,\"rew\":331.2927978515625,\"rew_std\":33.18178391165257,\"iqm\":332.3551737467448,\"iqm_confidence_interval\":[291.2009684244792,363.9695332845052],\"agent\":\"Reinforce\"},{\"env_step\":2734080.0,\"rew\":372.3266265869141,\"rew_std\":78.96542655118607,\"iqm\":378.66721598307294,\"iqm_confidence_interval\":[274.5861307779948,457.87548828125],\"agent\":\"Reinforce\"},{\"env_step\":2764800.0,\"rew\":325.185791015625,\"rew_std\":50.082658331132286,\"iqm\":322.7134602864583,\"iqm_confidence_interval\":[266.5505676269531,381.635498046875],\"agent\":\"Reinforce\"},{\"env_step\":2795520.0,\"rew\":274.6520294189453,\"rew_std\":49.78978121357848,\"iqm\":271.5624643961589,\"iqm_confidence_interval\":[220.35921732584634,334.1891581217448],\"agent\":\"Reinforce\"},{\"env_step\":2826240.0,\"rew\":322.1300994873047,\"rew_std\":89.95667783686396,\"iqm\":298.76662190755206,\"iqm_confidence_interval\":[235.33484903971353,432.0729471842448],\"agent\":\"Reinforce\"},{\"env_step\":2856960.0,\"rew\":340.1513916015625,\"rew_std\":37.89333285730595,\"iqm\":345.91286214192706,\"iqm_confidence_interval\":[296.1921793619792,381.60411580403644],\"agent\":\"Reinforce\"},{\"env_step\":2887680.0,\"rew\":366.31533203125,\"rew_std\":55.248782096101024,\"iqm\":362.03123982747394,\"iqm_confidence_interval\":[303.2717692057292,431.9060872395833],\"agent\":\"Reinforce\"},{\"env_step\":2918400.0,\"rew\":372.2232238769531,\"rew_std\":84.5105446284943,\"iqm\":377.65968831380206,\"iqm_confidence_interval\":[268.6006571451823,463.58217366536456],\"agent\":\"Reinforce\"},{\"env_step\":2949120.0,\"rew\":309.97247314453125,\"rew_std\":41.89657001617316,\"iqm\":311.08339436848956,\"iqm_confidence_interval\":[259.52565511067706,356.1070963541667],\"agent\":\"Reinforce\"},{\"env_step\":2979840.0,\"rew\":353.62855224609376,\"rew_std\":44.15351430823454,\"iqm\":354.0978698730469,\"iqm_confidence_interval\":[301.82387288411456,405.1334737141927],\"agent\":\"Reinforce\"},{\"env_step\":3010560.0,\"rew\":297.6916809082031,\"rew_std\":84.90951670827089,\"iqm\":291.50318400065106,\"iqm_confidence_interval\":[210.40422566731772,394.10882568359375],\"agent\":\"Reinforce\"},{\"env_step\":3041280.0,\"rew\":301.43951721191405,\"rew_std\":59.52449076004893,\"iqm\":282.8259633382161,\"iqm_confidence_interval\":[251.1397959391276,379.67467244466144],\"agent\":\"Reinforce\"},{\"env_step\":3072000.0,\"rew\":343.80126953125,\"rew_std\":65.90217959384564,\"iqm\":336.1062316894531,\"iqm_confidence_interval\":[274.6854248046875,427.3608703613281],\"agent\":\"Reinforce\"},{\"env_step\":0.0,\"rew\":-77.19025497436523,\"rew_std\":62.2254596836894,\"iqm\":-54.83956527709961,\"iqm_confidence_interval\":[-152.90450032552084,-29.95730209350586],\"agent\":\"SAC\"},{\"env_step\":5000.0,\"rew\":-35.89192981719971,\"rew_std\":17.41051894400482,\"iqm\":-36.26547304789225,\"iqm_confidence_interval\":[-55.806538899739586,-15.982662200927734],\"agent\":\"SAC\"},{\"env_step\":10000.0,\"rew\":-8.365598672628403,\"rew_std\":8.014129819343015,\"iqm\":-7.29702623685201,\"iqm_confidence_interval\":[-18.26535193125407,0.24046256144841513],\"agent\":\"SAC\"},{\"env_step\":15000.0,\"rew\":-20.333968925476075,\"rew_std\":31.329479140643837,\"iqm\":-21.02641010284424,\"iqm_confidence_interval\":[-57.44995625813802,14.767145792643229],\"agent\":\"SAC\"},{\"env_step\":20000.0,\"rew\":17.028606390953065,\"rew_std\":37.66061999840288,\"iqm\":8.315877874692282,\"iqm_confidence_interval\":[-17.764819820721943,63.04864088694254],\"agent\":\"SAC\"},{\"env_step\":25000.0,\"rew\":37.705559158325194,\"rew_std\":25.836473099852302,\"iqm\":32.60944366455078,\"iqm_confidence_interval\":[13.486283620198568,68.4135856628418],\"agent\":\"SAC\"},{\"env_step\":30000.0,\"rew\":75.73391571044922,\"rew_std\":22.290208907489014,\"iqm\":74.22437032063802,\"iqm_confidence_interval\":[52.38097127278646,102.75186920166016],\"agent\":\"SAC\"},{\"env_step\":35000.0,\"rew\":84.19118194580078,\"rew_std\":17.008699099731583,\"iqm\":85.2749532063802,\"iqm_confidence_interval\":[62.907440185546875,102.23167928059895],\"agent\":\"SAC\"},{\"env_step\":40000.0,\"rew\":129.09686126708985,\"rew_std\":22.589794187279907,\"iqm\":129.61712137858072,\"iqm_confidence_interval\":[101.76744079589844,154.85226440429688],\"agent\":\"SAC\"},{\"env_step\":45000.0,\"rew\":152.65745239257814,\"rew_std\":48.46755208574535,\"iqm\":155.56519826253256,\"iqm_confidence_interval\":[93.70250701904297,206.66668192545572],\"agent\":\"SAC\"},{\"env_step\":50000.0,\"rew\":155.83685607910155,\"rew_std\":31.52167141078223,\"iqm\":143.8017832438151,\"iqm_confidence_interval\":[132.99841817220053,195.23765563964844],\"agent\":\"SAC\"},{\"env_step\":55000.0,\"rew\":182.55461730957032,\"rew_std\":43.99510158347367,\"iqm\":188.25503540039062,\"iqm_confidence_interval\":[126.17987569173177,224.71722412109375],\"agent\":\"SAC\"},{\"env_step\":60000.0,\"rew\":176.60393676757812,\"rew_std\":43.76801045676628,\"iqm\":160.3560536702474,\"iqm_confidence_interval\":[142.53045145670572,231.58539835611978],\"agent\":\"SAC\"},{\"env_step\":65000.0,\"rew\":199.31842346191405,\"rew_std\":58.64699520848267,\"iqm\":183.22557576497397,\"iqm_confidence_interval\":[147.38675435384116,272.1580505371094],\"agent\":\"SAC\"},{\"env_step\":70000.0,\"rew\":219.99323272705078,\"rew_std\":85.8307555926905,\"iqm\":203.48806762695312,\"iqm_confidence_interval\":[130.2459971110026,322.3173828125],\"agent\":\"SAC\"},{\"env_step\":75000.0,\"rew\":188.92798461914063,\"rew_std\":35.46983211589195,\"iqm\":192.1189727783203,\"iqm_confidence_interval\":[146.61749267578125,228.26592508951822],\"agent\":\"SAC\"},{\"env_step\":80000.0,\"rew\":286.16457214355466,\"rew_std\":85.8509054317401,\"iqm\":306.4248555501302,\"iqm_confidence_interval\":[183.60598754882812,372.40846761067706],\"agent\":\"SAC\"},{\"env_step\":85000.0,\"rew\":284.43455505371094,\"rew_std\":96.45630722641503,\"iqm\":294.31304423014325,\"iqm_confidence_interval\":[165.90079243977866,385.88283284505206],\"agent\":\"SAC\"},{\"env_step\":90000.0,\"rew\":250.1466552734375,\"rew_std\":42.79532797670044,\"iqm\":254.65687052408853,\"iqm_confidence_interval\":[200.3510538736979,295.53907267252606],\"agent\":\"SAC\"},{\"env_step\":95000.0,\"rew\":252.18509826660156,\"rew_std\":35.802201568385726,\"iqm\":250.29408264160156,\"iqm_confidence_interval\":[210.97714233398438,294.58123779296875],\"agent\":\"SAC\"},{\"env_step\":100000.0,\"rew\":286.2181793212891,\"rew_std\":84.1511572584835,\"iqm\":300.5720520019531,\"iqm_confidence_interval\":[181.56843058268228,368.3089294433594],\"agent\":\"SAC\"},{\"env_step\":105000.0,\"rew\":264.8419158935547,\"rew_std\":35.2395765966379,\"iqm\":275.5354715983073,\"iqm_confidence_interval\":[222.22441609700522,293.87743123372394],\"agent\":\"SAC\"},{\"env_step\":110000.0,\"rew\":513.6656555175781,\"rew_std\":218.65707318919647,\"iqm\":428.05491129557294,\"iqm_confidence_interval\":[357.8131408691406,797.7045084635416],\"agent\":\"SAC\"},{\"env_step\":115000.0,\"rew\":497.7557800292969,\"rew_std\":366.65431431565366,\"iqm\":323.65578206380206,\"iqm_confidence_interval\":[290.1732482910156,945.1743876139323],\"agent\":\"SAC\"},{\"env_step\":120000.0,\"rew\":476.4174530029297,\"rew_std\":283.98567750072647,\"iqm\":374.6880594889323,\"iqm_confidence_interval\":[262.7366231282552,837.0888977050781],\"agent\":\"SAC\"},{\"env_step\":125000.0,\"rew\":453.8511901855469,\"rew_std\":223.38435370933277,\"iqm\":371.7033182779948,\"iqm_confidence_interval\":[277.9562479654948,729.7964579264323],\"agent\":\"SAC\"},{\"env_step\":130000.0,\"rew\":548.385791015625,\"rew_std\":283.5735249021172,\"iqm\":504.7932637532552,\"iqm_confidence_interval\":[261.8063456217448,908.9465535481771],\"agent\":\"SAC\"},{\"env_step\":135000.0,\"rew\":561.4510925292968,\"rew_std\":308.97892296397185,\"iqm\":446.41258748372394,\"iqm_confidence_interval\":[319.62439982096356,958.6476236979166],\"agent\":\"SAC\"},{\"env_step\":140000.0,\"rew\":454.39158020019534,\"rew_std\":285.15522952077515,\"iqm\":392.1344451904297,\"iqm_confidence_interval\":[168.9661102294922,801.3722330729166],\"agent\":\"SAC\"},{\"env_step\":145000.0,\"rew\":604.955941772461,\"rew_std\":565.4580665055674,\"iqm\":358.6328938802083,\"iqm_confidence_interval\":[242.29569498697916,1315.4267679850261],\"agent\":\"SAC\"},{\"env_step\":150000.0,\"rew\":579.629736328125,\"rew_std\":174.94201468630686,\"iqm\":614.0927429199219,\"iqm_confidence_interval\":[350.2121276855469,741.2681070963541],\"agent\":\"SAC\"},{\"env_step\":155000.0,\"rew\":801.4467224121094,\"rew_std\":331.04880898309625,\"iqm\":846.8262329101562,\"iqm_confidence_interval\":[391.0495198567708,1158.1105550130208],\"agent\":\"SAC\"},{\"env_step\":160000.0,\"rew\":779.7697937011719,\"rew_std\":433.51506755083886,\"iqm\":664.1381429036459,\"iqm_confidence_interval\":[394.3942057291667,1309.4696451822917],\"agent\":\"SAC\"},{\"env_step\":165000.0,\"rew\":1033.8955139160157,\"rew_std\":481.05420825123053,\"iqm\":1123.970926920573,\"iqm_confidence_interval\":[414.709716796875,1445.7089029947917],\"agent\":\"SAC\"},{\"env_step\":170000.0,\"rew\":800.0918518066406,\"rew_std\":329.7873973303998,\"iqm\":779.8632609049479,\"iqm_confidence_interval\":[441.96872965494794,1199.1294759114583],\"agent\":\"SAC\"},{\"env_step\":175000.0,\"rew\":915.3458435058594,\"rew_std\":477.158514471823,\"iqm\":856.32373046875,\"iqm_confidence_interval\":[407.51715087890625,1494.6025390625],\"agent\":\"SAC\"},{\"env_step\":180000.0,\"rew\":704.3885070800782,\"rew_std\":199.70826616083713,\"iqm\":752.0989583333334,\"iqm_confidence_interval\":[440.9189860026042,879.9899088541666],\"agent\":\"SAC\"},{\"env_step\":185000.0,\"rew\":686.2416809082031,\"rew_std\":325.6238141858492,\"iqm\":636.9073893229166,\"iqm_confidence_interval\":[371.8382568359375,1070.7754516601562],\"agent\":\"SAC\"},{\"env_step\":190000.0,\"rew\":1294.975811767578,\"rew_std\":770.3208203642287,\"iqm\":1214.5337524414062,\"iqm_confidence_interval\":[454.859130859375,2224.272705078125],\"agent\":\"SAC\"},{\"env_step\":195000.0,\"rew\":1870.000341796875,\"rew_std\":1260.2279022802952,\"iqm\":1780.2755533854167,\"iqm_confidence_interval\":[536.3902587890625,3447.1735026041665],\"agent\":\"SAC\"},{\"env_step\":200000.0,\"rew\":1622.0802001953125,\"rew_std\":623.6543352847739,\"iqm\":1583.7178141276042,\"iqm_confidence_interval\":[951.5055745442709,2395.7915852864585],\"agent\":\"SAC\"},{\"env_step\":205000.0,\"rew\":1854.9751220703124,\"rew_std\":1056.689688092062,\"iqm\":1635.9498291015625,\"iqm_confidence_interval\":[887.18212890625,3234.2255859375],\"agent\":\"SAC\"},{\"env_step\":210000.0,\"rew\":1839.5556640625,\"rew_std\":955.7378555572526,\"iqm\":1730.114013671875,\"iqm_confidence_interval\":[848.3274332682291,3050.9403483072915],\"agent\":\"SAC\"},{\"env_step\":215000.0,\"rew\":1429.5697509765625,\"rew_std\":648.9423857356364,\"iqm\":1418.4546508789062,\"iqm_confidence_interval\":[736.1454060872396,2217.648681640625],\"agent\":\"SAC\"},{\"env_step\":220000.0,\"rew\":1728.10546875,\"rew_std\":925.2835955326163,\"iqm\":1642.0914713541667,\"iqm_confidence_interval\":[722.1980387369791,2882.3997395833335],\"agent\":\"SAC\"},{\"env_step\":225000.0,\"rew\":1997.5099365234375,\"rew_std\":1183.7667825669266,\"iqm\":1655.85205078125,\"iqm_confidence_interval\":[985.1422119140625,3557.1197102864585],\"agent\":\"SAC\"},{\"env_step\":230000.0,\"rew\":2152.180712890625,\"rew_std\":840.7990009143479,\"iqm\":1817.1878662109375,\"iqm_confidence_interval\":[1581.4125162760417,3174.1283365885415],\"agent\":\"SAC\"},{\"env_step\":235000.0,\"rew\":2194.316857910156,\"rew_std\":673.0463746958201,\"iqm\":2331.4443359375,\"iqm_confidence_interval\":[1369.1734619140625,2877.5187174479165],\"agent\":\"SAC\"},{\"env_step\":240000.0,\"rew\":2253.086572265625,\"rew_std\":1177.115510302391,\"iqm\":2028.7047932942708,\"iqm_confidence_interval\":[1036.8601481119792,3703.5294596354165],\"agent\":\"SAC\"},{\"env_step\":245000.0,\"rew\":2316.2286376953125,\"rew_std\":1117.8252019819483,\"iqm\":2303.7169596354165,\"iqm_confidence_interval\":[1055.2244466145833,3587.3098958333335],\"agent\":\"SAC\"},{\"env_step\":250000.0,\"rew\":2549.455847167969,\"rew_std\":1084.187701336248,\"iqm\":2822.6012369791665,\"iqm_confidence_interval\":[1117.0762125651042,3513.427001953125],\"agent\":\"SAC\"},{\"env_step\":255000.0,\"rew\":2546.011181640625,\"rew_std\":862.4939491058706,\"iqm\":2500.5267333984375,\"iqm_confidence_interval\":[1539.4224039713542,3553.4488932291665],\"agent\":\"SAC\"},{\"env_step\":260000.0,\"rew\":2777.9132080078125,\"rew_std\":813.6251453201878,\"iqm\":2819.093017578125,\"iqm_confidence_interval\":[1862.713623046875,3660.3916829427085],\"agent\":\"SAC\"},{\"env_step\":265000.0,\"rew\":2675.9651123046874,\"rew_std\":1029.449430698658,\"iqm\":2811.8418782552085,\"iqm_confidence_interval\":[1408.0743815104167,3794.1632486979165],\"agent\":\"SAC\"},{\"env_step\":270000.0,\"rew\":2514.9052490234376,\"rew_std\":1124.8782698523578,\"iqm\":2861.392333984375,\"iqm_confidence_interval\":[1048.2747395833333,3495.075439453125],\"agent\":\"SAC\"},{\"env_step\":275000.0,\"rew\":3179.036376953125,\"rew_std\":607.7492449757316,\"iqm\":3177.6503092447915,\"iqm_confidence_interval\":[2465.8914388020835,3866.8968098958335],\"agent\":\"SAC\"},{\"env_step\":280000.0,\"rew\":2910.2163330078124,\"rew_std\":1210.8612926093572,\"iqm\":2791.435302734375,\"iqm_confidence_interval\":[1585.8204752604167,4416.846435546875],\"agent\":\"SAC\"},{\"env_step\":285000.0,\"rew\":3425.8759033203123,\"rew_std\":976.7204258971652,\"iqm\":3409.6282552083335,\"iqm_confidence_interval\":[2300.1615397135415,4559.906412760417],\"agent\":\"SAC\"},{\"env_step\":290000.0,\"rew\":3364.4786376953125,\"rew_std\":1288.5817990288117,\"iqm\":3554.5956217447915,\"iqm_confidence_interval\":[1742.9095052083333,4730.11376953125],\"agent\":\"SAC\"},{\"env_step\":295000.0,\"rew\":3307.5587890625,\"rew_std\":822.0034940166391,\"iqm\":3246.4475911458335,\"iqm_confidence_interval\":[2356.705810546875,4260.758626302083],\"agent\":\"SAC\"},{\"env_step\":300000.0,\"rew\":3306.75478515625,\"rew_std\":806.6955844456846,\"iqm\":3014.4081217447915,\"iqm_confidence_interval\":[2705.4403483072915,4350.734456380208],\"agent\":\"SAC\"},{\"env_step\":305000.0,\"rew\":3625.849853515625,\"rew_std\":980.7105282519498,\"iqm\":3727.7289225260415,\"iqm_confidence_interval\":[2494.114013671875,4673.794596354167],\"agent\":\"SAC\"},{\"env_step\":310000.0,\"rew\":3425.81767578125,\"rew_std\":610.7586462895265,\"iqm\":3526.7610677083335,\"iqm_confidence_interval\":[2645.7422688802085,4039.8333333333335],\"agent\":\"SAC\"},{\"env_step\":315000.0,\"rew\":4136.97744140625,\"rew_std\":674.0098307295987,\"iqm\":4147.991780598958,\"iqm_confidence_interval\":[3440.5952962239585,4941.0205078125],\"agent\":\"SAC\"},{\"env_step\":320000.0,\"rew\":3651.937939453125,\"rew_std\":619.3627563908244,\"iqm\":3881.1543782552085,\"iqm_confidence_interval\":[2836.1516927083335,4108.755126953125],\"agent\":\"SAC\"},{\"env_step\":325000.0,\"rew\":3813.688037109375,\"rew_std\":619.8694270427005,\"iqm\":3843.1497395833335,\"iqm_confidence_interval\":[3050.9657389322915,4457.54150390625],\"agent\":\"SAC\"},{\"env_step\":330000.0,\"rew\":4016.08896484375,\"rew_std\":877.8359813093385,\"iqm\":4268.130859375,\"iqm_confidence_interval\":[2952.9524739583335,4768.163736979167],\"agent\":\"SAC\"},{\"env_step\":335000.0,\"rew\":3351.1053955078123,\"rew_std\":1340.7030859505958,\"iqm\":3409.1995849609375,\"iqm_confidence_interval\":[1728.4311930338542,4773.09912109375],\"agent\":\"SAC\"},{\"env_step\":340000.0,\"rew\":4029.405517578125,\"rew_std\":694.3555798668357,\"iqm\":4012.6536458333335,\"iqm_confidence_interval\":[3209.7360026041665,4837.2890625],\"agent\":\"SAC\"},{\"env_step\":345000.0,\"rew\":4357.635205078125,\"rew_std\":243.9905210451901,\"iqm\":4462.39453125,\"iqm_confidence_interval\":[4045.9969075520835,4517.338541666667],\"agent\":\"SAC\"},{\"env_step\":350000.0,\"rew\":3720.547509765625,\"rew_std\":718.0328362710919,\"iqm\":3745.6464029947915,\"iqm_confidence_interval\":[2883.9214680989585,4566.412434895833],\"agent\":\"SAC\"},{\"env_step\":355000.0,\"rew\":3888.85146484375,\"rew_std\":922.4671085394701,\"iqm\":3855.592529296875,\"iqm_confidence_interval\":[2889.3636067708335,4916.275065104167],\"agent\":\"SAC\"},{\"env_step\":360000.0,\"rew\":4492.90849609375,\"rew_std\":427.0038617487227,\"iqm\":4377.559244791667,\"iqm_confidence_interval\":[4113.779296875,5044.82470703125],\"agent\":\"SAC\"},{\"env_step\":365000.0,\"rew\":3824.1125,\"rew_std\":466.36924421733886,\"iqm\":3795.31396484375,\"iqm_confidence_interval\":[3277.9092610677085,4345.49267578125],\"agent\":\"SAC\"},{\"env_step\":370000.0,\"rew\":3793.528271484375,\"rew_std\":423.99898561883765,\"iqm\":3723.669677734375,\"iqm_confidence_interval\":[3378.8328450520835,4334.930257161458],\"agent\":\"SAC\"},{\"env_step\":375000.0,\"rew\":4652.83203125,\"rew_std\":312.9602999232348,\"iqm\":4602.4091796875,\"iqm_confidence_interval\":[4353.67041015625,5055.286946614583],\"agent\":\"SAC\"},{\"env_step\":380000.0,\"rew\":4107.30537109375,\"rew_std\":775.9508890311822,\"iqm\":4418.271809895833,\"iqm_confidence_interval\":[3123.5266927083335,4674.585774739583],\"agent\":\"SAC\"},{\"env_step\":385000.0,\"rew\":4406.541162109375,\"rew_std\":367.5362551001566,\"iqm\":4326.8076171875,\"iqm_confidence_interval\":[4060.4236653645835,4876.874674479167],\"agent\":\"SAC\"},{\"env_step\":390000.0,\"rew\":4134.431591796875,\"rew_std\":558.0600379590566,\"iqm\":4135.73583984375,\"iqm_confidence_interval\":[3465.0011393229165,4748.268391927083],\"agent\":\"SAC\"},{\"env_step\":395000.0,\"rew\":4550.18330078125,\"rew_std\":226.7923284440831,\"iqm\":4571.38525390625,\"iqm_confidence_interval\":[4289.759602864583,4809.229329427083],\"agent\":\"SAC\"},{\"env_step\":400000.0,\"rew\":3855.42265625,\"rew_std\":591.9787384668502,\"iqm\":3861.1411946614585,\"iqm_confidence_interval\":[3179.0397135416665,4565.104817708333],\"agent\":\"SAC\"},{\"env_step\":405000.0,\"rew\":4034.952392578125,\"rew_std\":541.1052846818027,\"iqm\":3977.388427734375,\"iqm_confidence_interval\":[3428.2539876302085,4668.5390625],\"agent\":\"SAC\"},{\"env_step\":410000.0,\"rew\":4127.98525390625,\"rew_std\":706.011830856558,\"iqm\":4253.806477864583,\"iqm_confidence_interval\":[3245.044921875,4861.324381510417],\"agent\":\"SAC\"},{\"env_step\":415000.0,\"rew\":4885.615625,\"rew_std\":229.95295192904064,\"iqm\":4960.629069010417,\"iqm_confidence_interval\":[4596.09375,5081.218098958333],\"agent\":\"SAC\"},{\"env_step\":420000.0,\"rew\":4803.430078125,\"rew_std\":513.4374209524454,\"iqm\":4912.762369791667,\"iqm_confidence_interval\":[4155.508138020833,5321.6865234375],\"agent\":\"SAC\"},{\"env_step\":425000.0,\"rew\":4393.957666015625,\"rew_std\":427.68871366006164,\"iqm\":4471.551432291667,\"iqm_confidence_interval\":[3847.70751953125,4829.618489583333],\"agent\":\"SAC\"},{\"env_step\":430000.0,\"rew\":3938.97265625,\"rew_std\":863.6299788862947,\"iqm\":3929.3496907552085,\"iqm_confidence_interval\":[2924.7867024739585,4926.1904296875],\"agent\":\"SAC\"},{\"env_step\":435000.0,\"rew\":4272.44384765625,\"rew_std\":997.5040725809271,\"iqm\":4437.238199869792,\"iqm_confidence_interval\":[2987.696044921875,5159.943522135417],\"agent\":\"SAC\"},{\"env_step\":440000.0,\"rew\":4870.52685546875,\"rew_std\":395.9247725369098,\"iqm\":4934.081380208333,\"iqm_confidence_interval\":[4360.022623697917,5268.646647135417],\"agent\":\"SAC\"},{\"env_step\":445000.0,\"rew\":4458.2224609375,\"rew_std\":696.4949334455875,\"iqm\":4540.100667317708,\"iqm_confidence_interval\":[3592.3868001302085,5141.94677734375],\"agent\":\"SAC\"},{\"env_step\":450000.0,\"rew\":4328.62060546875,\"rew_std\":492.0183738827475,\"iqm\":4298.752360026042,\"iqm_confidence_interval\":[3862.9173990885415,4926.407877604167],\"agent\":\"SAC\"},{\"env_step\":455000.0,\"rew\":4464.820166015625,\"rew_std\":971.6304504932598,\"iqm\":4835.134114583333,\"iqm_confidence_interval\":[3282.0231119791665,5189.101888020833],\"agent\":\"SAC\"},{\"env_step\":460000.0,\"rew\":4928.2134765625,\"rew_std\":171.17832240165666,\"iqm\":4899.203287760417,\"iqm_confidence_interval\":[4749.197591145833,5139.430501302083],\"agent\":\"SAC\"},{\"env_step\":465000.0,\"rew\":4820.03369140625,\"rew_std\":660.0672310678303,\"iqm\":5072.874348958333,\"iqm_confidence_interval\":[3981.7281901041665,5301.434407552083],\"agent\":\"SAC\"},{\"env_step\":470000.0,\"rew\":4975.690625,\"rew_std\":416.3427877328529,\"iqm\":5020.641764322917,\"iqm_confidence_interval\":[4453.964680989583,5423.12646484375],\"agent\":\"SAC\"},{\"env_step\":475000.0,\"rew\":4900.019287109375,\"rew_std\":549.8527721873079,\"iqm\":5025.114908854167,\"iqm_confidence_interval\":[4240.366373697917,5388.130208333333],\"agent\":\"SAC\"},{\"env_step\":480000.0,\"rew\":4733.905322265625,\"rew_std\":876.2419560665137,\"iqm\":4990.281087239583,\"iqm_confidence_interval\":[3567.96240234375,5448.71728515625],\"agent\":\"SAC\"},{\"env_step\":485000.0,\"rew\":4593.1205078125,\"rew_std\":480.91717325260674,\"iqm\":4764.295084635417,\"iqm_confidence_interval\":[4002.5148111979165,4965.733723958333],\"agent\":\"SAC\"},{\"env_step\":490000.0,\"rew\":4962.76171875,\"rew_std\":297.40314389316654,\"iqm\":5018.493977864583,\"iqm_confidence_interval\":[4597.000325520833,5266.617350260417],\"agent\":\"SAC\"},{\"env_step\":495000.0,\"rew\":4791.8896484375,\"rew_std\":331.4742715277328,\"iqm\":4652.039388020833,\"iqm_confidence_interval\":[4566.775227864583,5213.990071614583],\"agent\":\"SAC\"},{\"env_step\":500000.0,\"rew\":4950.46767578125,\"rew_std\":438.3517698707152,\"iqm\":4896.762532552083,\"iqm_confidence_interval\":[4529.126790364583,5503.750813802083],\"agent\":\"SAC\"},{\"env_step\":0.0,\"rew\":829.9240783691406,\"rew_std\":213.16720514662308,\"iqm\":924.9030558268229,\"iqm_confidence_interval\":[572.5575561523438,960.5938924153646],\"agent\":\"TD3\"},{\"env_step\":5000.0,\"rew\":539.5705200195313,\"rew_std\":114.70532131045941,\"iqm\":527.8493448893229,\"iqm_confidence_interval\":[419.621826171875,684.4314371744791],\"agent\":\"TD3\"},{\"env_step\":10000.0,\"rew\":756.800830078125,\"rew_std\":51.67615449175576,\"iqm\":761.0208943684896,\"iqm_confidence_interval\":[694.5470377604166,815.1962687174479],\"agent\":\"TD3\"},{\"env_step\":15000.0,\"rew\":735.1518920898437,\"rew_std\":49.79578714039183,\"iqm\":729.9076741536459,\"iqm_confidence_interval\":[681.5853881835938,797.6240641276041],\"agent\":\"TD3\"},{\"env_step\":20000.0,\"rew\":566.7820190429687,\"rew_std\":128.86571468965482,\"iqm\":559.4737752278646,\"iqm_confidence_interval\":[415.3776041666667,710.1801961263021],\"agent\":\"TD3\"},{\"env_step\":25000.0,\"rew\":546.2086364746094,\"rew_std\":114.56158906049062,\"iqm\":598.8281453450521,\"iqm_confidence_interval\":[404.30999755859375,615.4661051432291],\"agent\":\"TD3\"},{\"env_step\":30000.0,\"rew\":584.0201599121094,\"rew_std\":69.366430886614,\"iqm\":573.4462280273438,\"iqm_confidence_interval\":[512.0141398111979,671.46533203125],\"agent\":\"TD3\"},{\"env_step\":35000.0,\"rew\":569.2431213378907,\"rew_std\":150.3781384089316,\"iqm\":571.3526204427084,\"iqm_confidence_interval\":[402.2006429036458,750.3717244466146],\"agent\":\"TD3\"},{\"env_step\":40000.0,\"rew\":539.8349243164063,\"rew_std\":133.25074037366434,\"iqm\":563.3495279947916,\"iqm_confidence_interval\":[379.60784912109375,677.0299275716146],\"agent\":\"TD3\"},{\"env_step\":45000.0,\"rew\":592.5176391601562,\"rew_std\":179.49365470179063,\"iqm\":607.0400797526041,\"iqm_confidence_interval\":[377.11484781901044,785.9901326497396],\"agent\":\"TD3\"},{\"env_step\":50000.0,\"rew\":650.7749572753906,\"rew_std\":186.60011518107405,\"iqm\":687.1291097005209,\"iqm_confidence_interval\":[417.8120930989583,843.447265625],\"agent\":\"TD3\"},{\"env_step\":55000.0,\"rew\":702.7271850585937,\"rew_std\":233.75226525935827,\"iqm\":699.2322184244791,\"iqm_confidence_interval\":[425.00307210286456,949.2679036458334],\"agent\":\"TD3\"},{\"env_step\":60000.0,\"rew\":770.480322265625,\"rew_std\":105.0838953860834,\"iqm\":786.7170613606771,\"iqm_confidence_interval\":[647.1668497721354,884.4464314778646],\"agent\":\"TD3\"},{\"env_step\":65000.0,\"rew\":767.2954956054688,\"rew_std\":185.80310917721403,\"iqm\":825.8852335611979,\"iqm_confidence_interval\":[521.1190795898438,921.1991373697916],\"agent\":\"TD3\"},{\"env_step\":70000.0,\"rew\":881.9124389648438,\"rew_std\":124.16713120431044,\"iqm\":892.2832438151041,\"iqm_confidence_interval\":[726.3288167317709,1013.8692016601562],\"agent\":\"TD3\"},{\"env_step\":75000.0,\"rew\":809.5596252441406,\"rew_std\":210.11544260847276,\"iqm\":840.7815755208334,\"iqm_confidence_interval\":[538.25390625,1003.7309773763021],\"agent\":\"TD3\"},{\"env_step\":80000.0,\"rew\":914.9486206054687,\"rew_std\":185.53253372227354,\"iqm\":866.2797037760416,\"iqm_confidence_interval\":[749.0588175455729,1146.3099772135417],\"agent\":\"TD3\"},{\"env_step\":85000.0,\"rew\":1038.5048583984376,\"rew_std\":354.55029263760997,\"iqm\":991.1633097330729,\"iqm_confidence_interval\":[685.3418986002604,1467.1781005859375],\"agent\":\"TD3\"},{\"env_step\":90000.0,\"rew\":1095.1505615234375,\"rew_std\":294.7722406276977,\"iqm\":1029.7857259114583,\"iqm_confidence_interval\":[825.5033569335938,1483.0994873046875],\"agent\":\"TD3\"},{\"env_step\":95000.0,\"rew\":926.5935485839843,\"rew_std\":358.78989947098245,\"iqm\":1008.4535522460938,\"iqm_confidence_interval\":[481.00408935546875,1286.0092366536458],\"agent\":\"TD3\"},{\"env_step\":100000.0,\"rew\":1249.812744140625,\"rew_std\":468.10361371058315,\"iqm\":1109.7845255533855,\"iqm_confidence_interval\":[836.6089070638021,1829.5582682291667],\"agent\":\"TD3\"},{\"env_step\":105000.0,\"rew\":1086.3578125,\"rew_std\":289.683933445901,\"iqm\":1119.0757853190105,\"iqm_confidence_interval\":[748.8553873697916,1412.0789794921875],\"agent\":\"TD3\"},{\"env_step\":110000.0,\"rew\":1384.1274536132812,\"rew_std\":511.3336586709261,\"iqm\":1256.2594807942708,\"iqm_confidence_interval\":[882.284423828125,2008.5397135416667],\"agent\":\"TD3\"},{\"env_step\":115000.0,\"rew\":1471.0005249023438,\"rew_std\":620.7393531156671,\"iqm\":1336.388448079427,\"iqm_confidence_interval\":[840.9969278971354,2220.077189127604],\"agent\":\"TD3\"},{\"env_step\":120000.0,\"rew\":1260.1736083984374,\"rew_std\":469.7023744926701,\"iqm\":1260.9583740234375,\"iqm_confidence_interval\":[691.15625,1786.0275065104167],\"agent\":\"TD3\"},{\"env_step\":125000.0,\"rew\":1511.67509765625,\"rew_std\":648.2241845549785,\"iqm\":1302.0400797526042,\"iqm_confidence_interval\":[942.7909342447916,2330.532185872396],\"agent\":\"TD3\"},{\"env_step\":130000.0,\"rew\":1458.1156372070313,\"rew_std\":549.6783235933749,\"iqm\":1327.2992350260417,\"iqm_confidence_interval\":[938.1090901692709,2141.26123046875],\"agent\":\"TD3\"},{\"env_step\":135000.0,\"rew\":1574.3567504882812,\"rew_std\":429.26752013044785,\"iqm\":1604.1842854817708,\"iqm_confidence_interval\":[1050.6182454427083,2058.0555419921875],\"agent\":\"TD3\"},{\"env_step\":140000.0,\"rew\":1368.9617309570312,\"rew_std\":371.94584620777067,\"iqm\":1352.2195638020833,\"iqm_confidence_interval\":[931.149658203125,1769.9747314453125],\"agent\":\"TD3\"},{\"env_step\":145000.0,\"rew\":1501.2026000976562,\"rew_std\":400.98189748060196,\"iqm\":1487.1974283854167,\"iqm_confidence_interval\":[1043.6548258463542,1973.142578125],\"agent\":\"TD3\"},{\"env_step\":150000.0,\"rew\":1727.3000610351562,\"rew_std\":601.7172140285786,\"iqm\":1626.8328043619792,\"iqm_confidence_interval\":[1127.7975260416667,2441.1529541015625],\"agent\":\"TD3\"},{\"env_step\":155000.0,\"rew\":1559.7651000976562,\"rew_std\":446.35813544513456,\"iqm\":1646.3952229817708,\"iqm_confidence_interval\":[1017.4626057942709,2022.75537109375],\"agent\":\"TD3\"},{\"env_step\":160000.0,\"rew\":1764.975537109375,\"rew_std\":732.2193744679822,\"iqm\":1591.4624837239583,\"iqm_confidence_interval\":[1065.2289632161458,2654.5350748697915],\"agent\":\"TD3\"},{\"env_step\":165000.0,\"rew\":1822.7712280273438,\"rew_std\":678.5803775092464,\"iqm\":1730.8170572916667,\"iqm_confidence_interval\":[1132.1419677734375,2662.698689778646],\"agent\":\"TD3\"},{\"env_step\":170000.0,\"rew\":1985.6068969726562,\"rew_std\":728.3031488916075,\"iqm\":1910.0275472005208,\"iqm_confidence_interval\":[1256.8134765625,2821.941202799479],\"agent\":\"TD3\"},{\"env_step\":175000.0,\"rew\":2109.0537353515624,\"rew_std\":782.0241694379353,\"iqm\":2076.4483642578125,\"iqm_confidence_interval\":[1259.3393147786458,3005.3229166666665],\"agent\":\"TD3\"},{\"env_step\":180000.0,\"rew\":2179.0469360351562,\"rew_std\":905.9575695451357,\"iqm\":2056.8221028645835,\"iqm_confidence_interval\":[1259.8915608723958,3278.7978515625],\"agent\":\"TD3\"},{\"env_step\":185000.0,\"rew\":2229.725476074219,\"rew_std\":865.0804432932722,\"iqm\":2190.3130696614585,\"iqm_confidence_interval\":[1324.1541748046875,3203.8898111979165],\"agent\":\"TD3\"},{\"env_step\":190000.0,\"rew\":2130.791638183594,\"rew_std\":708.9686972152747,\"iqm\":2196.904541015625,\"iqm_confidence_interval\":[1300.7169189453125,2856.4830729166665],\"agent\":\"TD3\"},{\"env_step\":195000.0,\"rew\":2176.3774291992186,\"rew_std\":1007.79712057956,\"iqm\":2057.643310546875,\"iqm_confidence_interval\":[1057.2288818359375,3397.5015462239585],\"agent\":\"TD3\"},{\"env_step\":200000.0,\"rew\":2305.0114013671873,\"rew_std\":957.4217813148268,\"iqm\":2259.0384928385415,\"iqm_confidence_interval\":[1214.5,3451.973876953125],\"agent\":\"TD3\"},{\"env_step\":205000.0,\"rew\":2238.530725097656,\"rew_std\":1034.0895668294195,\"iqm\":2172.598429361979,\"iqm_confidence_interval\":[1095.5213216145833,3516.4203287760415],\"agent\":\"TD3\"},{\"env_step\":210000.0,\"rew\":2403.3504272460937,\"rew_std\":953.5621026200623,\"iqm\":2470.398193359375,\"iqm_confidence_interval\":[1326.7755126953125,3386.0760904947915],\"agent\":\"TD3\"},{\"env_step\":215000.0,\"rew\":2477.0895263671873,\"rew_std\":938.5595760089883,\"iqm\":2550.865966796875,\"iqm_confidence_interval\":[1381.4463704427083,3520.93017578125],\"agent\":\"TD3\"},{\"env_step\":220000.0,\"rew\":2316.696423339844,\"rew_std\":853.0289853639126,\"iqm\":2350.9966634114585,\"iqm_confidence_interval\":[1342.1841227213542,3271.8939615885415],\"agent\":\"TD3\"},{\"env_step\":225000.0,\"rew\":2547.7952514648437,\"rew_std\":931.6966721701526,\"iqm\":2657.4053548177085,\"iqm_confidence_interval\":[1460.1538492838542,3533.2898763020835],\"agent\":\"TD3\"},{\"env_step\":230000.0,\"rew\":2465.588232421875,\"rew_std\":943.1721250006842,\"iqm\":2511.775390625,\"iqm_confidence_interval\":[1387.5060221354167,3513.1005045572915],\"agent\":\"TD3\"},{\"env_step\":235000.0,\"rew\":2547.3243774414063,\"rew_std\":940.3479030837134,\"iqm\":2770.5331217447915,\"iqm_confidence_interval\":[1395.3839925130208,3487.5193684895835],\"agent\":\"TD3\"},{\"env_step\":240000.0,\"rew\":2731.887939453125,\"rew_std\":1114.621718219357,\"iqm\":2842.3636067708335,\"iqm_confidence_interval\":[1444.1981608072917,3993.7285970052085],\"agent\":\"TD3\"},{\"env_step\":245000.0,\"rew\":2679.134765625,\"rew_std\":983.2848046758737,\"iqm\":2902.3509928385415,\"iqm_confidence_interval\":[1481.5174967447917,3653.83642578125],\"agent\":\"TD3\"},{\"env_step\":250000.0,\"rew\":2762.9610595703125,\"rew_std\":1060.952467688137,\"iqm\":2915.1644694010415,\"iqm_confidence_interval\":[1509.3499348958333,3902.3920084635415],\"agent\":\"TD3\"},{\"env_step\":255000.0,\"rew\":2753.7060668945314,\"rew_std\":1110.3489032409016,\"iqm\":2886.451171875,\"iqm_confidence_interval\":[1436.4121500651042,3987.7591959635415],\"agent\":\"TD3\"},{\"env_step\":260000.0,\"rew\":2606.6949462890625,\"rew_std\":1016.9661334599042,\"iqm\":2760.07666015625,\"iqm_confidence_interval\":[1398.0731608072917,3715.1659342447915],\"agent\":\"TD3\"},{\"env_step\":265000.0,\"rew\":2693.0078369140624,\"rew_std\":1088.9586023803274,\"iqm\":2883.4249674479165,\"iqm_confidence_interval\":[1411.1376953125,3859.88525390625],\"agent\":\"TD3\"},{\"env_step\":270000.0,\"rew\":2813.0802001953125,\"rew_std\":1107.950631195792,\"iqm\":2979.3451334635415,\"iqm_confidence_interval\":[1505.247314453125,4003.2862955729165],\"agent\":\"TD3\"},{\"env_step\":275000.0,\"rew\":2840.199169921875,\"rew_std\":1205.5980886899304,\"iqm\":2990.45751953125,\"iqm_confidence_interval\":[1433.0826822916667,4187.828857421875],\"agent\":\"TD3\"},{\"env_step\":280000.0,\"rew\":2649.196984863281,\"rew_std\":977.7908186825631,\"iqm\":2771.1956380208335,\"iqm_confidence_interval\":[1496.7768961588542,3610.7386067708335],\"agent\":\"TD3\"},{\"env_step\":285000.0,\"rew\":2984.6728271484376,\"rew_std\":1222.1267736710947,\"iqm\":3239.305419921875,\"iqm_confidence_interval\":[1495.8108723958333,4246.451497395833],\"agent\":\"TD3\"},{\"env_step\":290000.0,\"rew\":2653.69794921875,\"rew_std\":1022.5000328919515,\"iqm\":2808.8805338541665,\"iqm_confidence_interval\":[1406.3455403645833,3749.87744140625],\"agent\":\"TD3\"},{\"env_step\":295000.0,\"rew\":2903.6751953125,\"rew_std\":1186.7828461038976,\"iqm\":3059.5785319010415,\"iqm_confidence_interval\":[1480.8075358072917,4207.113037109375],\"agent\":\"TD3\"},{\"env_step\":300000.0,\"rew\":3046.1340087890626,\"rew_std\":1347.5070082991244,\"iqm\":3239.381591796875,\"iqm_confidence_interval\":[1406.5304361979167,4510.792643229167],\"agent\":\"TD3\"},{\"env_step\":305000.0,\"rew\":3028.121826171875,\"rew_std\":1355.1566766121175,\"iqm\":3201.7381184895835,\"iqm_confidence_interval\":[1385.89306640625,4520.408040364583],\"agent\":\"TD3\"},{\"env_step\":310000.0,\"rew\":3045.340930175781,\"rew_std\":1369.5901648511212,\"iqm\":3232.55810546875,\"iqm_confidence_interval\":[1463.2810465494792,4562.6923828125],\"agent\":\"TD3\"},{\"env_step\":315000.0,\"rew\":3028.4850219726563,\"rew_std\":1246.8533587402467,\"iqm\":3266.2167154947915,\"iqm_confidence_interval\":[1508.8992919921875,4332.45166015625],\"agent\":\"TD3\"},{\"env_step\":320000.0,\"rew\":2986.0309448242188,\"rew_std\":1193.0953476903455,\"iqm\":3256.34033203125,\"iqm_confidence_interval\":[1530.6516520182292,4195.118489583333],\"agent\":\"TD3\"},{\"env_step\":325000.0,\"rew\":3073.6957885742186,\"rew_std\":1349.3847990133245,\"iqm\":3310.2652994791665,\"iqm_confidence_interval\":[1459.5026448567708,4514.5234375],\"agent\":\"TD3\"},{\"env_step\":330000.0,\"rew\":3099.622509765625,\"rew_std\":1356.848981051641,\"iqm\":3403.2476399739585,\"iqm_confidence_interval\":[1420.4486490885417,4465.79931640625],\"agent\":\"TD3\"},{\"env_step\":335000.0,\"rew\":3085.056433105469,\"rew_std\":1285.395665056501,\"iqm\":3366.172607421875,\"iqm_confidence_interval\":[1542.4388427734375,4402.2119140625],\"agent\":\"TD3\"},{\"env_step\":340000.0,\"rew\":3071.9107177734377,\"rew_std\":1235.7570850757356,\"iqm\":3367.0099283854165,\"iqm_confidence_interval\":[1569.3037923177083,4306.1474609375],\"agent\":\"TD3\"},{\"env_step\":345000.0,\"rew\":3190.3670043945312,\"rew_std\":1315.1221008980763,\"iqm\":3472.8580729166665,\"iqm_confidence_interval\":[1540.4414469401042,4516.2958984375],\"agent\":\"TD3\"},{\"env_step\":350000.0,\"rew\":3151.562292480469,\"rew_std\":1319.802287809035,\"iqm\":3421.1626790364585,\"iqm_confidence_interval\":[1482.6082763671875,4491.857421875],\"agent\":\"TD3\"},{\"env_step\":355000.0,\"rew\":2993.9238891601562,\"rew_std\":1345.2920648248396,\"iqm\":3180.001708984375,\"iqm_confidence_interval\":[1400.4963785807292,4477.604817708333],\"agent\":\"TD3\"},{\"env_step\":360000.0,\"rew\":3240.46015625,\"rew_std\":1393.0734888624074,\"iqm\":3622.8191731770835,\"iqm_confidence_interval\":[1470.490478515625,4551.108072916667],\"agent\":\"TD3\"},{\"env_step\":365000.0,\"rew\":3034.42294921875,\"rew_std\":1246.2603451907094,\"iqm\":3263.3148600260415,\"iqm_confidence_interval\":[1509.1800130208333,4335.368489583333],\"agent\":\"TD3\"},{\"env_step\":370000.0,\"rew\":3270.672399902344,\"rew_std\":1393.160059416677,\"iqm\":3608.8076171875,\"iqm_confidence_interval\":[1505.1509195963542,4633.247395833333],\"agent\":\"TD3\"},{\"env_step\":375000.0,\"rew\":3249.9776000976562,\"rew_std\":1413.7088589889643,\"iqm\":3535.9663899739585,\"iqm_confidence_interval\":[1565.8721110026042,4724.507975260417],\"agent\":\"TD3\"},{\"env_step\":380000.0,\"rew\":3273.317138671875,\"rew_std\":1383.7569272549713,\"iqm\":3605.3651529947915,\"iqm_confidence_interval\":[1515.472412109375,4610.3828125],\"agent\":\"TD3\"},{\"env_step\":385000.0,\"rew\":3252.874890136719,\"rew_std\":1415.2156000510515,\"iqm\":3531.2469075520835,\"iqm_confidence_interval\":[1524.0929768880208,4721.978678385417],\"agent\":\"TD3\"},{\"env_step\":390000.0,\"rew\":3301.0752197265624,\"rew_std\":1415.3416560680325,\"iqm\":3627.8466796875,\"iqm_confidence_interval\":[1514.631103515625,4702.47021484375],\"agent\":\"TD3\"},{\"env_step\":395000.0,\"rew\":3278.423193359375,\"rew_std\":1397.7760225048944,\"iqm\":3655.669189453125,\"iqm_confidence_interval\":[1503.4281412760417,4591.7353515625],\"agent\":\"TD3\"},{\"env_step\":400000.0,\"rew\":2884.5741577148438,\"rew_std\":1445.511380441809,\"iqm\":2972.203084309896,\"iqm_confidence_interval\":[1176.9991861979167,4553.221842447917],\"agent\":\"TD3\"},{\"env_step\":405000.0,\"rew\":3330.7515502929687,\"rew_std\":1387.3228046538802,\"iqm\":3683.735107421875,\"iqm_confidence_interval\":[1551.3370361328125,4650.876139322917],\"agent\":\"TD3\"},{\"env_step\":410000.0,\"rew\":2710.1162353515624,\"rew_std\":1382.221121753278,\"iqm\":2593.720743815104,\"iqm_confidence_interval\":[1125.9298095703125,4308.127115885417],\"agent\":\"TD3\"},{\"env_step\":415000.0,\"rew\":3388.8285400390623,\"rew_std\":1412.7014251822077,\"iqm\":3769.463623046875,\"iqm_confidence_interval\":[1562.707763671875,4672.133626302083],\"agent\":\"TD3\"},{\"env_step\":420000.0,\"rew\":3448.588269042969,\"rew_std\":1405.2624520259374,\"iqm\":3859.49951171875,\"iqm_confidence_interval\":[1635.9779052734375,4720.6904296875],\"agent\":\"TD3\"},{\"env_step\":425000.0,\"rew\":3365.4913940429688,\"rew_std\":1484.9048095701287,\"iqm\":3660.5003255208335,\"iqm_confidence_interval\":[1520.9411214192708,4900.017740885417],\"agent\":\"TD3\"},{\"env_step\":430000.0,\"rew\":3435.064733886719,\"rew_std\":1556.7886671552396,\"iqm\":3886.5701497395835,\"iqm_confidence_interval\":[1424.7179768880208,4846.15966796875],\"agent\":\"TD3\"},{\"env_step\":435000.0,\"rew\":3276.301806640625,\"rew_std\":1451.3867925958466,\"iqm\":3560.5865071614585,\"iqm_confidence_interval\":[1544.50732421875,4797.141927083333],\"agent\":\"TD3\"},{\"env_step\":440000.0,\"rew\":3431.6516845703127,\"rew_std\":1528.6457745757746,\"iqm\":3875.1138509114585,\"iqm_confidence_interval\":[1508.0724283854167,4847.526204427083],\"agent\":\"TD3\"},{\"env_step\":445000.0,\"rew\":3552.8244384765626,\"rew_std\":1487.1323046080424,\"iqm\":3961.4072265625,\"iqm_confidence_interval\":[1658.235107421875,4947.947265625],\"agent\":\"TD3\"},{\"env_step\":450000.0,\"rew\":3356.4051879882813,\"rew_std\":1376.3847353292676,\"iqm\":3702.5616861979165,\"iqm_confidence_interval\":[1633.0870361328125,4680.08984375],\"agent\":\"TD3\"},{\"env_step\":455000.0,\"rew\":3563.1517944335938,\"rew_std\":1491.4385603812143,\"iqm\":3977.2855631510415,\"iqm_confidence_interval\":[1634.7741292317708,4936.918782552083],\"agent\":\"TD3\"},{\"env_step\":460000.0,\"rew\":3524.6409545898437,\"rew_std\":1430.2851011633504,\"iqm\":3919.8612467447915,\"iqm_confidence_interval\":[1705.1669514973958,4847.28564453125],\"agent\":\"TD3\"},{\"env_step\":465000.0,\"rew\":3604.3880004882812,\"rew_std\":1518.1113450254556,\"iqm\":3995.2644856770835,\"iqm_confidence_interval\":[1668.4000244140625,5048.280924479167],\"agent\":\"TD3\"},{\"env_step\":470000.0,\"rew\":3391.0613037109374,\"rew_std\":1477.569491993997,\"iqm\":3703.388916015625,\"iqm_confidence_interval\":[1602.2677408854167,4914.701822916667],\"agent\":\"TD3\"},{\"env_step\":475000.0,\"rew\":3521.8194580078125,\"rew_std\":1549.656846893595,\"iqm\":3898.186279296875,\"iqm_confidence_interval\":[1528.638427734375,4991.42919921875],\"agent\":\"TD3\"},{\"env_step\":480000.0,\"rew\":3536.843542480469,\"rew_std\":1479.3773751625279,\"iqm\":3875.009521484375,\"iqm_confidence_interval\":[1649.8277587890625,4982.876139322917],\"agent\":\"TD3\"},{\"env_step\":485000.0,\"rew\":3698.1513671875,\"rew_std\":1529.7254531111644,\"iqm\":4141.316487630208,\"iqm_confidence_interval\":[1714.9497884114583,5071.2822265625],\"agent\":\"TD3\"},{\"env_step\":490000.0,\"rew\":3550.1833251953126,\"rew_std\":1494.588728177185,\"iqm\":3913.3807779947915,\"iqm_confidence_interval\":[1690.8838704427083,5020.81591796875],\"agent\":\"TD3\"},{\"env_step\":495000.0,\"rew\":3703.76689453125,\"rew_std\":1576.607793115339,\"iqm\":4101.045084635417,\"iqm_confidence_interval\":[1677.4986165364583,5207.55615234375],\"agent\":\"TD3\"},{\"env_step\":500000.0,\"rew\":3575.063610839844,\"rew_std\":1511.873916898478,\"iqm\":3972.8521321614585,\"iqm_confidence_interval\":[1623.5158284505208,4996.941080729167],\"agent\":\"TD3\"}]" }, { "path": "docs/_static/js/v5.json", "content": "{\n \"$ref\": \"#/definitions/TopLevelSpec\",\n \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n \"definitions\": {\n \"Aggregate\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/NonArgAggregateOp\"\n },\n {\n \"$ref\": \"#/definitions/ArgmaxDef\"\n },\n {\n \"$ref\": \"#/definitions/ArgminDef\"\n }\n ]\n },\n \"AggregateOp\": {\n \"enum\": [\n \"argmax\",\n \"argmin\",\n \"average\",\n \"count\",\n \"distinct\",\n \"max\",\n \"mean\",\n \"median\",\n \"min\",\n \"missing\",\n \"product\",\n \"q1\",\n \"q3\",\n \"ci0\",\n \"ci1\",\n \"stderr\",\n \"stdev\",\n \"stdevp\",\n \"sum\",\n \"valid\",\n \"values\",\n \"variance\",\n \"variancep\"\n ],\n \"type\": \"string\"\n },\n \"AggregateTransform\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"description\": \"Array of objects that define fields to aggregate.\",\n \"items\": {\n \"$ref\": \"#/definitions/AggregatedFieldDef\"\n },\n \"type\": \"array\"\n },\n \"groupby\": {\n \"description\": \"The data fields to group by. If not specified, a single group containing all data objects will be used.\",\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"aggregate\"\n ],\n \"type\": \"object\"\n },\n \"AggregatedFieldDef\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"as\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"The output field names to use for each aggregated field.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"The data field for which to compute aggregate function. This is required for all aggregation operations except `\\\"count\\\"`.\"\n },\n \"op\": {\n \"$ref\": \"#/definitions/AggregateOp\",\n \"description\": \"The aggregation operation to apply to the fields (e.g., `\\\"sum\\\"`, `\\\"average\\\"`, or `\\\"count\\\"`). See the [full list of supported aggregation operations](https://vega.github.io/vega-lite/docs/aggregate.html#ops) for more information.\"\n }\n },\n \"required\": [\n \"op\",\n \"as\"\n ],\n \"type\": \"object\"\n },\n \"Align\": {\n \"enum\": [\n \"left\",\n \"center\",\n \"right\"\n ],\n \"type\": \"string\"\n },\n \"AllSortString\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SortOrder\"\n },\n {\n \"$ref\": \"#/definitions/SortByChannel\"\n },\n {\n \"$ref\": \"#/definitions/SortByChannelDesc\"\n }\n ]\n },\n \"AnyMark\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/CompositeMark\"\n },\n {\n \"$ref\": \"#/definitions/CompositeMarkDef\"\n },\n {\n \"$ref\": \"#/definitions/Mark\"\n },\n {\n \"$ref\": \"#/definitions/MarkDef\"\n }\n ]\n },\n \"AnyMarkConfig\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/MarkConfig\"\n },\n {\n \"$ref\": \"#/definitions/AreaConfig\"\n },\n {\n \"$ref\": \"#/definitions/BarConfig\"\n },\n {\n \"$ref\": \"#/definitions/RectConfig\"\n },\n {\n \"$ref\": \"#/definitions/LineConfig\"\n },\n {\n \"$ref\": \"#/definitions/TickConfig\"\n }\n ]\n },\n \"AreaConfig\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"align\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Align\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The horizontal alignment of the text or ranged marks (area, bar, image, rect, rule). One of `\\\"left\\\"`, `\\\"right\\\"`, `\\\"center\\\"`.\\n\\n__Note:__ Expression reference is *not* supported for range marks.\"\n },\n \"angle\": {\n \"anyOf\": [\n {\n \"description\": \"The rotation angle of the text, in degrees.\",\n \"maximum\": 360,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"aria\": {\n \"anyOf\": [\n {\n \"description\": \"A boolean flag indicating if [ARIA attributes](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) should be included (SVG output only). If `false`, the \\\"aria-hidden\\\" attribute will be set on the output SVG element, removing the mark item from the ARIA accessibility tree.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"ariaRole\": {\n \"anyOf\": [\n {\n \"description\": \"Sets the type of user interface element of the mark item for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If specified, this property determines the \\\"role\\\" attribute. Warning: this property is experimental and may be changed in the future.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"ariaRoleDescription\": {\n \"anyOf\": [\n {\n \"description\": \"A human-readable, author-localized description for the role of the mark item for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If specified, this property determines the \\\"aria-roledescription\\\" attribute. Warning: this property is experimental and may be changed in the future.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"aspect\": {\n \"anyOf\": [\n {\n \"description\": \"Whether to keep aspect ratio of image marks.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"baseline\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextBaseline\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"For text marks, the vertical text baseline. One of `\\\"alphabetic\\\"` (default), `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`, `\\\"line-top\\\"`, `\\\"line-bottom\\\"`, or an expression reference that provides one of the valid values. The `\\\"line-top\\\"` and `\\\"line-bottom\\\"` values operate similarly to `\\\"top\\\"` and `\\\"bottom\\\"`, but are calculated relative to the `lineHeight` rather than `fontSize` alone.\\n\\nFor range marks, the vertical alignment of the marks. One of `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`.\\n\\n__Note:__ Expression reference is *not* supported for range marks.\"\n },\n \"blend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Blend\",\n \"description\": \"The color blend mode for drawing an item on its current background. Any valid [CSS mix-blend-mode](https://developer.mozilla.org/en-US/docs/Web/CSS/mix-blend-mode) value can be used.\\n\\n__Default value: `\\\"source-over\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"color\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default color.\\n\\n__Default value:__ ■ `\\\"#4682b4\\\"`\\n\\n__Note:__\\n- This property cannot be used in a [style config](https://vega.github.io/vega-lite/docs/mark.html#style-config).\\n- The `fill` and `stroke` properties have higher precedence than `color` and will override `color`.\"\n },\n \"cornerRadius\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles or arcs' corners.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusBottomLeft\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' bottom left corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusBottomRight\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' bottom right corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusTopLeft\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' top right corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusTopRight\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' top left corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cursor\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Cursor\",\n \"description\": \"The mouse cursor used over the mark. Any valid [CSS cursor type](https://developer.mozilla.org/en-US/docs/Web/CSS/cursor#Values) can be used.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"description\": {\n \"anyOf\": [\n {\n \"description\": \"A text description of the mark item for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If specified, this property determines the [\\\"aria-label\\\" attribute](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/ARIA_Techniques/Using_the_aria-label_attribute).\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"dir\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextDirection\",\n \"description\": \"The direction of the text. One of `\\\"ltr\\\"` (left-to-right) or `\\\"rtl\\\"` (right-to-left). This property determines on which side is truncated in response to the limit parameter.\\n\\n__Default value:__ `\\\"ltr\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"dx\": {\n \"anyOf\": [\n {\n \"description\": \"The horizontal offset, in pixels, between the text label and its anchor point. The offset is applied after rotation by the _angle_ property.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"dy\": {\n \"anyOf\": [\n {\n \"description\": \"The vertical offset, in pixels, between the text label and its anchor point. The offset is applied after rotation by the _angle_ property.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"ellipsis\": {\n \"anyOf\": [\n {\n \"description\": \"The ellipsis string for text truncated in response to the limit parameter.\\n\\n__Default value:__ `\\\"…\\\"`\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"endAngle\": {\n \"anyOf\": [\n {\n \"description\": \"The end angle in radians for arc marks. A value of `0` indicates up (north), increasing values proceed clockwise.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fill\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default fill color. This property has higher precedence than `config.color`. Set to `null` to remove fill.\\n\\n__Default value:__ (None)\"\n },\n \"fillOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"The fill opacity (value between [0,1]).\\n\\n__Default value:__ `1`\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"filled\": {\n \"description\": \"Whether the mark's color should be used as fill color instead of stroke color.\\n\\n__Default value:__ `false` for all `point`, `line`, and `rule` marks as well as `geoshape` marks for [`graticule`](https://vega.github.io/vega-lite/docs/data.html#graticule) data sources; otherwise, `true`.\\n\\n__Note:__ This property cannot be used in a [style config](https://vega.github.io/vega-lite/docs/mark.html#style-config).\",\n \"type\": \"boolean\"\n },\n \"font\": {\n \"anyOf\": [\n {\n \"description\": \"The typeface to set the text in (e.g., `\\\"Helvetica Neue\\\"`).\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontSize\": {\n \"anyOf\": [\n {\n \"description\": \"The font size, in pixels.\\n\\n__Default value:__ `11`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontStyle\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\",\n \"description\": \"The font style (e.g., `\\\"italic\\\"`).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontWeight\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\",\n \"description\": \"The font weight. This can be either a string (e.g `\\\"bold\\\"`, `\\\"normal\\\"`) or a number (`100`, `200`, `300`, ..., `900` where `\\\"normal\\\"` = `400` and `\\\"bold\\\"` = `700`).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"height\": {\n \"anyOf\": [\n {\n \"description\": \"Height of the marks.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"href\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/URI\",\n \"description\": \"A URL to load upon mouse click. If defined, the mark acts as a hyperlink.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"innerRadius\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The inner radius in pixels of arc marks. `innerRadius` is an alias for `radius2`.\\n\\n__Default value:__ `0`\",\n \"minimum\": 0\n },\n \"interpolate\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Interpolate\",\n \"description\": \"The line interpolation method to use for line and area marks. One of the following:\\n- `\\\"linear\\\"`: piecewise linear segments, as in a polyline.\\n- `\\\"linear-closed\\\"`: close the linear segments to form a polygon.\\n- `\\\"step\\\"`: alternate between horizontal and vertical segments, as in a step function.\\n- `\\\"step-before\\\"`: alternate between vertical and horizontal segments, as in a step function.\\n- `\\\"step-after\\\"`: alternate between horizontal and vertical segments, as in a step function.\\n- `\\\"basis\\\"`: a B-spline, with control point duplication on the ends.\\n- `\\\"basis-open\\\"`: an open B-spline; may not intersect the start or end.\\n- `\\\"basis-closed\\\"`: a closed B-spline, as in a loop.\\n- `\\\"cardinal\\\"`: a Cardinal spline, with control point duplication on the ends.\\n- `\\\"cardinal-open\\\"`: an open Cardinal spline; may not intersect the start or end, but will intersect other control points.\\n- `\\\"cardinal-closed\\\"`: a closed Cardinal spline, as in a loop.\\n- `\\\"bundle\\\"`: equivalent to basis, except the tension parameter is used to straighten the spline.\\n- `\\\"monotone\\\"`: cubic interpolation that preserves monotonicity in y.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"invalid\": {\n \"description\": \"Defines how Vega-Lite should handle marks for invalid values (`null` and `NaN`).\\n- If set to `\\\"filter\\\"` (default), all data items with null values will be skipped (for line, trail, and area marks) or filtered (for other marks).\\n- If `null`, all data items are included. In this case, invalid values will be interpreted as zeroes.\",\n \"enum\": [\n \"filter\",\n null\n ],\n \"type\": [\n \"string\",\n \"null\"\n ]\n },\n \"limit\": {\n \"anyOf\": [\n {\n \"description\": \"The maximum length of the text mark in pixels. The text value will be automatically truncated if the rendered size exceeds the limit.\\n\\n__Default value:__ `0` -- indicating no limit\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"line\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/OverlayMarkDef\"\n }\n ],\n \"description\": \"A flag for overlaying line on top of area marks, or an object defining the properties of the overlayed lines.\\n\\n- If this value is an empty object (`{}`) or `true`, lines with default properties will be used.\\n\\n- If this value is `false`, no lines would be automatically added to area marks.\\n\\n__Default value:__ `false`.\"\n },\n \"lineBreak\": {\n \"anyOf\": [\n {\n \"description\": \"A delimiter, such as a newline character, upon which to break text strings into multiple lines. This property is ignored if the text is array-valued.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"lineHeight\": {\n \"anyOf\": [\n {\n \"description\": \"The line height in pixels (the spacing between subsequent lines of text) for multi-line text marks.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"opacity\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The overall opacity (value between [0,1]).\\n\\n__Default value:__ `0.7` for non-aggregate plots with `point`, `tick`, `circle`, or `square` marks or layered `bar` charts and `1` otherwise.\",\n \"maximum\": 1,\n \"minimum\": 0\n },\n \"order\": {\n \"description\": \"For line and trail marks, this `order` property can be set to `null` or `false` to make the lines use the original order in the data sources.\",\n \"type\": [\n \"null\",\n \"boolean\"\n ]\n },\n \"orient\": {\n \"$ref\": \"#/definitions/Orientation\",\n \"description\": \"The orientation of a non-stacked bar, tick, area, and line charts. The value is either horizontal (default) or vertical.\\n- For bar, rule and tick, this determines whether the size of the bar and tick should be applied to x or y dimension.\\n- For area, this property determines the orient property of the Vega output.\\n- For line and trail marks, this property determines the sort order of the points in the line if `config.sortLineBy` is not specified. For stacked charts, this is always determined by the orientation of the stack; therefore explicitly specified value will be ignored.\"\n },\n \"outerRadius\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The outer radius in pixels of arc marks. `outerRadius` is an alias for `radius`.\\n\\n__Default value:__ `0`\",\n \"minimum\": 0\n },\n \"padAngle\": {\n \"anyOf\": [\n {\n \"description\": \"The angular padding applied to sides of the arc, in radians.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"point\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/OverlayMarkDef\"\n },\n {\n \"const\": \"transparent\",\n \"type\": \"string\"\n }\n ],\n \"description\": \"A flag for overlaying points on top of line or area marks, or an object defining the properties of the overlayed points.\\n\\n- If this property is `\\\"transparent\\\"`, transparent points will be used (for enhancing tooltips and selections).\\n\\n- If this property is an empty object (`{}`) or `true`, filled points with default properties will be used.\\n\\n- If this property is `false`, no points would be automatically added to line or area marks.\\n\\n__Default value:__ `false`.\"\n },\n \"radius\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"For arc mark, the primary (outer) radius in pixels.\\n\\nFor text marks, polar coordinate radial offset, in pixels, of the text from the origin determined by the `x` and `y` properties.\\n\\n__Default value:__ `min(plot_width, plot_height)/2`\",\n \"minimum\": 0\n },\n \"radius2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The secondary (inner) radius in pixels of arc marks.\\n\\n__Default value:__ `0`\",\n \"minimum\": 0\n },\n \"shape\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SymbolShape\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Shape of the point marks. Supported values include:\\n- plotting shapes: `\\\"circle\\\"`, `\\\"square\\\"`, `\\\"cross\\\"`, `\\\"diamond\\\"`, `\\\"triangle-up\\\"`, `\\\"triangle-down\\\"`, `\\\"triangle-right\\\"`, or `\\\"triangle-left\\\"`.\\n- the line symbol `\\\"stroke\\\"`\\n- centered directional shapes `\\\"arrow\\\"`, `\\\"wedge\\\"`, or `\\\"triangle\\\"`\\n- a custom [SVG path string](https://developer.mozilla.org/en-US/docs/Web/SVG/Tutorial/Paths) (For correct sizing, custom shape paths should be defined within a square bounding box with coordinates ranging from -1 to 1 along both the x and y dimensions.)\\n\\n__Default value:__ `\\\"circle\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"size\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default size for marks.\\n- For `point`/`circle`/`square`, this represents the pixel area of the marks. Note that this value sets the area of the symbol; the side lengths will increase with the square root of this value.\\n- For `bar`, this represents the band size of the bar, in pixels.\\n- For `text`, this represents the font size, in pixels.\\n\\n__Default value:__\\n- `30` for point, circle, square marks; width/height's `step`\\n- `2` for bar marks with discrete dimensions;\\n- `5` for bar marks with continuous dimensions;\\n- `11` for text marks.\",\n \"minimum\": 0\n },\n \"smooth\": {\n \"anyOf\": [\n {\n \"description\": \"A boolean flag (default true) indicating if the image should be smoothed when resized. If false, individual pixels should be scaled directly rather than interpolated with smoothing. For SVG rendering, this option may not work in some browsers due to lack of standardization.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"startAngle\": {\n \"anyOf\": [\n {\n \"description\": \"The start angle in radians for arc marks. A value of `0` indicates up (north), increasing values proceed clockwise.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"stroke\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default stroke color. This property has higher precedence than `config.color`. Set to `null` to remove stroke.\\n\\n__Default value:__ (None)\"\n },\n \"strokeCap\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeCap\",\n \"description\": \"The stroke cap for line ending style. One of `\\\"butt\\\"`, `\\\"round\\\"`, or `\\\"square\\\"`.\\n\\n__Default value:__ `\\\"butt\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeDash\": {\n \"anyOf\": [\n {\n \"description\": \"An array of alternating stroke, space lengths for creating dashed or dotted lines.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeDashOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The offset (in pixels) into which to begin drawing with the stroke dash array.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeJoin\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeJoin\",\n \"description\": \"The stroke line join method. One of `\\\"miter\\\"`, `\\\"round\\\"` or `\\\"bevel\\\"`.\\n\\n__Default value:__ `\\\"miter\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeMiterLimit\": {\n \"anyOf\": [\n {\n \"description\": \"The miter limit at which to bevel a line join.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The offset in pixels at which to draw the group stroke and fill. If unspecified, the default behavior is to dynamically offset stroked groups such that 1 pixel stroke widths align with the pixel grid.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"The stroke opacity (value between [0,1]).\\n\\n__Default value:__ `1`\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeWidth\": {\n \"anyOf\": [\n {\n \"description\": \"The stroke width, in pixels.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"tension\": {\n \"anyOf\": [\n {\n \"description\": \"Depending on the interpolation type, sets the tension parameter (for line and area marks).\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"text\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\",\n \"description\": \"Placeholder text if the `text` channel is not specified\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"theta\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"- For arc marks, the arc length in radians if theta2 is not specified, otherwise the start arc angle. (A value of 0 indicates up or “north”, increasing values proceed clockwise.)\\n\\n- For text marks, polar coordinate angle in radians.\",\n \"maximum\": 360,\n \"minimum\": 0\n },\n \"theta2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The end angle of arc marks in radians. A value of 0 indicates up or “north”, increasing values proceed clockwise.\"\n },\n \"timeUnitBandPosition\": {\n \"description\": \"Default relative band position for a time unit. If set to `0`, the marks will be positioned at the beginning of the time unit band step. If set to `0.5`, the marks will be positioned in the middle of the time unit band step.\",\n \"type\": \"number\"\n },\n \"timeUnitBandSize\": {\n \"description\": \"Default relative band size for a time unit. If set to `1`, the bandwidth of the marks will be equal to the time unit band step. If set to `0.5`, bandwidth of the marks will be half of the time unit band step.\",\n \"type\": \"number\"\n },\n \"tooltip\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/TooltipContent\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"The tooltip text string to show upon mouse hover or an object defining which fields should the tooltip be derived from.\\n\\n- If `tooltip` is `true` or `{\\\"content\\\": \\\"encoding\\\"}`, then all fields from `encoding` will be used.\\n- If `tooltip` is `{\\\"content\\\": \\\"data\\\"}`, then all fields that appear in the highlighted data point will be used.\\n- If set to `null` or `false`, then no tooltip will be used.\\n\\nSee the [`tooltip`](https://vega.github.io/vega-lite/docs/tooltip.html) documentation for a detailed discussion about tooltip in Vega-Lite.\\n\\n__Default value:__ `null`\"\n },\n \"url\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/URI\",\n \"description\": \"The URL of the image file for image marks.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"width\": {\n \"anyOf\": [\n {\n \"description\": \"Width of the marks.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"x\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"X coordinates of the marks, or width of horizontal `\\\"bar\\\"` and `\\\"area\\\"` without specified `x2` or `width`.\\n\\nThe `value` of this channel can be a number or a string `\\\"width\\\"` for the width of the plot.\"\n },\n \"x2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"X2 coordinates for ranged `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"rect\\\"`, and `\\\"rule\\\"`.\\n\\nThe `value` of this channel can be a number or a string `\\\"width\\\"` for the width of the plot.\"\n },\n \"y\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Y coordinates of the marks, or height of vertical `\\\"bar\\\"` and `\\\"area\\\"` without specified `y2` or `height`.\\n\\nThe `value` of this channel can be a number or a string `\\\"height\\\"` for the height of the plot.\"\n },\n \"y2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Y2 coordinates for ranged `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"rect\\\"`, and `\\\"rule\\\"`.\\n\\nThe `value` of this channel can be a number or a string `\\\"height\\\"` for the height of the plot.\"\n }\n },\n \"type\": \"object\"\n },\n \"ArgmaxDef\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"argmax\": {\n \"$ref\": \"#/definitions/FieldName\"\n }\n },\n \"required\": [\n \"argmax\"\n ],\n \"type\": \"object\"\n },\n \"ArgminDef\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"argmin\": {\n \"$ref\": \"#/definitions/FieldName\"\n }\n },\n \"required\": [\n \"argmin\"\n ],\n \"type\": \"object\"\n },\n \"AutoSizeParams\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"contains\": {\n \"description\": \"Determines how size calculation should be performed, one of `\\\"content\\\"` or `\\\"padding\\\"`. The default setting (`\\\"content\\\"`) interprets the width and height settings as the data rectangle (plotting) dimensions, to which padding is then added. In contrast, the `\\\"padding\\\"` setting includes the padding within the view size calculations, such that the width and height settings indicate the **total** intended size of the view.\\n\\n__Default value__: `\\\"content\\\"`\",\n \"enum\": [\n \"content\",\n \"padding\"\n ],\n \"type\": \"string\"\n },\n \"resize\": {\n \"description\": \"A boolean flag indicating if autosize layout should be re-calculated on every view update.\\n\\n__Default value__: `false`\",\n \"type\": \"boolean\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/AutosizeType\",\n \"description\": \"The sizing format type. One of `\\\"pad\\\"`, `\\\"fit\\\"`, `\\\"fit-x\\\"`, `\\\"fit-y\\\"`, or `\\\"none\\\"`. See the [autosize type](https://vega.github.io/vega-lite/docs/size.html#autosize) documentation for descriptions of each.\\n\\n__Default value__: `\\\"pad\\\"`\"\n }\n },\n \"type\": \"object\"\n },\n \"AutosizeType\": {\n \"enum\": [\n \"pad\",\n \"none\",\n \"fit\",\n \"fit-x\",\n \"fit-y\"\n ],\n \"type\": \"string\"\n },\n \"Axis\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aria\": {\n \"anyOf\": [\n {\n \"description\": \"A boolean flag indicating if [ARIA attributes](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) should be included (SVG output only). If `false`, the \\\"aria-hidden\\\" attribute will be set on the output SVG group, removing the axis from the ARIA accessibility tree.\\n\\n__Default value:__ `true`\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"bandPosition\": {\n \"anyOf\": [\n {\n \"description\": \"An interpolation fraction indicating where, for `band` scales, axis ticks should be positioned. A value of `0` places ticks at the left edge of their bands. A value of `0.5` places ticks in the middle of their bands.\\n\\n __Default value:__ `0.5`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"description\": {\n \"anyOf\": [\n {\n \"description\": \"A text description of this axis for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If the `aria` property is true, for SVG output the [\\\"aria-label\\\" attribute](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/ARIA_Techniques/Using_the_aria-label_attribute) will be set to this description. If the description is unspecified it will be automatically generated.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"domain\": {\n \"description\": \"A boolean flag indicating if the domain (the axis baseline) should be included as part of the axis.\\n\\n__Default value:__ `true`\",\n \"type\": \"boolean\"\n },\n \"domainCap\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeCap\",\n \"description\": \"The stroke cap for the domain line's ending style. One of `\\\"butt\\\"`, `\\\"round\\\"` or `\\\"square\\\"`.\\n\\n__Default value:__ `\\\"butt\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"domainColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"Color of axis domain line.\\n\\n__Default value:__ `\\\"gray\\\"`.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"domainDash\": {\n \"anyOf\": [\n {\n \"description\": \"An array of alternating [stroke, space] lengths for dashed domain lines.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"domainDashOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The pixel offset at which to start drawing with the domain dash array.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"domainOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"Opacity of the axis domain line.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"domainWidth\": {\n \"anyOf\": [\n {\n \"description\": \"Stroke width of axis domain line\\n\\n__Default value:__ `1`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"format\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Dict\"\n }\n ],\n \"description\": \"When used with the default `\\\"number\\\"` and `\\\"time\\\"` format type, the text formatting pattern for labels of guides (axes, legends, headers) and text marks.\\n\\n- If the format type is `\\\"number\\\"` (e.g., for quantitative fields), this is D3's [number format pattern](https://github.com/d3/d3-format#locale_format).\\n- If the format type is `\\\"time\\\"` (e.g., for temporal fields), this is D3's [time format pattern](https://github.com/d3/d3-time-format#locale_format).\\n\\nSee the [format documentation](https://vega.github.io/vega-lite/docs/format.html) for more examples.\\n\\nWhen used with a [custom `formatType`](https://vega.github.io/vega-lite/docs/config.html#custom-format-type), this value will be passed as `format` alongside `datum.value` to the registered function.\\n\\n__Default value:__ Derived from [numberFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for number format and from [timeFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for time format.\"\n },\n \"formatType\": {\n \"description\": \"The format type for labels. One of `\\\"number\\\"`, `\\\"time\\\"`, or a [registered custom format type](https://vega.github.io/vega-lite/docs/config.html#custom-format-type).\\n\\n__Default value:__\\n- `\\\"time\\\"` for temporal fields and ordinal and nominal fields with `timeUnit`.\\n- `\\\"number\\\"` for quantitative fields as well as ordinal and nominal fields without `timeUnit`.\",\n \"type\": \"string\"\n },\n \"grid\": {\n \"description\": \"A boolean flag indicating if grid lines should be included as part of the axis\\n\\n__Default value:__ `true` for [continuous scales](https://vega.github.io/vega-lite/docs/scale.html#continuous) that are not binned; otherwise, `false`.\",\n \"type\": \"boolean\"\n },\n \"gridCap\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeCap\",\n \"description\": \"The stroke cap for grid lines' ending style. One of `\\\"butt\\\"`, `\\\"round\\\"` or `\\\"square\\\"`.\\n\\n__Default value:__ `\\\"butt\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"gridColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"Color of gridlines.\\n\\n__Default value:__ `\\\"lightGray\\\"`.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisColor\"\n }\n ]\n },\n \"gridDash\": {\n \"anyOf\": [\n {\n \"description\": \"An array of alternating [stroke, space] lengths for dashed grid lines.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumberArray\"\n }\n ]\n },\n \"gridDashOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The pixel offset at which to start drawing with the grid dash array.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumber\"\n }\n ]\n },\n \"gridOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"The stroke opacity of grid (value between [0,1])\\n\\n__Default value:__ `1`\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumber\"\n }\n ]\n },\n \"gridWidth\": {\n \"anyOf\": [\n {\n \"description\": \"The grid width, in pixels.\\n\\n__Default value:__ `1`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumber\"\n }\n ]\n },\n \"labelAlign\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Align\",\n \"description\": \"Horizontal text alignment of axis tick labels, overriding the default setting for the current axis orientation.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisLabelAlign\"\n }\n ]\n },\n \"labelAngle\": {\n \"anyOf\": [\n {\n \"description\": \"The rotation angle of the axis labels.\\n\\n__Default value:__ `-90` for nominal and ordinal fields; `0` otherwise.\",\n \"maximum\": 360,\n \"minimum\": -360,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelBaseline\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextBaseline\",\n \"description\": \"Vertical text baseline of axis tick labels, overriding the default setting for the current axis orientation. One of `\\\"alphabetic\\\"` (default), `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`, `\\\"line-top\\\"`, or `\\\"line-bottom\\\"`. The `\\\"line-top\\\"` and `\\\"line-bottom\\\"` values operate similarly to `\\\"top\\\"` and `\\\"bottom\\\"`, but are calculated relative to the *lineHeight* rather than *fontSize* alone.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisLabelBaseline\"\n }\n ]\n },\n \"labelBound\": {\n \"anyOf\": [\n {\n \"description\": \"Indicates if labels should be hidden if they exceed the axis range. If `false` (the default) no bounds overlap analysis is performed. If `true`, labels will be hidden if they exceed the axis range by more than 1 pixel. If this property is a number, it specifies the pixel tolerance: the maximum amount by which a label bounding box may exceed the axis range.\\n\\n__Default value:__ `false`.\",\n \"type\": [\n \"number\",\n \"boolean\"\n ]\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"The color of the tick label, can be in hex color code or regular color name.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisColor\"\n }\n ]\n },\n \"labelExpr\": {\n \"description\": \"[Vega expression](https://vega.github.io/vega/docs/expressions/) for customizing labels.\\n\\n__Note:__ The label text and value can be assessed via the `label` and `value` properties of the axis's backing `datum` object.\",\n \"type\": \"string\"\n },\n \"labelFlush\": {\n \"description\": \"Indicates if the first and last axis labels should be aligned flush with the scale range. Flush alignment for a horizontal axis will left-align the first label and right-align the last label. For vertical axes, bottom and top text baselines are applied instead. If this property is a number, it also indicates the number of pixels by which to offset the first and last labels; for example, a value of 2 will flush-align the first and last labels and also push them 2 pixels outward from the center of the axis. The additional adjustment can sometimes help the labels better visually group with corresponding axis ticks.\\n\\n__Default value:__ `true` for axis of a continuous x-scale. Otherwise, `false`.\",\n \"type\": [\n \"boolean\",\n \"number\"\n ]\n },\n \"labelFlushOffset\": {\n \"anyOf\": [\n {\n \"description\": \"Indicates the number of pixels by which to offset flush-adjusted labels. For example, a value of `2` will push flush-adjusted labels 2 pixels outward from the center of the axis. Offsets can help the labels better visually group with corresponding axis ticks.\\n\\n__Default value:__ `0`.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelFont\": {\n \"anyOf\": [\n {\n \"description\": \"The font of the tick label.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisString\"\n }\n ]\n },\n \"labelFontSize\": {\n \"anyOf\": [\n {\n \"description\": \"The font size of the label, in pixels.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumber\"\n }\n ]\n },\n \"labelFontStyle\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\",\n \"description\": \"Font style of the title.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisLabelFontStyle\"\n }\n ]\n },\n \"labelFontWeight\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\",\n \"description\": \"Font weight of axis tick labels.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisLabelFontWeight\"\n }\n ]\n },\n \"labelLimit\": {\n \"anyOf\": [\n {\n \"description\": \"Maximum allowed pixel width of axis tick labels.\\n\\n__Default value:__ `180`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelLineHeight\": {\n \"anyOf\": [\n {\n \"description\": \"Line height in pixels for multi-line label text or label text with `\\\"line-top\\\"` or `\\\"line-bottom\\\"` baseline.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelOffset\": {\n \"anyOf\": [\n {\n \"description\": \"Position offset in pixels to apply to labels, in addition to tickOffset.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumber\"\n }\n ]\n },\n \"labelOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"The opacity of the labels.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumber\"\n }\n ]\n },\n \"labelOverlap\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LabelOverlap\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The strategy to use for resolving overlap of axis labels. If `false` (the default), no overlap reduction is attempted. If set to `true` or `\\\"parity\\\"`, a strategy of removing every other label is used (this works well for standard linear axes). If set to `\\\"greedy\\\"`, a linear scan of the labels is performed, removing any labels that overlaps with the last visible label (this often works better for log-scaled axes).\\n\\n__Default value:__ `true` for non-nominal fields with non-log scales; `\\\"greedy\\\"` for log scales; otherwise `false`.\"\n },\n \"labelPadding\": {\n \"anyOf\": [\n {\n \"description\": \"The padding in pixels between labels and ticks.\\n\\n__Default value:__ `2`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumber\"\n }\n ]\n },\n \"labelSeparation\": {\n \"anyOf\": [\n {\n \"description\": \"The minimum separation that must be between label bounding boxes for them to be considered non-overlapping (default `0`). This property is ignored if *labelOverlap* resolution is not enabled.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labels\": {\n \"description\": \"A boolean flag indicating if labels should be included as part of the axis.\\n\\n__Default value:__ `true`.\",\n \"type\": \"boolean\"\n },\n \"maxExtent\": {\n \"anyOf\": [\n {\n \"description\": \"The maximum extent in pixels that axis ticks and labels should use. This determines a maximum offset value for axis titles.\\n\\n__Default value:__ `undefined`.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"minExtent\": {\n \"anyOf\": [\n {\n \"description\": \"The minimum extent in pixels that axis ticks and labels should use. This determines a minimum offset value for axis titles.\\n\\n__Default value:__ `30` for y-axis; `undefined` for x-axis.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"offset\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The offset, in pixels, by which to displace the axis from the edge of the enclosing group or data rectangle.\\n\\n__Default value:__ derived from the [axis config](https://vega.github.io/vega-lite/docs/config.html#facet-scale-config)'s `offset` (`0` by default)\"\n },\n \"orient\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/AxisOrient\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The orientation of the axis. One of `\\\"top\\\"`, `\\\"bottom\\\"`, `\\\"left\\\"` or `\\\"right\\\"`. The orientation can be used to further specialize the axis type (e.g., a y-axis oriented towards the right edge of the chart).\\n\\n__Default value:__ `\\\"bottom\\\"` for x-axes and `\\\"left\\\"` for y-axes.\"\n },\n \"position\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The anchor position of the axis in pixels. For x-axes with top or bottom orientation, this sets the axis group x coordinate. For y-axes with left or right orientation, this sets the axis group y coordinate.\\n\\n__Default value__: `0`\"\n },\n \"style\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A string or array of strings indicating the name of custom styles to apply to the axis. A style is a named collection of axis property defined within the [style configuration](https://vega.github.io/vega-lite/docs/mark.html#style-config). If style is an array, later styles will override earlier styles.\\n\\n__Default value:__ (none) __Note:__ Any specified style will augment the default style. For example, an x-axis mark with `\\\"style\\\": \\\"foo\\\"` will use `config.axisX` and `config.style.foo` (the specified style `\\\"foo\\\"` has higher precedence).\"\n },\n \"tickBand\": {\n \"anyOf\": [\n {\n \"description\": \"For band scales, indicates if ticks and grid lines should be placed at the `\\\"center\\\"` of a band (default) or at the band `\\\"extent\\\"`s to indicate intervals\",\n \"enum\": [\n \"center\",\n \"extent\"\n ],\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"tickCap\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeCap\",\n \"description\": \"The stroke cap for the tick lines' ending style. One of `\\\"butt\\\"`, `\\\"round\\\"` or `\\\"square\\\"`.\\n\\n__Default value:__ `\\\"butt\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"tickColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"The color of the axis's tick.\\n\\n__Default value:__ `\\\"gray\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisColor\"\n }\n ]\n },\n \"tickCount\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/TimeInterval\"\n },\n {\n \"$ref\": \"#/definitions/TimeIntervalStep\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A desired number of ticks, for axes visualizing quantitative scales. The resulting number may be different so that values are \\\"nice\\\" (multiples of 2, 5, 10) and lie within the underlying scale's range.\\n\\nFor scales of type `\\\"time\\\"` or `\\\"utc\\\"`, the tick count can instead be a time interval specifier. Legal string values are `\\\"millisecond\\\"`, `\\\"second\\\"`, `\\\"minute\\\"`, `\\\"hour\\\"`, `\\\"day\\\"`, `\\\"week\\\"`, `\\\"month\\\"`, and `\\\"year\\\"`. Alternatively, an object-valued interval specifier of the form `{\\\"interval\\\": \\\"month\\\", \\\"step\\\": 3}` includes a desired number of interval steps. Here, ticks are generated for each quarter (Jan, Apr, Jul, Oct) boundary.\\n\\n__Default value__: Determine using a formula `ceil(width/40)` for x and `ceil(height/40)` for y.\",\n \"minimum\": 0\n },\n \"tickDash\": {\n \"anyOf\": [\n {\n \"description\": \"An array of alternating [stroke, space] lengths for dashed tick mark lines.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumberArray\"\n }\n ]\n },\n \"tickDashOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The pixel offset at which to start drawing with the tick mark dash array.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumber\"\n }\n ]\n },\n \"tickExtra\": {\n \"description\": \"Boolean flag indicating if an extra axis tick should be added for the initial position of the axis. This flag is useful for styling axes for `band` scales such that ticks are placed on band boundaries rather in the middle of a band. Use in conjunction with `\\\"bandPosition\\\": 1` and an axis `\\\"padding\\\"` value of `0`.\",\n \"type\": \"boolean\"\n },\n \"tickMinStep\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The minimum desired step between axis ticks, in terms of scale domain values. For example, a value of `1` indicates that ticks should not be less than 1 unit apart. If `tickMinStep` is specified, the `tickCount` value will be adjusted, if necessary, to enforce the minimum step value.\"\n },\n \"tickOffset\": {\n \"anyOf\": [\n {\n \"description\": \"Position offset in pixels to apply to ticks, labels, and gridlines.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"tickOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"Opacity of the ticks.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumber\"\n }\n ]\n },\n \"tickRound\": {\n \"description\": \"Boolean flag indicating if pixel position values should be rounded to the nearest integer.\\n\\n__Default value:__ `true`\",\n \"type\": \"boolean\"\n },\n \"tickSize\": {\n \"anyOf\": [\n {\n \"description\": \"The size in pixels of axis ticks.\\n\\n__Default value:__ `5`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumber\"\n }\n ]\n },\n \"tickWidth\": {\n \"anyOf\": [\n {\n \"description\": \"The width, in pixels, of ticks.\\n\\n__Default value:__ `1`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumber\"\n }\n ]\n },\n \"ticks\": {\n \"description\": \"Boolean value that determines whether the axis should include ticks.\\n\\n__Default value:__ `true`\",\n \"type\": \"boolean\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"titleAlign\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Align\",\n \"description\": \"Horizontal text alignment of axis titles.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleAnchor\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TitleAnchor\",\n \"description\": \"Text anchor position for placing axis titles.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleAngle\": {\n \"anyOf\": [\n {\n \"description\": \"Angle in degrees of axis titles.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleBaseline\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextBaseline\",\n \"description\": \"Vertical text baseline for axis titles. One of `\\\"alphabetic\\\"` (default), `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`, `\\\"line-top\\\"`, or `\\\"line-bottom\\\"`. The `\\\"line-top\\\"` and `\\\"line-bottom\\\"` values operate similarly to `\\\"top\\\"` and `\\\"bottom\\\"`, but are calculated relative to the *lineHeight* rather than *fontSize* alone.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"Color of the title, can be in hex color code or regular color name.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleFont\": {\n \"anyOf\": [\n {\n \"description\": \"Font of the title. (e.g., `\\\"Helvetica Neue\\\"`).\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleFontSize\": {\n \"anyOf\": [\n {\n \"description\": \"Font size of the title.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleFontStyle\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\",\n \"description\": \"Font style of the title.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleFontWeight\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\",\n \"description\": \"Font weight of the title. This can be either a string (e.g `\\\"bold\\\"`, `\\\"normal\\\"`) or a number (`100`, `200`, `300`, ..., `900` where `\\\"normal\\\"` = `400` and `\\\"bold\\\"` = `700`).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleLimit\": {\n \"anyOf\": [\n {\n \"description\": \"Maximum allowed pixel width of axis titles.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleLineHeight\": {\n \"anyOf\": [\n {\n \"description\": \"Line height in pixels for multi-line title text or title text with `\\\"line-top\\\"` or `\\\"line-bottom\\\"` baseline.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"Opacity of the axis title.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titlePadding\": {\n \"anyOf\": [\n {\n \"description\": \"The padding, in pixels, between title and axis.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleX\": {\n \"anyOf\": [\n {\n \"description\": \"X-coordinate of the axis title relative to the axis group.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleY\": {\n \"anyOf\": [\n {\n \"description\": \"Y-coordinate of the axis title relative to the axis group.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"translate\": {\n \"anyOf\": [\n {\n \"description\": \"Coordinate space translation offset for axis layout. By default, axes are translated by a 0.5 pixel offset for both the x and y coordinates in order to align stroked lines with the pixel grid. However, for vector graphics output these pixel-specific adjustments may be undesirable, in which case translate can be changed (for example, to zero).\\n\\n__Default value:__ `0.5`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"values\": {\n \"anyOf\": [\n {\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n {\n \"items\": {\n \"type\": \"boolean\"\n },\n \"type\": \"array\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/DateTime\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Explicitly set the visible axis tick values.\"\n },\n \"zindex\": {\n \"description\": \"A non-negative integer indicating the z-index of the axis. If zindex is 0, axes should be drawn behind all chart elements. To put them in front, set `zindex` to `1` or more.\\n\\n__Default value:__ `0` (behind the marks).\",\n \"minimum\": 0,\n \"type\": \"number\"\n }\n },\n \"type\": \"object\"\n },\n \"AxisConfig\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aria\": {\n \"anyOf\": [\n {\n \"description\": \"A boolean flag indicating if [ARIA attributes](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) should be included (SVG output only). If `false`, the \\\"aria-hidden\\\" attribute will be set on the output SVG group, removing the axis from the ARIA accessibility tree.\\n\\n__Default value:__ `true`\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"bandPosition\": {\n \"anyOf\": [\n {\n \"description\": \"An interpolation fraction indicating where, for `band` scales, axis ticks should be positioned. A value of `0` places ticks at the left edge of their bands. A value of `0.5` places ticks in the middle of their bands.\\n\\n __Default value:__ `0.5`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"description\": {\n \"anyOf\": [\n {\n \"description\": \"A text description of this axis for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If the `aria` property is true, for SVG output the [\\\"aria-label\\\" attribute](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/ARIA_Techniques/Using_the_aria-label_attribute) will be set to this description. If the description is unspecified it will be automatically generated.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"disable\": {\n \"description\": \"Disable axis by default.\",\n \"type\": \"boolean\"\n },\n \"domain\": {\n \"description\": \"A boolean flag indicating if the domain (the axis baseline) should be included as part of the axis.\\n\\n__Default value:__ `true`\",\n \"type\": \"boolean\"\n },\n \"domainCap\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeCap\",\n \"description\": \"The stroke cap for the domain line's ending style. One of `\\\"butt\\\"`, `\\\"round\\\"` or `\\\"square\\\"`.\\n\\n__Default value:__ `\\\"butt\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"domainColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"Color of axis domain line.\\n\\n__Default value:__ `\\\"gray\\\"`.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"domainDash\": {\n \"anyOf\": [\n {\n \"description\": \"An array of alternating [stroke, space] lengths for dashed domain lines.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"domainDashOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The pixel offset at which to start drawing with the domain dash array.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"domainOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"Opacity of the axis domain line.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"domainWidth\": {\n \"anyOf\": [\n {\n \"description\": \"Stroke width of axis domain line\\n\\n__Default value:__ `1`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"format\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Dict\"\n }\n ],\n \"description\": \"When used with the default `\\\"number\\\"` and `\\\"time\\\"` format type, the text formatting pattern for labels of guides (axes, legends, headers) and text marks.\\n\\n- If the format type is `\\\"number\\\"` (e.g., for quantitative fields), this is D3's [number format pattern](https://github.com/d3/d3-format#locale_format).\\n- If the format type is `\\\"time\\\"` (e.g., for temporal fields), this is D3's [time format pattern](https://github.com/d3/d3-time-format#locale_format).\\n\\nSee the [format documentation](https://vega.github.io/vega-lite/docs/format.html) for more examples.\\n\\nWhen used with a [custom `formatType`](https://vega.github.io/vega-lite/docs/config.html#custom-format-type), this value will be passed as `format` alongside `datum.value` to the registered function.\\n\\n__Default value:__ Derived from [numberFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for number format and from [timeFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for time format.\"\n },\n \"formatType\": {\n \"description\": \"The format type for labels. One of `\\\"number\\\"`, `\\\"time\\\"`, or a [registered custom format type](https://vega.github.io/vega-lite/docs/config.html#custom-format-type).\\n\\n__Default value:__\\n- `\\\"time\\\"` for temporal fields and ordinal and nominal fields with `timeUnit`.\\n- `\\\"number\\\"` for quantitative fields as well as ordinal and nominal fields without `timeUnit`.\",\n \"type\": \"string\"\n },\n \"grid\": {\n \"description\": \"A boolean flag indicating if grid lines should be included as part of the axis\\n\\n__Default value:__ `true` for [continuous scales](https://vega.github.io/vega-lite/docs/scale.html#continuous) that are not binned; otherwise, `false`.\",\n \"type\": \"boolean\"\n },\n \"gridCap\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeCap\",\n \"description\": \"The stroke cap for grid lines' ending style. One of `\\\"butt\\\"`, `\\\"round\\\"` or `\\\"square\\\"`.\\n\\n__Default value:__ `\\\"butt\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"gridColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"Color of gridlines.\\n\\n__Default value:__ `\\\"lightGray\\\"`.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisColor\"\n }\n ]\n },\n \"gridDash\": {\n \"anyOf\": [\n {\n \"description\": \"An array of alternating [stroke, space] lengths for dashed grid lines.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumberArray\"\n }\n ]\n },\n \"gridDashOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The pixel offset at which to start drawing with the grid dash array.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumber\"\n }\n ]\n },\n \"gridOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"The stroke opacity of grid (value between [0,1])\\n\\n__Default value:__ `1`\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumber\"\n }\n ]\n },\n \"gridWidth\": {\n \"anyOf\": [\n {\n \"description\": \"The grid width, in pixels.\\n\\n__Default value:__ `1`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumber\"\n }\n ]\n },\n \"labelAlign\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Align\",\n \"description\": \"Horizontal text alignment of axis tick labels, overriding the default setting for the current axis orientation.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisLabelAlign\"\n }\n ]\n },\n \"labelAngle\": {\n \"anyOf\": [\n {\n \"description\": \"The rotation angle of the axis labels.\\n\\n__Default value:__ `-90` for nominal and ordinal fields; `0` otherwise.\",\n \"maximum\": 360,\n \"minimum\": -360,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelBaseline\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextBaseline\",\n \"description\": \"Vertical text baseline of axis tick labels, overriding the default setting for the current axis orientation. One of `\\\"alphabetic\\\"` (default), `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`, `\\\"line-top\\\"`, or `\\\"line-bottom\\\"`. The `\\\"line-top\\\"` and `\\\"line-bottom\\\"` values operate similarly to `\\\"top\\\"` and `\\\"bottom\\\"`, but are calculated relative to the *lineHeight* rather than *fontSize* alone.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisLabelBaseline\"\n }\n ]\n },\n \"labelBound\": {\n \"anyOf\": [\n {\n \"description\": \"Indicates if labels should be hidden if they exceed the axis range. If `false` (the default) no bounds overlap analysis is performed. If `true`, labels will be hidden if they exceed the axis range by more than 1 pixel. If this property is a number, it specifies the pixel tolerance: the maximum amount by which a label bounding box may exceed the axis range.\\n\\n__Default value:__ `false`.\",\n \"type\": [\n \"number\",\n \"boolean\"\n ]\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"The color of the tick label, can be in hex color code or regular color name.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisColor\"\n }\n ]\n },\n \"labelExpr\": {\n \"description\": \"[Vega expression](https://vega.github.io/vega/docs/expressions/) for customizing labels.\\n\\n__Note:__ The label text and value can be assessed via the `label` and `value` properties of the axis's backing `datum` object.\",\n \"type\": \"string\"\n },\n \"labelFlush\": {\n \"description\": \"Indicates if the first and last axis labels should be aligned flush with the scale range. Flush alignment for a horizontal axis will left-align the first label and right-align the last label. For vertical axes, bottom and top text baselines are applied instead. If this property is a number, it also indicates the number of pixels by which to offset the first and last labels; for example, a value of 2 will flush-align the first and last labels and also push them 2 pixels outward from the center of the axis. The additional adjustment can sometimes help the labels better visually group with corresponding axis ticks.\\n\\n__Default value:__ `true` for axis of a continuous x-scale. Otherwise, `false`.\",\n \"type\": [\n \"boolean\",\n \"number\"\n ]\n },\n \"labelFlushOffset\": {\n \"anyOf\": [\n {\n \"description\": \"Indicates the number of pixels by which to offset flush-adjusted labels. For example, a value of `2` will push flush-adjusted labels 2 pixels outward from the center of the axis. Offsets can help the labels better visually group with corresponding axis ticks.\\n\\n__Default value:__ `0`.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelFont\": {\n \"anyOf\": [\n {\n \"description\": \"The font of the tick label.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisString\"\n }\n ]\n },\n \"labelFontSize\": {\n \"anyOf\": [\n {\n \"description\": \"The font size of the label, in pixels.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumber\"\n }\n ]\n },\n \"labelFontStyle\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\",\n \"description\": \"Font style of the title.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisLabelFontStyle\"\n }\n ]\n },\n \"labelFontWeight\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\",\n \"description\": \"Font weight of axis tick labels.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisLabelFontWeight\"\n }\n ]\n },\n \"labelLimit\": {\n \"anyOf\": [\n {\n \"description\": \"Maximum allowed pixel width of axis tick labels.\\n\\n__Default value:__ `180`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelLineHeight\": {\n \"anyOf\": [\n {\n \"description\": \"Line height in pixels for multi-line label text or label text with `\\\"line-top\\\"` or `\\\"line-bottom\\\"` baseline.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelOffset\": {\n \"anyOf\": [\n {\n \"description\": \"Position offset in pixels to apply to labels, in addition to tickOffset.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumber\"\n }\n ]\n },\n \"labelOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"The opacity of the labels.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumber\"\n }\n ]\n },\n \"labelOverlap\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LabelOverlap\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The strategy to use for resolving overlap of axis labels. If `false` (the default), no overlap reduction is attempted. If set to `true` or `\\\"parity\\\"`, a strategy of removing every other label is used (this works well for standard linear axes). If set to `\\\"greedy\\\"`, a linear scan of the labels is performed, removing any labels that overlaps with the last visible label (this often works better for log-scaled axes).\\n\\n__Default value:__ `true` for non-nominal fields with non-log scales; `\\\"greedy\\\"` for log scales; otherwise `false`.\"\n },\n \"labelPadding\": {\n \"anyOf\": [\n {\n \"description\": \"The padding in pixels between labels and ticks.\\n\\n__Default value:__ `2`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumber\"\n }\n ]\n },\n \"labelSeparation\": {\n \"anyOf\": [\n {\n \"description\": \"The minimum separation that must be between label bounding boxes for them to be considered non-overlapping (default `0`). This property is ignored if *labelOverlap* resolution is not enabled.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labels\": {\n \"description\": \"A boolean flag indicating if labels should be included as part of the axis.\\n\\n__Default value:__ `true`.\",\n \"type\": \"boolean\"\n },\n \"maxExtent\": {\n \"anyOf\": [\n {\n \"description\": \"The maximum extent in pixels that axis ticks and labels should use. This determines a maximum offset value for axis titles.\\n\\n__Default value:__ `undefined`.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"minExtent\": {\n \"anyOf\": [\n {\n \"description\": \"The minimum extent in pixels that axis ticks and labels should use. This determines a minimum offset value for axis titles.\\n\\n__Default value:__ `30` for y-axis; `undefined` for x-axis.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"offset\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The offset, in pixels, by which to displace the axis from the edge of the enclosing group or data rectangle.\\n\\n__Default value:__ derived from the [axis config](https://vega.github.io/vega-lite/docs/config.html#facet-scale-config)'s `offset` (`0` by default)\"\n },\n \"orient\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/AxisOrient\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The orientation of the axis. One of `\\\"top\\\"`, `\\\"bottom\\\"`, `\\\"left\\\"` or `\\\"right\\\"`. The orientation can be used to further specialize the axis type (e.g., a y-axis oriented towards the right edge of the chart).\\n\\n__Default value:__ `\\\"bottom\\\"` for x-axes and `\\\"left\\\"` for y-axes.\"\n },\n \"position\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The anchor position of the axis in pixels. For x-axes with top or bottom orientation, this sets the axis group x coordinate. For y-axes with left or right orientation, this sets the axis group y coordinate.\\n\\n__Default value__: `0`\"\n },\n \"style\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A string or array of strings indicating the name of custom styles to apply to the axis. A style is a named collection of axis property defined within the [style configuration](https://vega.github.io/vega-lite/docs/mark.html#style-config). If style is an array, later styles will override earlier styles.\\n\\n__Default value:__ (none) __Note:__ Any specified style will augment the default style. For example, an x-axis mark with `\\\"style\\\": \\\"foo\\\"` will use `config.axisX` and `config.style.foo` (the specified style `\\\"foo\\\"` has higher precedence).\"\n },\n \"tickBand\": {\n \"anyOf\": [\n {\n \"description\": \"For band scales, indicates if ticks and grid lines should be placed at the `\\\"center\\\"` of a band (default) or at the band `\\\"extent\\\"`s to indicate intervals\",\n \"enum\": [\n \"center\",\n \"extent\"\n ],\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"tickCap\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeCap\",\n \"description\": \"The stroke cap for the tick lines' ending style. One of `\\\"butt\\\"`, `\\\"round\\\"` or `\\\"square\\\"`.\\n\\n__Default value:__ `\\\"butt\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"tickColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"The color of the axis's tick.\\n\\n__Default value:__ `\\\"gray\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisColor\"\n }\n ]\n },\n \"tickCount\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/TimeInterval\"\n },\n {\n \"$ref\": \"#/definitions/TimeIntervalStep\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A desired number of ticks, for axes visualizing quantitative scales. The resulting number may be different so that values are \\\"nice\\\" (multiples of 2, 5, 10) and lie within the underlying scale's range.\\n\\nFor scales of type `\\\"time\\\"` or `\\\"utc\\\"`, the tick count can instead be a time interval specifier. Legal string values are `\\\"millisecond\\\"`, `\\\"second\\\"`, `\\\"minute\\\"`, `\\\"hour\\\"`, `\\\"day\\\"`, `\\\"week\\\"`, `\\\"month\\\"`, and `\\\"year\\\"`. Alternatively, an object-valued interval specifier of the form `{\\\"interval\\\": \\\"month\\\", \\\"step\\\": 3}` includes a desired number of interval steps. Here, ticks are generated for each quarter (Jan, Apr, Jul, Oct) boundary.\\n\\n__Default value__: Determine using a formula `ceil(width/40)` for x and `ceil(height/40)` for y.\",\n \"minimum\": 0\n },\n \"tickDash\": {\n \"anyOf\": [\n {\n \"description\": \"An array of alternating [stroke, space] lengths for dashed tick mark lines.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumberArray\"\n }\n ]\n },\n \"tickDashOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The pixel offset at which to start drawing with the tick mark dash array.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumber\"\n }\n ]\n },\n \"tickExtra\": {\n \"description\": \"Boolean flag indicating if an extra axis tick should be added for the initial position of the axis. This flag is useful for styling axes for `band` scales such that ticks are placed on band boundaries rather in the middle of a band. Use in conjunction with `\\\"bandPosition\\\": 1` and an axis `\\\"padding\\\"` value of `0`.\",\n \"type\": \"boolean\"\n },\n \"tickMinStep\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The minimum desired step between axis ticks, in terms of scale domain values. For example, a value of `1` indicates that ticks should not be less than 1 unit apart. If `tickMinStep` is specified, the `tickCount` value will be adjusted, if necessary, to enforce the minimum step value.\"\n },\n \"tickOffset\": {\n \"anyOf\": [\n {\n \"description\": \"Position offset in pixels to apply to ticks, labels, and gridlines.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"tickOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"Opacity of the ticks.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumber\"\n }\n ]\n },\n \"tickRound\": {\n \"description\": \"Boolean flag indicating if pixel position values should be rounded to the nearest integer.\\n\\n__Default value:__ `true`\",\n \"type\": \"boolean\"\n },\n \"tickSize\": {\n \"anyOf\": [\n {\n \"description\": \"The size in pixels of axis ticks.\\n\\n__Default value:__ `5`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumber\"\n }\n ]\n },\n \"tickWidth\": {\n \"anyOf\": [\n {\n \"description\": \"The width, in pixels, of ticks.\\n\\n__Default value:__ `1`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalAxisNumber\"\n }\n ]\n },\n \"ticks\": {\n \"description\": \"Boolean value that determines whether the axis should include ticks.\\n\\n__Default value:__ `true`\",\n \"type\": \"boolean\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"titleAlign\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Align\",\n \"description\": \"Horizontal text alignment of axis titles.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleAnchor\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TitleAnchor\",\n \"description\": \"Text anchor position for placing axis titles.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleAngle\": {\n \"anyOf\": [\n {\n \"description\": \"Angle in degrees of axis titles.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleBaseline\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextBaseline\",\n \"description\": \"Vertical text baseline for axis titles. One of `\\\"alphabetic\\\"` (default), `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`, `\\\"line-top\\\"`, or `\\\"line-bottom\\\"`. The `\\\"line-top\\\"` and `\\\"line-bottom\\\"` values operate similarly to `\\\"top\\\"` and `\\\"bottom\\\"`, but are calculated relative to the *lineHeight* rather than *fontSize* alone.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"Color of the title, can be in hex color code or regular color name.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleFont\": {\n \"anyOf\": [\n {\n \"description\": \"Font of the title. (e.g., `\\\"Helvetica Neue\\\"`).\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleFontSize\": {\n \"anyOf\": [\n {\n \"description\": \"Font size of the title.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleFontStyle\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\",\n \"description\": \"Font style of the title.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleFontWeight\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\",\n \"description\": \"Font weight of the title. This can be either a string (e.g `\\\"bold\\\"`, `\\\"normal\\\"`) or a number (`100`, `200`, `300`, ..., `900` where `\\\"normal\\\"` = `400` and `\\\"bold\\\"` = `700`).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleLimit\": {\n \"anyOf\": [\n {\n \"description\": \"Maximum allowed pixel width of axis titles.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleLineHeight\": {\n \"anyOf\": [\n {\n \"description\": \"Line height in pixels for multi-line title text or title text with `\\\"line-top\\\"` or `\\\"line-bottom\\\"` baseline.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"Opacity of the axis title.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titlePadding\": {\n \"anyOf\": [\n {\n \"description\": \"The padding, in pixels, between title and axis.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleX\": {\n \"anyOf\": [\n {\n \"description\": \"X-coordinate of the axis title relative to the axis group.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleY\": {\n \"anyOf\": [\n {\n \"description\": \"Y-coordinate of the axis title relative to the axis group.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"translate\": {\n \"anyOf\": [\n {\n \"description\": \"Coordinate space translation offset for axis layout. By default, axes are translated by a 0.5 pixel offset for both the x and y coordinates in order to align stroked lines with the pixel grid. However, for vector graphics output these pixel-specific adjustments may be undesirable, in which case translate can be changed (for example, to zero).\\n\\n__Default value:__ `0.5`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"values\": {\n \"anyOf\": [\n {\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n {\n \"items\": {\n \"type\": \"boolean\"\n },\n \"type\": \"array\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/DateTime\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Explicitly set the visible axis tick values.\"\n },\n \"zindex\": {\n \"description\": \"A non-negative integer indicating the z-index of the axis. If zindex is 0, axes should be drawn behind all chart elements. To put them in front, set `zindex` to `1` or more.\\n\\n__Default value:__ `0` (behind the marks).\",\n \"minimum\": 0,\n \"type\": \"number\"\n }\n },\n \"type\": \"object\"\n },\n \"AxisOrient\": {\n \"enum\": [\n \"top\",\n \"bottom\",\n \"left\",\n \"right\"\n ],\n \"type\": \"string\"\n },\n \"AxisResolveMap\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"x\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"y\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n }\n },\n \"type\": \"object\"\n },\n \"BBox\": {\n \"anyOf\": [\n {\n \"items\": {\n \"type\": \"number\"\n },\n \"maxItems\": 4,\n \"minItems\": 4,\n \"type\": \"array\"\n },\n {\n \"items\": {\n \"type\": \"number\"\n },\n \"maxItems\": 6,\n \"minItems\": 6,\n \"type\": \"array\"\n }\n ],\n \"description\": \"Bounding box https://tools.ietf.org/html/rfc7946#section-5\"\n },\n \"BarConfig\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"align\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Align\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The horizontal alignment of the text or ranged marks (area, bar, image, rect, rule). One of `\\\"left\\\"`, `\\\"right\\\"`, `\\\"center\\\"`.\\n\\n__Note:__ Expression reference is *not* supported for range marks.\"\n },\n \"angle\": {\n \"anyOf\": [\n {\n \"description\": \"The rotation angle of the text, in degrees.\",\n \"maximum\": 360,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"aria\": {\n \"anyOf\": [\n {\n \"description\": \"A boolean flag indicating if [ARIA attributes](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) should be included (SVG output only). If `false`, the \\\"aria-hidden\\\" attribute will be set on the output SVG element, removing the mark item from the ARIA accessibility tree.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"ariaRole\": {\n \"anyOf\": [\n {\n \"description\": \"Sets the type of user interface element of the mark item for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If specified, this property determines the \\\"role\\\" attribute. Warning: this property is experimental and may be changed in the future.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"ariaRoleDescription\": {\n \"anyOf\": [\n {\n \"description\": \"A human-readable, author-localized description for the role of the mark item for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If specified, this property determines the \\\"aria-roledescription\\\" attribute. Warning: this property is experimental and may be changed in the future.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"aspect\": {\n \"anyOf\": [\n {\n \"description\": \"Whether to keep aspect ratio of image marks.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"baseline\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextBaseline\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"For text marks, the vertical text baseline. One of `\\\"alphabetic\\\"` (default), `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`, `\\\"line-top\\\"`, `\\\"line-bottom\\\"`, or an expression reference that provides one of the valid values. The `\\\"line-top\\\"` and `\\\"line-bottom\\\"` values operate similarly to `\\\"top\\\"` and `\\\"bottom\\\"`, but are calculated relative to the `lineHeight` rather than `fontSize` alone.\\n\\nFor range marks, the vertical alignment of the marks. One of `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`.\\n\\n__Note:__ Expression reference is *not* supported for range marks.\"\n },\n \"binSpacing\": {\n \"description\": \"Offset between bars for binned field. The ideal value for this is either 0 (preferred by statisticians) or 1 (Vega-Lite default, D3 example style).\\n\\n__Default value:__ `1`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"blend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Blend\",\n \"description\": \"The color blend mode for drawing an item on its current background. Any valid [CSS mix-blend-mode](https://developer.mozilla.org/en-US/docs/Web/CSS/mix-blend-mode) value can be used.\\n\\n__Default value: `\\\"source-over\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"color\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default color.\\n\\n__Default value:__ ■ `\\\"#4682b4\\\"`\\n\\n__Note:__\\n- This property cannot be used in a [style config](https://vega.github.io/vega-lite/docs/mark.html#style-config).\\n- The `fill` and `stroke` properties have higher precedence than `color` and will override `color`.\"\n },\n \"continuousBandSize\": {\n \"description\": \"The default size of the bars on continuous scales.\\n\\n__Default value:__ `5`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"cornerRadius\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles or arcs' corners.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusBottomLeft\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' bottom left corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusBottomRight\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' bottom right corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusEnd\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"- For vertical bars, top-left and top-right corner radius.\\n\\n- For horizontal bars, top-right and bottom-right corner radius.\"\n },\n \"cornerRadiusTopLeft\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' top right corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusTopRight\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' top left corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cursor\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Cursor\",\n \"description\": \"The mouse cursor used over the mark. Any valid [CSS cursor type](https://developer.mozilla.org/en-US/docs/Web/CSS/cursor#Values) can be used.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"description\": {\n \"anyOf\": [\n {\n \"description\": \"A text description of the mark item for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If specified, this property determines the [\\\"aria-label\\\" attribute](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/ARIA_Techniques/Using_the_aria-label_attribute).\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"dir\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextDirection\",\n \"description\": \"The direction of the text. One of `\\\"ltr\\\"` (left-to-right) or `\\\"rtl\\\"` (right-to-left). This property determines on which side is truncated in response to the limit parameter.\\n\\n__Default value:__ `\\\"ltr\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"discreteBandSize\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/RelativeBandSize\"\n }\n ],\n \"description\": \"The default size of the bars with discrete dimensions. If unspecified, the default size is `step-2`, which provides 2 pixel offset between bars.\",\n \"minimum\": 0\n },\n \"dx\": {\n \"anyOf\": [\n {\n \"description\": \"The horizontal offset, in pixels, between the text label and its anchor point. The offset is applied after rotation by the _angle_ property.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"dy\": {\n \"anyOf\": [\n {\n \"description\": \"The vertical offset, in pixels, between the text label and its anchor point. The offset is applied after rotation by the _angle_ property.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"ellipsis\": {\n \"anyOf\": [\n {\n \"description\": \"The ellipsis string for text truncated in response to the limit parameter.\\n\\n__Default value:__ `\\\"…\\\"`\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"endAngle\": {\n \"anyOf\": [\n {\n \"description\": \"The end angle in radians for arc marks. A value of `0` indicates up (north), increasing values proceed clockwise.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fill\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default fill color. This property has higher precedence than `config.color`. Set to `null` to remove fill.\\n\\n__Default value:__ (None)\"\n },\n \"fillOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"The fill opacity (value between [0,1]).\\n\\n__Default value:__ `1`\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"filled\": {\n \"description\": \"Whether the mark's color should be used as fill color instead of stroke color.\\n\\n__Default value:__ `false` for all `point`, `line`, and `rule` marks as well as `geoshape` marks for [`graticule`](https://vega.github.io/vega-lite/docs/data.html#graticule) data sources; otherwise, `true`.\\n\\n__Note:__ This property cannot be used in a [style config](https://vega.github.io/vega-lite/docs/mark.html#style-config).\",\n \"type\": \"boolean\"\n },\n \"font\": {\n \"anyOf\": [\n {\n \"description\": \"The typeface to set the text in (e.g., `\\\"Helvetica Neue\\\"`).\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontSize\": {\n \"anyOf\": [\n {\n \"description\": \"The font size, in pixels.\\n\\n__Default value:__ `11`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontStyle\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\",\n \"description\": \"The font style (e.g., `\\\"italic\\\"`).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontWeight\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\",\n \"description\": \"The font weight. This can be either a string (e.g `\\\"bold\\\"`, `\\\"normal\\\"`) or a number (`100`, `200`, `300`, ..., `900` where `\\\"normal\\\"` = `400` and `\\\"bold\\\"` = `700`).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"height\": {\n \"anyOf\": [\n {\n \"description\": \"Height of the marks.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"href\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/URI\",\n \"description\": \"A URL to load upon mouse click. If defined, the mark acts as a hyperlink.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"innerRadius\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The inner radius in pixels of arc marks. `innerRadius` is an alias for `radius2`.\\n\\n__Default value:__ `0`\",\n \"minimum\": 0\n },\n \"interpolate\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Interpolate\",\n \"description\": \"The line interpolation method to use for line and area marks. One of the following:\\n- `\\\"linear\\\"`: piecewise linear segments, as in a polyline.\\n- `\\\"linear-closed\\\"`: close the linear segments to form a polygon.\\n- `\\\"step\\\"`: alternate between horizontal and vertical segments, as in a step function.\\n- `\\\"step-before\\\"`: alternate between vertical and horizontal segments, as in a step function.\\n- `\\\"step-after\\\"`: alternate between horizontal and vertical segments, as in a step function.\\n- `\\\"basis\\\"`: a B-spline, with control point duplication on the ends.\\n- `\\\"basis-open\\\"`: an open B-spline; may not intersect the start or end.\\n- `\\\"basis-closed\\\"`: a closed B-spline, as in a loop.\\n- `\\\"cardinal\\\"`: a Cardinal spline, with control point duplication on the ends.\\n- `\\\"cardinal-open\\\"`: an open Cardinal spline; may not intersect the start or end, but will intersect other control points.\\n- `\\\"cardinal-closed\\\"`: a closed Cardinal spline, as in a loop.\\n- `\\\"bundle\\\"`: equivalent to basis, except the tension parameter is used to straighten the spline.\\n- `\\\"monotone\\\"`: cubic interpolation that preserves monotonicity in y.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"invalid\": {\n \"description\": \"Defines how Vega-Lite should handle marks for invalid values (`null` and `NaN`).\\n- If set to `\\\"filter\\\"` (default), all data items with null values will be skipped (for line, trail, and area marks) or filtered (for other marks).\\n- If `null`, all data items are included. In this case, invalid values will be interpreted as zeroes.\",\n \"enum\": [\n \"filter\",\n null\n ],\n \"type\": [\n \"string\",\n \"null\"\n ]\n },\n \"limit\": {\n \"anyOf\": [\n {\n \"description\": \"The maximum length of the text mark in pixels. The text value will be automatically truncated if the rendered size exceeds the limit.\\n\\n__Default value:__ `0` -- indicating no limit\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"lineBreak\": {\n \"anyOf\": [\n {\n \"description\": \"A delimiter, such as a newline character, upon which to break text strings into multiple lines. This property is ignored if the text is array-valued.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"lineHeight\": {\n \"anyOf\": [\n {\n \"description\": \"The line height in pixels (the spacing between subsequent lines of text) for multi-line text marks.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"minBandSize\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The minimum band size for bar and rectangle marks. __Default value:__ `0.25`\"\n },\n \"opacity\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The overall opacity (value between [0,1]).\\n\\n__Default value:__ `0.7` for non-aggregate plots with `point`, `tick`, `circle`, or `square` marks or layered `bar` charts and `1` otherwise.\",\n \"maximum\": 1,\n \"minimum\": 0\n },\n \"order\": {\n \"description\": \"For line and trail marks, this `order` property can be set to `null` or `false` to make the lines use the original order in the data sources.\",\n \"type\": [\n \"null\",\n \"boolean\"\n ]\n },\n \"orient\": {\n \"$ref\": \"#/definitions/Orientation\",\n \"description\": \"The orientation of a non-stacked bar, tick, area, and line charts. The value is either horizontal (default) or vertical.\\n- For bar, rule and tick, this determines whether the size of the bar and tick should be applied to x or y dimension.\\n- For area, this property determines the orient property of the Vega output.\\n- For line and trail marks, this property determines the sort order of the points in the line if `config.sortLineBy` is not specified. For stacked charts, this is always determined by the orientation of the stack; therefore explicitly specified value will be ignored.\"\n },\n \"outerRadius\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The outer radius in pixels of arc marks. `outerRadius` is an alias for `radius`.\\n\\n__Default value:__ `0`\",\n \"minimum\": 0\n },\n \"padAngle\": {\n \"anyOf\": [\n {\n \"description\": \"The angular padding applied to sides of the arc, in radians.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"radius\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"For arc mark, the primary (outer) radius in pixels.\\n\\nFor text marks, polar coordinate radial offset, in pixels, of the text from the origin determined by the `x` and `y` properties.\\n\\n__Default value:__ `min(plot_width, plot_height)/2`\",\n \"minimum\": 0\n },\n \"radius2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The secondary (inner) radius in pixels of arc marks.\\n\\n__Default value:__ `0`\",\n \"minimum\": 0\n },\n \"shape\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SymbolShape\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Shape of the point marks. Supported values include:\\n- plotting shapes: `\\\"circle\\\"`, `\\\"square\\\"`, `\\\"cross\\\"`, `\\\"diamond\\\"`, `\\\"triangle-up\\\"`, `\\\"triangle-down\\\"`, `\\\"triangle-right\\\"`, or `\\\"triangle-left\\\"`.\\n- the line symbol `\\\"stroke\\\"`\\n- centered directional shapes `\\\"arrow\\\"`, `\\\"wedge\\\"`, or `\\\"triangle\\\"`\\n- a custom [SVG path string](https://developer.mozilla.org/en-US/docs/Web/SVG/Tutorial/Paths) (For correct sizing, custom shape paths should be defined within a square bounding box with coordinates ranging from -1 to 1 along both the x and y dimensions.)\\n\\n__Default value:__ `\\\"circle\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"size\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default size for marks.\\n- For `point`/`circle`/`square`, this represents the pixel area of the marks. Note that this value sets the area of the symbol; the side lengths will increase with the square root of this value.\\n- For `bar`, this represents the band size of the bar, in pixels.\\n- For `text`, this represents the font size, in pixels.\\n\\n__Default value:__\\n- `30` for point, circle, square marks; width/height's `step`\\n- `2` for bar marks with discrete dimensions;\\n- `5` for bar marks with continuous dimensions;\\n- `11` for text marks.\",\n \"minimum\": 0\n },\n \"smooth\": {\n \"anyOf\": [\n {\n \"description\": \"A boolean flag (default true) indicating if the image should be smoothed when resized. If false, individual pixels should be scaled directly rather than interpolated with smoothing. For SVG rendering, this option may not work in some browsers due to lack of standardization.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"startAngle\": {\n \"anyOf\": [\n {\n \"description\": \"The start angle in radians for arc marks. A value of `0` indicates up (north), increasing values proceed clockwise.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"stroke\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default stroke color. This property has higher precedence than `config.color`. Set to `null` to remove stroke.\\n\\n__Default value:__ (None)\"\n },\n \"strokeCap\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeCap\",\n \"description\": \"The stroke cap for line ending style. One of `\\\"butt\\\"`, `\\\"round\\\"`, or `\\\"square\\\"`.\\n\\n__Default value:__ `\\\"butt\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeDash\": {\n \"anyOf\": [\n {\n \"description\": \"An array of alternating stroke, space lengths for creating dashed or dotted lines.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeDashOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The offset (in pixels) into which to begin drawing with the stroke dash array.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeJoin\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeJoin\",\n \"description\": \"The stroke line join method. One of `\\\"miter\\\"`, `\\\"round\\\"` or `\\\"bevel\\\"`.\\n\\n__Default value:__ `\\\"miter\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeMiterLimit\": {\n \"anyOf\": [\n {\n \"description\": \"The miter limit at which to bevel a line join.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The offset in pixels at which to draw the group stroke and fill. If unspecified, the default behavior is to dynamically offset stroked groups such that 1 pixel stroke widths align with the pixel grid.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"The stroke opacity (value between [0,1]).\\n\\n__Default value:__ `1`\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeWidth\": {\n \"anyOf\": [\n {\n \"description\": \"The stroke width, in pixels.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"tension\": {\n \"anyOf\": [\n {\n \"description\": \"Depending on the interpolation type, sets the tension parameter (for line and area marks).\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"text\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\",\n \"description\": \"Placeholder text if the `text` channel is not specified\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"theta\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"- For arc marks, the arc length in radians if theta2 is not specified, otherwise the start arc angle. (A value of 0 indicates up or “north”, increasing values proceed clockwise.)\\n\\n- For text marks, polar coordinate angle in radians.\",\n \"maximum\": 360,\n \"minimum\": 0\n },\n \"theta2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The end angle of arc marks in radians. A value of 0 indicates up or “north”, increasing values proceed clockwise.\"\n },\n \"timeUnitBandPosition\": {\n \"description\": \"Default relative band position for a time unit. If set to `0`, the marks will be positioned at the beginning of the time unit band step. If set to `0.5`, the marks will be positioned in the middle of the time unit band step.\",\n \"type\": \"number\"\n },\n \"timeUnitBandSize\": {\n \"description\": \"Default relative band size for a time unit. If set to `1`, the bandwidth of the marks will be equal to the time unit band step. If set to `0.5`, bandwidth of the marks will be half of the time unit band step.\",\n \"type\": \"number\"\n },\n \"tooltip\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/TooltipContent\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"The tooltip text string to show upon mouse hover or an object defining which fields should the tooltip be derived from.\\n\\n- If `tooltip` is `true` or `{\\\"content\\\": \\\"encoding\\\"}`, then all fields from `encoding` will be used.\\n- If `tooltip` is `{\\\"content\\\": \\\"data\\\"}`, then all fields that appear in the highlighted data point will be used.\\n- If set to `null` or `false`, then no tooltip will be used.\\n\\nSee the [`tooltip`](https://vega.github.io/vega-lite/docs/tooltip.html) documentation for a detailed discussion about tooltip in Vega-Lite.\\n\\n__Default value:__ `null`\"\n },\n \"url\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/URI\",\n \"description\": \"The URL of the image file for image marks.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"width\": {\n \"anyOf\": [\n {\n \"description\": \"Width of the marks.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"x\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"X coordinates of the marks, or width of horizontal `\\\"bar\\\"` and `\\\"area\\\"` without specified `x2` or `width`.\\n\\nThe `value` of this channel can be a number or a string `\\\"width\\\"` for the width of the plot.\"\n },\n \"x2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"X2 coordinates for ranged `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"rect\\\"`, and `\\\"rule\\\"`.\\n\\nThe `value` of this channel can be a number or a string `\\\"width\\\"` for the width of the plot.\"\n },\n \"y\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Y coordinates of the marks, or height of vertical `\\\"bar\\\"` and `\\\"area\\\"` without specified `y2` or `height`.\\n\\nThe `value` of this channel can be a number or a string `\\\"height\\\"` for the height of the plot.\"\n },\n \"y2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Y2 coordinates for ranged `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"rect\\\"`, and `\\\"rule\\\"`.\\n\\nThe `value` of this channel can be a number or a string `\\\"height\\\"` for the height of the plot.\"\n }\n },\n \"type\": \"object\"\n },\n \"BaseTitleNoValueRefs\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"align\": {\n \"$ref\": \"#/definitions/Align\",\n \"description\": \"Horizontal text alignment for title text. One of `\\\"left\\\"`, `\\\"center\\\"`, or `\\\"right\\\"`.\"\n },\n \"anchor\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TitleAnchor\",\n \"description\": \"The anchor position for placing the title and subtitle text. One of `\\\"start\\\"`, `\\\"middle\\\"`, or `\\\"end\\\"`. For example, with an orientation of top these anchor positions map to a left-, center-, or right-aligned title.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"angle\": {\n \"anyOf\": [\n {\n \"description\": \"Angle in degrees of title and subtitle text.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"aria\": {\n \"anyOf\": [\n {\n \"description\": \"A boolean flag indicating if [ARIA attributes](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) should be included (SVG output only). If `false`, the \\\"aria-hidden\\\" attribute will be set on the output SVG group, removing the title from the ARIA accessibility tree.\\n\\n__Default value:__ `true`\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"baseline\": {\n \"$ref\": \"#/definitions/TextBaseline\",\n \"description\": \"Vertical text baseline for title and subtitle text. One of `\\\"alphabetic\\\"` (default), `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`, `\\\"line-top\\\"`, or `\\\"line-bottom\\\"`. The `\\\"line-top\\\"` and `\\\"line-bottom\\\"` values operate similarly to `\\\"top\\\"` and `\\\"bottom\\\"`, but are calculated relative to the *lineHeight* rather than *fontSize* alone.\"\n },\n \"color\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"Text color for title text.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"dx\": {\n \"anyOf\": [\n {\n \"description\": \"Delta offset for title and subtitle text x-coordinate.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"dy\": {\n \"anyOf\": [\n {\n \"description\": \"Delta offset for title and subtitle text y-coordinate.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"font\": {\n \"anyOf\": [\n {\n \"description\": \"Font name for title text.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontSize\": {\n \"anyOf\": [\n {\n \"description\": \"Font size in pixels for title text.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontStyle\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\",\n \"description\": \"Font style for title text.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontWeight\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\",\n \"description\": \"Font weight for title text. This can be either a string (e.g `\\\"bold\\\"`, `\\\"normal\\\"`) or a number (`100`, `200`, `300`, ..., `900` where `\\\"normal\\\"` = `400` and `\\\"bold\\\"` = `700`).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"frame\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TitleFrame\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"The reference frame for the anchor position, one of `\\\"bounds\\\"` (to anchor relative to the full bounding box) or `\\\"group\\\"` (to anchor relative to the group width or height).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"limit\": {\n \"anyOf\": [\n {\n \"description\": \"The maximum allowed length in pixels of title and subtitle text.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"lineHeight\": {\n \"anyOf\": [\n {\n \"description\": \"Line height in pixels for multi-line title text or title text with `\\\"line-top\\\"` or `\\\"line-bottom\\\"` baseline.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"offset\": {\n \"anyOf\": [\n {\n \"description\": \"The orthogonal offset in pixels by which to displace the title group from its position along the edge of the chart.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"orient\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TitleOrient\",\n \"description\": \"Default title orientation (`\\\"top\\\"`, `\\\"bottom\\\"`, `\\\"left\\\"`, or `\\\"right\\\"`)\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"subtitleColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"Text color for subtitle text.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"subtitleFont\": {\n \"anyOf\": [\n {\n \"description\": \"Font name for subtitle text.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"subtitleFontSize\": {\n \"anyOf\": [\n {\n \"description\": \"Font size in pixels for subtitle text.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"subtitleFontStyle\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\",\n \"description\": \"Font style for subtitle text.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"subtitleFontWeight\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\",\n \"description\": \"Font weight for subtitle text. This can be either a string (e.g `\\\"bold\\\"`, `\\\"normal\\\"`) or a number (`100`, `200`, `300`, ..., `900` where `\\\"normal\\\"` = `400` and `\\\"bold\\\"` = `700`).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"subtitleLineHeight\": {\n \"anyOf\": [\n {\n \"description\": \"Line height in pixels for multi-line subtitle text.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"subtitlePadding\": {\n \"anyOf\": [\n {\n \"description\": \"The padding in pixels between title and subtitle text.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"zindex\": {\n \"anyOf\": [\n {\n \"description\": \"The integer z-index indicating the layering of the title group relative to other axis, mark, and legend groups.\\n\\n__Default value:__ `0`.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n }\n },\n \"type\": \"object\"\n },\n \"Baseline\": {\n \"enum\": [\n \"top\",\n \"middle\",\n \"bottom\"\n ],\n \"type\": \"string\"\n },\n \"BinExtent\": {\n \"anyOf\": [\n {\n \"items\": {\n \"type\": \"number\"\n },\n \"maxItems\": 2,\n \"minItems\": 2,\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ParameterExtent\"\n }\n ]\n },\n \"BinParams\": {\n \"additionalProperties\": false,\n \"description\": \"Binning properties or boolean flag for determining whether to bin data or not.\",\n \"properties\": {\n \"anchor\": {\n \"description\": \"A value in the binned domain at which to anchor the bins, shifting the bin boundaries if necessary to ensure that a boundary aligns with the anchor value.\\n\\n__Default value:__ the minimum bin extent value\",\n \"type\": \"number\"\n },\n \"base\": {\n \"description\": \"The number base to use for automatic bin determination (default is base 10).\\n\\n__Default value:__ `10`\",\n \"type\": \"number\"\n },\n \"binned\": {\n \"description\": \"When set to `true`, Vega-Lite treats the input data as already binned.\",\n \"type\": \"boolean\"\n },\n \"divide\": {\n \"description\": \"Scale factors indicating allowable subdivisions. The default value is [5, 2], which indicates that for base 10 numbers (the default base), the method may consider dividing bin sizes by 5 and/or 2. For example, for an initial step size of 10, the method can check if bin sizes of 2 (= 10/5), 5 (= 10/2), or 1 (= 10/(5*2)) might also satisfy the given constraints.\\n\\n__Default value:__ `[5, 2]`\",\n \"items\": {\n \"type\": \"number\"\n },\n \"maxItems\": 2,\n \"minItems\": 1,\n \"type\": \"array\"\n },\n \"extent\": {\n \"$ref\": \"#/definitions/BinExtent\",\n \"description\": \"A two-element (`[min, max]`) array indicating the range of desired bin values.\"\n },\n \"maxbins\": {\n \"description\": \"Maximum number of bins.\\n\\n__Default value:__ `6` for `row`, `column` and `shape` channels; `10` for other channels\",\n \"minimum\": 2,\n \"type\": \"number\"\n },\n \"minstep\": {\n \"description\": \"A minimum allowable step size (particularly useful for integer values).\",\n \"type\": \"number\"\n },\n \"nice\": {\n \"description\": \"If true, attempts to make the bin boundaries use human-friendly boundaries, such as multiples of ten.\\n\\n__Default value:__ `true`\",\n \"type\": \"boolean\"\n },\n \"step\": {\n \"description\": \"An exact step size to use between bins.\\n\\n__Note:__ If provided, options such as maxbins will be ignored.\",\n \"type\": \"number\"\n },\n \"steps\": {\n \"description\": \"An array of allowable step sizes to choose from.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"minItems\": 1,\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"BinTransform\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"as\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FieldName\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"The output fields at which to write the start and end bin values. This can be either a string or an array of strings with two elements denoting the name for the fields for bin start and bin end respectively. If a single string (e.g., `\\\"val\\\"`) is provided, the end field will be `\\\"val_end\\\"`.\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"const\": true,\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n }\n ],\n \"description\": \"An object indicating bin properties, or simply `true` for using default bin parameters.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"The data field to bin.\"\n }\n },\n \"required\": [\n \"bin\",\n \"field\",\n \"as\"\n ],\n \"type\": \"object\"\n },\n \"BindCheckbox\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"debounce\": {\n \"description\": \"If defined, delays event handling until the specified milliseconds have elapsed since the last event was fired.\",\n \"type\": \"number\"\n },\n \"element\": {\n \"$ref\": \"#/definitions/Element\",\n \"description\": \"An optional CSS selector string indicating the parent element to which the input element should be added. By default, all input elements are added within the parent container of the Vega view.\"\n },\n \"input\": {\n \"const\": \"checkbox\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"By default, the signal name is used to label input elements. This `name` property can be used instead to specify a custom label for the bound signal.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"input\"\n ],\n \"type\": \"object\"\n },\n \"BindDirect\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"debounce\": {\n \"description\": \"If defined, delays event handling until the specified milliseconds have elapsed since the last event was fired.\",\n \"type\": \"number\"\n },\n \"element\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Element\"\n },\n {\n \"additionalProperties\": false,\n \"type\": \"object\"\n }\n ],\n \"description\": \"An input element that exposes a _value_ property and supports the [EventTarget](https://developer.mozilla.org/en-US/docs/Web/API/EventTarget) interface, or a CSS selector string to such an element. When the element updates and dispatches an event, the _value_ property will be used as the new, bound signal value. When the signal updates independent of the element, the _value_ property will be set to the signal value and a new event will be dispatched on the element.\"\n },\n \"event\": {\n \"description\": \"The event (default `\\\"input\\\"`) to listen for to track changes on the external element.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"element\"\n ],\n \"type\": \"object\"\n },\n \"BindInput\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"autocomplete\": {\n \"description\": \"A hint for form autofill. See the [HTML autocomplete attribute](https://developer.mozilla.org/en-US/docs/Web/HTML/Attributes/autocomplete) for additional information.\",\n \"type\": \"string\"\n },\n \"debounce\": {\n \"description\": \"If defined, delays event handling until the specified milliseconds have elapsed since the last event was fired.\",\n \"type\": \"number\"\n },\n \"element\": {\n \"$ref\": \"#/definitions/Element\",\n \"description\": \"An optional CSS selector string indicating the parent element to which the input element should be added. By default, all input elements are added within the parent container of the Vega view.\"\n },\n \"input\": {\n \"description\": \"The type of input element to use. The valid values are `\\\"checkbox\\\"`, `\\\"radio\\\"`, `\\\"range\\\"`, `\\\"select\\\"`, and any other legal [HTML form input type](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input).\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"By default, the signal name is used to label input elements. This `name` property can be used instead to specify a custom label for the bound signal.\",\n \"type\": \"string\"\n },\n \"placeholder\": {\n \"description\": \"Text that appears in the form control when it has no value set.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"BindRadioSelect\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"debounce\": {\n \"description\": \"If defined, delays event handling until the specified milliseconds have elapsed since the last event was fired.\",\n \"type\": \"number\"\n },\n \"element\": {\n \"$ref\": \"#/definitions/Element\",\n \"description\": \"An optional CSS selector string indicating the parent element to which the input element should be added. By default, all input elements are added within the parent container of the Vega view.\"\n },\n \"input\": {\n \"enum\": [\n \"radio\",\n \"select\"\n ],\n \"type\": \"string\"\n },\n \"labels\": {\n \"description\": \"An array of label strings to represent the `options` values. If unspecified, the `options` value will be coerced to a string and used as the label.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"name\": {\n \"description\": \"By default, the signal name is used to label input elements. This `name` property can be used instead to specify a custom label for the bound signal.\",\n \"type\": \"string\"\n },\n \"options\": {\n \"description\": \"An array of options to select from.\",\n \"items\": {},\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"input\",\n \"options\"\n ],\n \"type\": \"object\"\n },\n \"BindRange\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"debounce\": {\n \"description\": \"If defined, delays event handling until the specified milliseconds have elapsed since the last event was fired.\",\n \"type\": \"number\"\n },\n \"element\": {\n \"$ref\": \"#/definitions/Element\",\n \"description\": \"An optional CSS selector string indicating the parent element to which the input element should be added. By default, all input elements are added within the parent container of the Vega view.\"\n },\n \"input\": {\n \"const\": \"range\",\n \"type\": \"string\"\n },\n \"max\": {\n \"description\": \"Sets the maximum slider value. Defaults to the larger of the signal value and `100`.\",\n \"type\": \"number\"\n },\n \"min\": {\n \"description\": \"Sets the minimum slider value. Defaults to the smaller of the signal value and `0`.\",\n \"type\": \"number\"\n },\n \"name\": {\n \"description\": \"By default, the signal name is used to label input elements. This `name` property can be used instead to specify a custom label for the bound signal.\",\n \"type\": \"string\"\n },\n \"step\": {\n \"description\": \"Sets the minimum slider increment. If undefined, the step size will be automatically determined based on the `min` and `max` values.\",\n \"type\": \"number\"\n }\n },\n \"required\": [\n \"input\"\n ],\n \"type\": \"object\"\n },\n \"Binding\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/BindCheckbox\"\n },\n {\n \"$ref\": \"#/definitions/BindRadioSelect\"\n },\n {\n \"$ref\": \"#/definitions/BindRange\"\n },\n {\n \"$ref\": \"#/definitions/BindInput\"\n },\n {\n \"$ref\": \"#/definitions/BindDirect\"\n }\n ]\n },\n \"BinnedTimeUnit\": {\n \"anyOf\": [\n {\n \"enum\": [\n \"binnedyear\",\n \"binnedyearquarter\",\n \"binnedyearquartermonth\",\n \"binnedyearmonth\",\n \"binnedyearmonthdate\",\n \"binnedyearmonthdatehours\",\n \"binnedyearmonthdatehoursminutes\",\n \"binnedyearmonthdatehoursminutesseconds\",\n \"binnedyearweek\",\n \"binnedyearweekday\",\n \"binnedyearweekdayhours\",\n \"binnedyearweekdayhoursminutes\",\n \"binnedyearweekdayhoursminutesseconds\",\n \"binnedyeardayofyear\"\n ],\n \"type\": \"string\"\n },\n {\n \"enum\": [\n \"binnedutcyear\",\n \"binnedutcyearquarter\",\n \"binnedutcyearquartermonth\",\n \"binnedutcyearmonth\",\n \"binnedutcyearmonthdate\",\n \"binnedutcyearmonthdatehours\",\n \"binnedutcyearmonthdatehoursminutes\",\n \"binnedutcyearmonthdatehoursminutesseconds\",\n \"binnedutcyearweek\",\n \"binnedutcyearweekday\",\n \"binnedutcyearweekdayhours\",\n \"binnedutcyearweekdayhoursminutes\",\n \"binnedutcyearweekdayhoursminutesseconds\",\n \"binnedutcyeardayofyear\"\n ],\n \"type\": \"string\"\n }\n ]\n },\n \"Blend\": {\n \"enum\": [\n null,\n \"multiply\",\n \"screen\",\n \"overlay\",\n \"darken\",\n \"lighten\",\n \"color-dodge\",\n \"color-burn\",\n \"hard-light\",\n \"soft-light\",\n \"difference\",\n \"exclusion\",\n \"hue\",\n \"saturation\",\n \"color\",\n \"luminosity\"\n ],\n \"type\": [\n \"null\",\n \"string\"\n ]\n },\n \"BoxPlot\": {\n \"const\": \"boxplot\",\n \"type\": \"string\"\n },\n \"BoxPlotConfig\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"box\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/AnyMarkConfig\"\n }\n ]\n },\n \"extent\": {\n \"anyOf\": [\n {\n \"const\": \"min-max\",\n \"type\": \"string\"\n },\n {\n \"type\": \"number\"\n }\n ],\n \"description\": \"The extent of the whiskers. Available options include:\\n- `\\\"min-max\\\"`: min and max are the lower and upper whiskers respectively.\\n- A number representing multiple of the interquartile range. This number will be multiplied by the IQR to determine whisker boundary, which spans from the smallest data to the largest data within the range _[Q1 - k * IQR, Q3 + k * IQR]_ where _Q1_ and _Q3_ are the first and third quartiles while _IQR_ is the interquartile range (_Q3-Q1_).\\n\\n__Default value:__ `1.5`.\"\n },\n \"median\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/AnyMarkConfig\"\n }\n ]\n },\n \"outliers\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/AnyMarkConfig\"\n }\n ]\n },\n \"rule\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/AnyMarkConfig\"\n }\n ]\n },\n \"size\": {\n \"description\": \"Size of the box and median tick of a box plot\",\n \"type\": \"number\"\n },\n \"ticks\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/AnyMarkConfig\"\n }\n ]\n }\n },\n \"type\": \"object\"\n },\n \"BoxPlotDef\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"box\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/AnyMarkConfig\"\n }\n ]\n },\n \"clip\": {\n \"description\": \"Whether a composite mark be clipped to the enclosing group’s width and height.\",\n \"type\": \"boolean\"\n },\n \"color\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default color.\\n\\n__Default value:__ ■ `\\\"#4682b4\\\"`\\n\\n__Note:__\\n- This property cannot be used in a [style config](https://vega.github.io/vega-lite/docs/mark.html#style-config).\\n- The `fill` and `stroke` properties have higher precedence than `color` and will override `color`.\"\n },\n \"extent\": {\n \"anyOf\": [\n {\n \"const\": \"min-max\",\n \"type\": \"string\"\n },\n {\n \"type\": \"number\"\n }\n ],\n \"description\": \"The extent of the whiskers. Available options include:\\n- `\\\"min-max\\\"`: min and max are the lower and upper whiskers respectively.\\n- A number representing multiple of the interquartile range. This number will be multiplied by the IQR to determine whisker boundary, which spans from the smallest data to the largest data within the range _[Q1 - k * IQR, Q3 + k * IQR]_ where _Q1_ and _Q3_ are the first and third quartiles while _IQR_ is the interquartile range (_Q3-Q1_).\\n\\n__Default value:__ `1.5`.\"\n },\n \"invalid\": {\n \"description\": \"Defines how Vega-Lite should handle marks for invalid values (`null` and `NaN`).\\n- If set to `\\\"filter\\\"` (default), all data items with null values will be skipped (for line, trail, and area marks) or filtered (for other marks).\\n- If `null`, all data items are included. In this case, invalid values will be interpreted as zeroes.\",\n \"enum\": [\n \"filter\",\n null\n ],\n \"type\": [\n \"string\",\n \"null\"\n ]\n },\n \"median\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/AnyMarkConfig\"\n }\n ]\n },\n \"opacity\": {\n \"description\": \"The opacity (value between [0,1]) of the mark.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"orient\": {\n \"$ref\": \"#/definitions/Orientation\",\n \"description\": \"Orientation of the box plot. This is normally automatically determined based on types of fields on x and y channels. However, an explicit `orient` be specified when the orientation is ambiguous.\\n\\n__Default value:__ `\\\"vertical\\\"`.\"\n },\n \"outliers\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/AnyMarkConfig\"\n }\n ]\n },\n \"rule\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/AnyMarkConfig\"\n }\n ]\n },\n \"size\": {\n \"description\": \"Size of the box and median tick of a box plot\",\n \"type\": \"number\"\n },\n \"ticks\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/AnyMarkConfig\"\n }\n ]\n },\n \"type\": {\n \"$ref\": \"#/definitions/BoxPlot\",\n \"description\": \"The mark type. This could a primitive mark type (one of `\\\"bar\\\"`, `\\\"circle\\\"`, `\\\"square\\\"`, `\\\"tick\\\"`, `\\\"line\\\"`, `\\\"area\\\"`, `\\\"point\\\"`, `\\\"geoshape\\\"`, `\\\"rule\\\"`, and `\\\"text\\\"`) or a composite mark type (`\\\"boxplot\\\"`, `\\\"errorband\\\"`, `\\\"errorbar\\\"`).\"\n }\n },\n \"required\": [\n \"type\"\n ],\n \"type\": \"object\"\n },\n \"BrushConfig\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"cursor\": {\n \"$ref\": \"#/definitions/Cursor\",\n \"description\": \"The mouse cursor used over the interval mark. Any valid [CSS cursor type](https://developer.mozilla.org/en-US/docs/Web/CSS/cursor#Values) can be used.\"\n },\n \"fill\": {\n \"$ref\": \"#/definitions/Color\",\n \"description\": \"The fill color of the interval mark.\\n\\n__Default value:__ `\\\"#333333\\\"`\"\n },\n \"fillOpacity\": {\n \"description\": \"The fill opacity of the interval mark (a value between `0` and `1`).\\n\\n__Default value:__ `0.125`\",\n \"type\": \"number\"\n },\n \"stroke\": {\n \"$ref\": \"#/definitions/Color\",\n \"description\": \"The stroke color of the interval mark.\\n\\n__Default value:__ `\\\"#ffffff\\\"`\"\n },\n \"strokeDash\": {\n \"description\": \"An array of alternating stroke and space lengths, for creating dashed or dotted lines.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n \"strokeDashOffset\": {\n \"description\": \"The offset (in pixels) with which to begin drawing the stroke dash array.\",\n \"type\": \"number\"\n },\n \"strokeOpacity\": {\n \"description\": \"The stroke opacity of the interval mark (a value between `0` and `1`).\",\n \"type\": \"number\"\n },\n \"strokeWidth\": {\n \"description\": \"The stroke width of the interval mark.\",\n \"type\": \"number\"\n }\n },\n \"type\": \"object\"\n },\n \"CalculateTransform\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"as\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"The field for storing the computed formula value.\"\n },\n \"calculate\": {\n \"description\": \"A [expression](https://vega.github.io/vega-lite/docs/types.html#expression) string. Use the variable `datum` to refer to the current data object.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"calculate\",\n \"as\"\n ],\n \"type\": \"object\"\n },\n \"Categorical\": {\n \"enum\": [\n \"accent\",\n \"category10\",\n \"category20\",\n \"category20b\",\n \"category20c\",\n \"dark2\",\n \"paired\",\n \"pastel1\",\n \"pastel2\",\n \"set1\",\n \"set2\",\n \"set3\",\n \"tableau10\",\n \"tableau20\"\n ],\n \"type\": \"string\"\n },\n \"Color\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ColorName\"\n },\n {\n \"$ref\": \"#/definitions/HexColor\"\n },\n {\n \"type\": \"string\"\n }\n ]\n },\n \"ColorDef\": {\n \"$ref\": \"#/definitions/MarkPropDef<(Gradient|string|null)>\"\n },\n \"ColorName\": {\n \"enum\": [\n \"black\",\n \"silver\",\n \"gray\",\n \"white\",\n \"maroon\",\n \"red\",\n \"purple\",\n \"fuchsia\",\n \"green\",\n \"lime\",\n \"olive\",\n \"yellow\",\n \"navy\",\n \"blue\",\n \"teal\",\n \"aqua\",\n \"orange\",\n \"aliceblue\",\n \"antiquewhite\",\n \"aquamarine\",\n \"azure\",\n \"beige\",\n \"bisque\",\n \"blanchedalmond\",\n \"blueviolet\",\n \"brown\",\n \"burlywood\",\n \"cadetblue\",\n \"chartreuse\",\n \"chocolate\",\n \"coral\",\n \"cornflowerblue\",\n \"cornsilk\",\n \"crimson\",\n \"cyan\",\n \"darkblue\",\n \"darkcyan\",\n \"darkgoldenrod\",\n \"darkgray\",\n \"darkgreen\",\n \"darkgrey\",\n \"darkkhaki\",\n \"darkmagenta\",\n \"darkolivegreen\",\n \"darkorange\",\n \"darkorchid\",\n \"darkred\",\n \"darksalmon\",\n \"darkseagreen\",\n \"darkslateblue\",\n \"darkslategray\",\n \"darkslategrey\",\n \"darkturquoise\",\n \"darkviolet\",\n \"deeppink\",\n \"deepskyblue\",\n \"dimgray\",\n \"dimgrey\",\n \"dodgerblue\",\n \"firebrick\",\n \"floralwhite\",\n \"forestgreen\",\n \"gainsboro\",\n \"ghostwhite\",\n \"gold\",\n \"goldenrod\",\n \"greenyellow\",\n \"grey\",\n \"honeydew\",\n \"hotpink\",\n \"indianred\",\n \"indigo\",\n \"ivory\",\n \"khaki\",\n \"lavender\",\n \"lavenderblush\",\n \"lawngreen\",\n \"lemonchiffon\",\n \"lightblue\",\n \"lightcoral\",\n \"lightcyan\",\n \"lightgoldenrodyellow\",\n \"lightgray\",\n \"lightgreen\",\n \"lightgrey\",\n \"lightpink\",\n \"lightsalmon\",\n \"lightseagreen\",\n \"lightskyblue\",\n \"lightslategray\",\n \"lightslategrey\",\n \"lightsteelblue\",\n \"lightyellow\",\n \"limegreen\",\n \"linen\",\n \"magenta\",\n \"mediumaquamarine\",\n \"mediumblue\",\n \"mediumorchid\",\n \"mediumpurple\",\n \"mediumseagreen\",\n \"mediumslateblue\",\n \"mediumspringgreen\",\n \"mediumturquoise\",\n \"mediumvioletred\",\n \"midnightblue\",\n \"mintcream\",\n \"mistyrose\",\n \"moccasin\",\n \"navajowhite\",\n \"oldlace\",\n \"olivedrab\",\n \"orangered\",\n \"orchid\",\n \"palegoldenrod\",\n \"palegreen\",\n \"paleturquoise\",\n \"palevioletred\",\n \"papayawhip\",\n \"peachpuff\",\n \"peru\",\n \"pink\",\n \"plum\",\n \"powderblue\",\n \"rosybrown\",\n \"royalblue\",\n \"saddlebrown\",\n \"salmon\",\n \"sandybrown\",\n \"seagreen\",\n \"seashell\",\n \"sienna\",\n \"skyblue\",\n \"slateblue\",\n \"slategray\",\n \"slategrey\",\n \"snow\",\n \"springgreen\",\n \"steelblue\",\n \"tan\",\n \"thistle\",\n \"tomato\",\n \"turquoise\",\n \"violet\",\n \"wheat\",\n \"whitesmoke\",\n \"yellowgreen\",\n \"rebeccapurple\"\n ],\n \"type\": \"string\"\n },\n \"ColorScheme\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Categorical\"\n },\n {\n \"$ref\": \"#/definitions/SequentialSingleHue\"\n },\n {\n \"$ref\": \"#/definitions/SequentialMultiHue\"\n },\n {\n \"$ref\": \"#/definitions/Diverging\"\n },\n {\n \"$ref\": \"#/definitions/Cyclical\"\n }\n ]\n },\n \"Encoding\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"angle\": {\n \"$ref\": \"#/definitions/NumericMarkPropDef\",\n \"description\": \"Rotation angle of point and text marks.\"\n },\n \"color\": {\n \"$ref\": \"#/definitions/ColorDef\",\n \"description\": \"Color of the marks – either fill or stroke color based on the `filled` property of mark definition. By default, `color` represents fill color for `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"tick\\\"`, `\\\"text\\\"`, `\\\"trail\\\"`, `\\\"circle\\\"`, and `\\\"square\\\"` / stroke color for `\\\"line\\\"` and `\\\"point\\\"`.\\n\\n__Default value:__ If undefined, the default color depends on [mark config](https://vega.github.io/vega-lite/docs/config.html#mark-config)'s `color` property.\\n\\n_Note:_ 1) For fine-grained control over both fill and stroke colors of the marks, please use the `fill` and `stroke` channels. The `fill` or `stroke` encodings have higher precedence than `color`, thus may override the `color` encoding if conflicting encodings are specified. 2) See the scale documentation for more information about customizing [color scheme](https://vega.github.io/vega-lite/docs/scale.html#scheme).\"\n },\n \"description\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StringFieldDefWithCondition\"\n },\n {\n \"$ref\": \"#/definitions/StringValueDefWithCondition\"\n }\n ],\n \"description\": \"A text description of this mark for ARIA accessibility (SVG output only). For SVG output the `\\\"aria-label\\\"` attribute will be set to this description.\"\n },\n \"detail\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FieldDefWithoutScale\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/FieldDefWithoutScale\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"Additional levels of detail for grouping data in aggregate views and in line, trail, and area marks without mapping data to a specific visual channel.\"\n },\n \"fill\": {\n \"$ref\": \"#/definitions/ColorDef\",\n \"description\": \"Fill color of the marks. __Default value:__ If undefined, the default color depends on [mark config](https://vega.github.io/vega-lite/docs/config.html#mark-config)'s `color` property.\\n\\n_Note:_ The `fill` encoding has higher precedence than `color`, thus may override the `color` encoding if conflicting encodings are specified.\"\n },\n \"fillOpacity\": {\n \"$ref\": \"#/definitions/NumericMarkPropDef\",\n \"description\": \"Fill opacity of the marks.\\n\\n__Default value:__ If undefined, the default opacity depends on [mark config](https://vega.github.io/vega-lite/docs/config.html#mark-config)'s `fillOpacity` property.\"\n },\n \"href\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StringFieldDefWithCondition\"\n },\n {\n \"$ref\": \"#/definitions/StringValueDefWithCondition\"\n }\n ],\n \"description\": \"A URL to load upon mouse click.\"\n },\n \"key\": {\n \"$ref\": \"#/definitions/FieldDefWithoutScale\",\n \"description\": \"A data field to use as a unique key for data binding. When a visualization’s data is updated, the key value will be used to match data elements to existing mark instances. Use a key channel to enable object constancy for transitions over dynamic data.\"\n },\n \"latitude\": {\n \"$ref\": \"#/definitions/LatLongDef\",\n \"description\": \"Latitude position of geographically projected marks.\"\n },\n \"latitude2\": {\n \"$ref\": \"#/definitions/Position2Def\",\n \"description\": \"Latitude-2 position for geographically projected ranged `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"rect\\\"`, and `\\\"rule\\\"`.\"\n },\n \"longitude\": {\n \"$ref\": \"#/definitions/LatLongDef\",\n \"description\": \"Longitude position of geographically projected marks.\"\n },\n \"longitude2\": {\n \"$ref\": \"#/definitions/Position2Def\",\n \"description\": \"Longitude-2 position for geographically projected ranged `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"rect\\\"`, and `\\\"rule\\\"`.\"\n },\n \"opacity\": {\n \"$ref\": \"#/definitions/NumericMarkPropDef\",\n \"description\": \"Opacity of the marks.\\n\\n__Default value:__ If undefined, the default opacity depends on [mark config](https://vega.github.io/vega-lite/docs/config.html#mark-config)'s `opacity` property.\"\n },\n \"order\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/OrderFieldDef\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/OrderFieldDef\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/OrderValueDef\"\n },\n {\n \"$ref\": \"#/definitions/OrderOnlyDef\"\n }\n ],\n \"description\": \"Order of the marks.\\n- For stacked marks, this `order` channel encodes [stack order](https://vega.github.io/vega-lite/docs/stack.html#order).\\n- For line and trail marks, this `order` channel encodes order of data points in the lines. This can be useful for creating [a connected scatterplot](https://vega.github.io/vega-lite/examples/connected_scatterplot.html). Setting `order` to `{\\\"value\\\": null}` makes the line marks use the original order in the data sources.\\n- Otherwise, this `order` channel encodes layer order of the marks.\\n\\n__Note__: In aggregate plots, `order` field should be `aggregate`d to avoid creating additional aggregation grouping.\"\n },\n \"radius\": {\n \"$ref\": \"#/definitions/PolarDef\",\n \"description\": \"The outer radius in pixels of arc marks.\"\n },\n \"radius2\": {\n \"$ref\": \"#/definitions/Position2Def\",\n \"description\": \"The inner radius in pixels of arc marks.\"\n },\n \"shape\": {\n \"$ref\": \"#/definitions/ShapeDef\",\n \"description\": \"Shape of the mark.\\n\\n1. For `point` marks the supported values include: - plotting shapes: `\\\"circle\\\"`, `\\\"square\\\"`, `\\\"cross\\\"`, `\\\"diamond\\\"`, `\\\"triangle-up\\\"`, `\\\"triangle-down\\\"`, `\\\"triangle-right\\\"`, or `\\\"triangle-left\\\"`. - the line symbol `\\\"stroke\\\"` - centered directional shapes `\\\"arrow\\\"`, `\\\"wedge\\\"`, or `\\\"triangle\\\"` - a custom [SVG path string](https://developer.mozilla.org/en-US/docs/Web/SVG/Tutorial/Paths) (For correct sizing, custom shape paths should be defined within a square bounding box with coordinates ranging from -1 to 1 along both the x and y dimensions.)\\n\\n2. For `geoshape` marks it should be a field definition of the geojson data\\n\\n__Default value:__ If undefined, the default shape depends on [mark config](https://vega.github.io/vega-lite/docs/config.html#point-config)'s `shape` property. (`\\\"circle\\\"` if unset.)\"\n },\n \"size\": {\n \"$ref\": \"#/definitions/NumericMarkPropDef\",\n \"description\": \"Size of the mark.\\n- For `\\\"point\\\"`, `\\\"square\\\"` and `\\\"circle\\\"`, – the symbol size, or pixel area of the mark.\\n- For `\\\"bar\\\"` and `\\\"tick\\\"` – the bar and tick's size.\\n- For `\\\"text\\\"` – the text's font size.\\n- Size is unsupported for `\\\"line\\\"`, `\\\"area\\\"`, and `\\\"rect\\\"`. (Use `\\\"trail\\\"` instead of line with varying size)\"\n },\n \"stroke\": {\n \"$ref\": \"#/definitions/ColorDef\",\n \"description\": \"Stroke color of the marks. __Default value:__ If undefined, the default color depends on [mark config](https://vega.github.io/vega-lite/docs/config.html#mark-config)'s `color` property.\\n\\n_Note:_ The `stroke` encoding has higher precedence than `color`, thus may override the `color` encoding if conflicting encodings are specified.\"\n },\n \"strokeDash\": {\n \"$ref\": \"#/definitions/NumericArrayMarkPropDef\",\n \"description\": \"Stroke dash of the marks.\\n\\n__Default value:__ `[1,0]` (No dash).\"\n },\n \"strokeOpacity\": {\n \"$ref\": \"#/definitions/NumericMarkPropDef\",\n \"description\": \"Stroke opacity of the marks.\\n\\n__Default value:__ If undefined, the default opacity depends on [mark config](https://vega.github.io/vega-lite/docs/config.html#mark-config)'s `strokeOpacity` property.\"\n },\n \"strokeWidth\": {\n \"$ref\": \"#/definitions/NumericMarkPropDef\",\n \"description\": \"Stroke width of the marks.\\n\\n__Default value:__ If undefined, the default stroke width depends on [mark config](https://vega.github.io/vega-lite/docs/config.html#mark-config)'s `strokeWidth` property.\"\n },\n \"text\": {\n \"$ref\": \"#/definitions/TextDef\",\n \"description\": \"Text of the `text` mark.\"\n },\n \"theta\": {\n \"$ref\": \"#/definitions/PolarDef\",\n \"description\": \"- For arc marks, the arc length in radians if theta2 is not specified, otherwise the start arc angle. (A value of 0 indicates up or “north”, increasing values proceed clockwise.)\\n\\n- For text marks, polar coordinate angle in radians.\"\n },\n \"theta2\": {\n \"$ref\": \"#/definitions/Position2Def\",\n \"description\": \"The end angle of arc marks in radians. A value of 0 indicates up or “north”, increasing values proceed clockwise.\"\n },\n \"tooltip\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StringFieldDefWithCondition\"\n },\n {\n \"$ref\": \"#/definitions/StringValueDefWithCondition\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/StringFieldDef\"\n },\n \"type\": \"array\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"The tooltip text to show upon mouse hover. Specifying `tooltip` encoding overrides [the `tooltip` property in the mark definition](https://vega.github.io/vega-lite/docs/mark.html#mark-def).\\n\\nSee the [`tooltip`](https://vega.github.io/vega-lite/docs/tooltip.html) documentation for a detailed discussion about tooltip in Vega-Lite.\"\n },\n \"url\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StringFieldDefWithCondition\"\n },\n {\n \"$ref\": \"#/definitions/StringValueDefWithCondition\"\n }\n ],\n \"description\": \"The URL of an image mark.\"\n },\n \"x\": {\n \"$ref\": \"#/definitions/PositionDef\",\n \"description\": \"X coordinates of the marks, or width of horizontal `\\\"bar\\\"` and `\\\"area\\\"` without specified `x2` or `width`.\\n\\nThe `value` of this channel can be a number or a string `\\\"width\\\"` for the width of the plot.\"\n },\n \"x2\": {\n \"$ref\": \"#/definitions/Position2Def\",\n \"description\": \"X2 coordinates for ranged `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"rect\\\"`, and `\\\"rule\\\"`.\\n\\nThe `value` of this channel can be a number or a string `\\\"width\\\"` for the width of the plot.\"\n },\n \"xError\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SecondaryFieldDef\"\n },\n {\n \"$ref\": \"#/definitions/ValueDef\"\n }\n ],\n \"description\": \"Error value of x coordinates for error specified `\\\"errorbar\\\"` and `\\\"errorband\\\"`.\"\n },\n \"xError2\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SecondaryFieldDef\"\n },\n {\n \"$ref\": \"#/definitions/ValueDef\"\n }\n ],\n \"description\": \"Secondary error value of x coordinates for error specified `\\\"errorbar\\\"` and `\\\"errorband\\\"`.\"\n },\n \"xOffset\": {\n \"$ref\": \"#/definitions/OffsetDef\",\n \"description\": \"Offset of x-position of the marks\"\n },\n \"y\": {\n \"$ref\": \"#/definitions/PositionDef\",\n \"description\": \"Y coordinates of the marks, or height of vertical `\\\"bar\\\"` and `\\\"area\\\"` without specified `y2` or `height`.\\n\\nThe `value` of this channel can be a number or a string `\\\"height\\\"` for the height of the plot.\"\n },\n \"y2\": {\n \"$ref\": \"#/definitions/Position2Def\",\n \"description\": \"Y2 coordinates for ranged `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"rect\\\"`, and `\\\"rule\\\"`.\\n\\nThe `value` of this channel can be a number or a string `\\\"height\\\"` for the height of the plot.\"\n },\n \"yError\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SecondaryFieldDef\"\n },\n {\n \"$ref\": \"#/definitions/ValueDef\"\n }\n ],\n \"description\": \"Error value of y coordinates for error specified `\\\"errorbar\\\"` and `\\\"errorband\\\"`.\"\n },\n \"yError2\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SecondaryFieldDef\"\n },\n {\n \"$ref\": \"#/definitions/ValueDef\"\n }\n ],\n \"description\": \"Secondary error value of y coordinates for error specified `\\\"errorbar\\\"` and `\\\"errorband\\\"`.\"\n },\n \"yOffset\": {\n \"$ref\": \"#/definitions/OffsetDef\",\n \"description\": \"Offset of y-position of the marks\"\n }\n },\n \"type\": \"object\"\n },\n \"CompositeMark\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/BoxPlot\"\n },\n {\n \"$ref\": \"#/definitions/ErrorBar\"\n },\n {\n \"$ref\": \"#/definitions/ErrorBand\"\n }\n ]\n },\n \"CompositeMarkDef\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/BoxPlotDef\"\n },\n {\n \"$ref\": \"#/definitions/ErrorBarDef\"\n },\n {\n \"$ref\": \"#/definitions/ErrorBandDef\"\n }\n ]\n },\n \"CompositionConfig\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"columns\": {\n \"description\": \"The number of columns to include in the view composition layout.\\n\\n__Default value__: `undefined` -- An infinite number of columns (a single row) will be assumed. This is equivalent to `hconcat` (for `concat`) and to using the `column` channel (for `facet` and `repeat`).\\n\\n__Note__:\\n\\n1) This property is only for:\\n- the general (wrappable) `concat` operator (not `hconcat`/`vconcat`)\\n- the `facet` and `repeat` operator with one field/repetition definition (without row/column nesting)\\n\\n2) Setting the `columns` to `1` is equivalent to `vconcat` (for `concat`) and to using the `row` channel (for `facet` and `repeat`).\",\n \"type\": \"number\"\n },\n \"spacing\": {\n \"description\": \"The default spacing in pixels between composed sub-views.\\n\\n__Default value__: `20`\",\n \"type\": \"number\"\n }\n },\n \"type\": \"object\"\n },\n \"ConditionalMarkPropFieldOrDatumDef\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalParameter\"\n }\n ]\n },\n \"ConditionalMarkPropFieldOrDatumDef\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate>\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalParameter>\"\n }\n ]\n },\n \"ConditionalStringFieldDef\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalParameter\"\n }\n ]\n },\n \"ConditionalValueDef<(Gradient|string|null|ExprRef)>\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate>\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalParameter>\"\n }\n ]\n },\n \"ConditionalValueDef<(Text|ExprRef)>\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate>\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalParameter>\"\n }\n ]\n },\n \"ConditionalValueDef<(number[]|ExprRef)>\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate>\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalParameter>\"\n }\n ]\n },\n \"ConditionalValueDef<(number|ExprRef)>\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate>\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalParameter>\"\n }\n ]\n },\n \"ConditionalValueDef<(string|ExprRef)>\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate>\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalParameter>\"\n }\n ]\n },\n \"ConditionalValueDef<(string|null|ExprRef)>\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate>\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalParameter>\"\n }\n ]\n },\n \"ConditionalValueDef\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate>\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalParameter>\"\n }\n ]\n },\n \"ConditionalAxisColor\": {\n \"$ref\": \"#/definitions/ConditionalAxisProperty<(Color|null)>\"\n },\n \"ConditionalAxisLabelAlign\": {\n \"$ref\": \"#/definitions/ConditionalAxisProperty<(Align|null)>\"\n },\n \"ConditionalAxisLabelBaseline\": {\n \"$ref\": \"#/definitions/ConditionalAxisProperty<(TextBaseline|null)>\"\n },\n \"ConditionalAxisLabelFontStyle\": {\n \"$ref\": \"#/definitions/ConditionalAxisProperty<(FontStyle|null)>\"\n },\n \"ConditionalAxisLabelFontWeight\": {\n \"$ref\": \"#/definitions/ConditionalAxisProperty<(FontWeight|null)>\"\n },\n \"ConditionalAxisNumber\": {\n \"$ref\": \"#/definitions/ConditionalAxisProperty<(number|null)>\"\n },\n \"ConditionalAxisNumberArray\": {\n \"$ref\": \"#/definitions/ConditionalAxisProperty<(number[]|null)>\"\n },\n \"ConditionalAxisProperty<(Align|null)>\": {\n \"anyOf\": [\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(Align|null)>|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(Align|null)>|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ]\n },\n \"value\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Align\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"condition\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(Align|null)>|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(Align|null)>|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ]\n },\n \"expr\": {\n \"description\": \"Vega expression (which can refer to Vega-Lite parameters).\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"condition\",\n \"expr\"\n ],\n \"type\": \"object\"\n }\n ]\n },\n \"ConditionalAxisProperty<(Color|null)>\": {\n \"anyOf\": [\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(Color|null)>|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(Color|null)>|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ]\n },\n \"value\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"condition\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(Color|null)>|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(Color|null)>|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ]\n },\n \"expr\": {\n \"description\": \"Vega expression (which can refer to Vega-Lite parameters).\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"condition\",\n \"expr\"\n ],\n \"type\": \"object\"\n }\n ]\n },\n \"ConditionalAxisProperty<(FontStyle|null)>\": {\n \"anyOf\": [\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(FontStyle|null)>|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(FontStyle|null)>|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ]\n },\n \"value\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"condition\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(FontStyle|null)>|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(FontStyle|null)>|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ]\n },\n \"expr\": {\n \"description\": \"Vega expression (which can refer to Vega-Lite parameters).\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"condition\",\n \"expr\"\n ],\n \"type\": \"object\"\n }\n ]\n },\n \"ConditionalAxisProperty<(FontWeight|null)>\": {\n \"anyOf\": [\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(FontWeight|null)>|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(FontWeight|null)>|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ]\n },\n \"value\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"condition\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(FontWeight|null)>|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(FontWeight|null)>|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ]\n },\n \"expr\": {\n \"description\": \"Vega expression (which can refer to Vega-Lite parameters).\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"condition\",\n \"expr\"\n ],\n \"type\": \"object\"\n }\n ]\n },\n \"ConditionalAxisProperty<(TextBaseline|null)>\": {\n \"anyOf\": [\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(TextBaseline|null)>|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(TextBaseline|null)>|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ]\n },\n \"value\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextBaseline\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"condition\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(TextBaseline|null)>|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(TextBaseline|null)>|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ]\n },\n \"expr\": {\n \"description\": \"Vega expression (which can refer to Vega-Lite parameters).\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"condition\",\n \"expr\"\n ],\n \"type\": \"object\"\n }\n ]\n },\n \"ConditionalAxisProperty<(number[]|null)>\": {\n \"anyOf\": [\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(number[]|null)>|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(number[]|null)>|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ]\n },\n \"value\": {\n \"anyOf\": [\n {\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"condition\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(number[]|null)>|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(number[]|null)>|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ]\n },\n \"expr\": {\n \"description\": \"Vega expression (which can refer to Vega-Lite parameters).\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"condition\",\n \"expr\"\n ],\n \"type\": \"object\"\n }\n ]\n },\n \"ConditionalAxisProperty<(number|null)>\": {\n \"anyOf\": [\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(number|null)>|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(number|null)>|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ]\n },\n \"value\": {\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\",\n \"type\": [\n \"number\",\n \"null\"\n ]\n }\n },\n \"required\": [\n \"condition\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(number|null)>|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(number|null)>|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ]\n },\n \"expr\": {\n \"description\": \"Vega expression (which can refer to Vega-Lite parameters).\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"condition\",\n \"expr\"\n ],\n \"type\": \"object\"\n }\n ]\n },\n \"ConditionalAxisProperty<(string|null)>\": {\n \"anyOf\": [\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(string|null)>|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(string|null)>|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ]\n },\n \"value\": {\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\",\n \"type\": [\n \"string\",\n \"null\"\n ]\n }\n },\n \"required\": [\n \"condition\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(string|null)>|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalPredicate<(ValueDef<(string|null)>|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ]\n },\n \"expr\": {\n \"description\": \"Vega expression (which can refer to Vega-Lite parameters).\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"condition\",\n \"expr\"\n ],\n \"type\": \"object\"\n }\n ]\n },\n \"ConditionalAxisString\": {\n \"$ref\": \"#/definitions/ConditionalAxisProperty<(string|null)>\"\n },\n \"ConditionalParameter\": {\n \"anyOf\": [\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"empty\": {\n \"description\": \"For selection parameters, the predicate of empty selections returns true by default. Override this behavior, by setting this property `empty: false`.\",\n \"type\": \"boolean\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"legend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Legend\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the legend. If `null`, the legend for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [legend properties](https://vega.github.io/vega-lite/docs/legend.html) are applied.\\n\\n__See also:__ [`legend`](https://vega.github.io/vega-lite/docs/legend.html) documentation.\"\n },\n \"param\": {\n \"$ref\": \"#/definitions/ParameterName\",\n \"description\": \"Filter using a parameter name.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"required\": [\n \"param\"\n ],\n \"type\": \"object\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"empty\": {\n \"description\": \"For selection parameters, the predicate of empty selections returns true by default. Override this behavior, by setting this property `empty: false`.\",\n \"type\": \"boolean\"\n },\n \"legend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Legend\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the legend. If `null`, the legend for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [legend properties](https://vega.github.io/vega-lite/docs/legend.html) are applied.\\n\\n__See also:__ [`legend`](https://vega.github.io/vega-lite/docs/legend.html) documentation.\"\n },\n \"param\": {\n \"$ref\": \"#/definitions/ParameterName\",\n \"description\": \"Filter using a parameter name.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"required\": [\n \"param\"\n ],\n \"type\": \"object\"\n }\n ]\n },\n \"ConditionalParameter>\": {\n \"anyOf\": [\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"empty\": {\n \"description\": \"For selection parameters, the predicate of empty selections returns true by default. Override this behavior, by setting this property `empty: false`.\",\n \"type\": \"boolean\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"legend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Legend\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the legend. If `null`, the legend for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [legend properties](https://vega.github.io/vega-lite/docs/legend.html) are applied.\\n\\n__See also:__ [`legend`](https://vega.github.io/vega-lite/docs/legend.html) documentation.\"\n },\n \"param\": {\n \"$ref\": \"#/definitions/ParameterName\",\n \"description\": \"Filter using a parameter name.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/TypeForShape\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"required\": [\n \"param\"\n ],\n \"type\": \"object\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"empty\": {\n \"description\": \"For selection parameters, the predicate of empty selections returns true by default. Override this behavior, by setting this property `empty: false`.\",\n \"type\": \"boolean\"\n },\n \"legend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Legend\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the legend. If `null`, the legend for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [legend properties](https://vega.github.io/vega-lite/docs/legend.html) are applied.\\n\\n__See also:__ [`legend`](https://vega.github.io/vega-lite/docs/legend.html) documentation.\"\n },\n \"param\": {\n \"$ref\": \"#/definitions/ParameterName\",\n \"description\": \"Filter using a parameter name.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"required\": [\n \"param\"\n ],\n \"type\": \"object\"\n }\n ]\n },\n \"ConditionalParameter\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"const\": \"binned\",\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"empty\": {\n \"description\": \"For selection parameters, the predicate of empty selections returns true by default. Override this behavior, by setting this property `empty: false`.\",\n \"type\": \"boolean\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"format\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Dict\"\n }\n ],\n \"description\": \"When used with the default `\\\"number\\\"` and `\\\"time\\\"` format type, the text formatting pattern for labels of guides (axes, legends, headers) and text marks.\\n\\n- If the format type is `\\\"number\\\"` (e.g., for quantitative fields), this is D3's [number format pattern](https://github.com/d3/d3-format#locale_format).\\n- If the format type is `\\\"time\\\"` (e.g., for temporal fields), this is D3's [time format pattern](https://github.com/d3/d3-time-format#locale_format).\\n\\nSee the [format documentation](https://vega.github.io/vega-lite/docs/format.html) for more examples.\\n\\nWhen used with a [custom `formatType`](https://vega.github.io/vega-lite/docs/config.html#custom-format-type), this value will be passed as `format` alongside `datum.value` to the registered function.\\n\\n__Default value:__ Derived from [numberFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for number format and from [timeFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for time format.\"\n },\n \"formatType\": {\n \"description\": \"The format type for labels. One of `\\\"number\\\"`, `\\\"time\\\"`, or a [registered custom format type](https://vega.github.io/vega-lite/docs/config.html#custom-format-type).\\n\\n__Default value:__\\n- `\\\"time\\\"` for temporal fields and ordinal and nominal fields with `timeUnit`.\\n- `\\\"number\\\"` for quantitative fields as well as ordinal and nominal fields without `timeUnit`.\",\n \"type\": \"string\"\n },\n \"param\": {\n \"$ref\": \"#/definitions/ParameterName\",\n \"description\": \"Filter using a parameter name.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"required\": [\n \"param\"\n ],\n \"type\": \"object\"\n },\n \"ConditionalParameter>\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"empty\": {\n \"description\": \"For selection parameters, the predicate of empty selections returns true by default. Override this behavior, by setting this property `empty: false`.\",\n \"type\": \"boolean\"\n },\n \"param\": {\n \"$ref\": \"#/definitions/ParameterName\",\n \"description\": \"Filter using a parameter name.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"param\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"ConditionalParameter>\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"empty\": {\n \"description\": \"For selection parameters, the predicate of empty selections returns true by default. Override this behavior, by setting this property `empty: false`.\",\n \"type\": \"boolean\"\n },\n \"param\": {\n \"$ref\": \"#/definitions/ParameterName\",\n \"description\": \"Filter using a parameter name.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"param\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"ConditionalParameter>\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"empty\": {\n \"description\": \"For selection parameters, the predicate of empty selections returns true by default. Override this behavior, by setting this property `empty: false`.\",\n \"type\": \"boolean\"\n },\n \"param\": {\n \"$ref\": \"#/definitions/ParameterName\",\n \"description\": \"Filter using a parameter name.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"param\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"ConditionalParameter>\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"empty\": {\n \"description\": \"For selection parameters, the predicate of empty selections returns true by default. Override this behavior, by setting this property `empty: false`.\",\n \"type\": \"boolean\"\n },\n \"param\": {\n \"$ref\": \"#/definitions/ParameterName\",\n \"description\": \"Filter using a parameter name.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"param\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"ConditionalParameter>\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"empty\": {\n \"description\": \"For selection parameters, the predicate of empty selections returns true by default. Override this behavior, by setting this property `empty: false`.\",\n \"type\": \"boolean\"\n },\n \"param\": {\n \"$ref\": \"#/definitions/ParameterName\",\n \"description\": \"Filter using a parameter name.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"param\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"ConditionalParameter>\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"empty\": {\n \"description\": \"For selection parameters, the predicate of empty selections returns true by default. Override this behavior, by setting this property `empty: false`.\",\n \"type\": \"boolean\"\n },\n \"param\": {\n \"$ref\": \"#/definitions/ParameterName\",\n \"description\": \"Filter using a parameter name.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"param\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"ConditionalParameter>\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"empty\": {\n \"description\": \"For selection parameters, the predicate of empty selections returns true by default. Override this behavior, by setting this property `empty: false`.\",\n \"type\": \"boolean\"\n },\n \"param\": {\n \"$ref\": \"#/definitions/ParameterName\",\n \"description\": \"Filter using a parameter name.\"\n },\n \"value\": {\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\",\n \"type\": \"number\"\n }\n },\n \"required\": [\n \"param\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"ConditionalPredicate<(ValueDef<(Align|null)>|ExprRef)>\": {\n \"anyOf\": [\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Align\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"test\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"expr\": {\n \"description\": \"Vega expression (which can refer to Vega-Lite parameters).\",\n \"type\": \"string\"\n },\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n }\n },\n \"required\": [\n \"expr\",\n \"test\"\n ],\n \"type\": \"object\"\n }\n ]\n },\n \"ConditionalPredicate<(ValueDef<(Color|null)>|ExprRef)>\": {\n \"anyOf\": [\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"test\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"expr\": {\n \"description\": \"Vega expression (which can refer to Vega-Lite parameters).\",\n \"type\": \"string\"\n },\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n }\n },\n \"required\": [\n \"expr\",\n \"test\"\n ],\n \"type\": \"object\"\n }\n ]\n },\n \"ConditionalPredicate<(ValueDef<(FontStyle|null)>|ExprRef)>\": {\n \"anyOf\": [\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"test\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"expr\": {\n \"description\": \"Vega expression (which can refer to Vega-Lite parameters).\",\n \"type\": \"string\"\n },\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n }\n },\n \"required\": [\n \"expr\",\n \"test\"\n ],\n \"type\": \"object\"\n }\n ]\n },\n \"ConditionalPredicate<(ValueDef<(FontWeight|null)>|ExprRef)>\": {\n \"anyOf\": [\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"test\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"expr\": {\n \"description\": \"Vega expression (which can refer to Vega-Lite parameters).\",\n \"type\": \"string\"\n },\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n }\n },\n \"required\": [\n \"expr\",\n \"test\"\n ],\n \"type\": \"object\"\n }\n ]\n },\n \"ConditionalPredicate<(ValueDef<(TextBaseline|null)>|ExprRef)>\": {\n \"anyOf\": [\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextBaseline\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"test\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"expr\": {\n \"description\": \"Vega expression (which can refer to Vega-Lite parameters).\",\n \"type\": \"string\"\n },\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n }\n },\n \"required\": [\n \"expr\",\n \"test\"\n ],\n \"type\": \"object\"\n }\n ]\n },\n \"ConditionalPredicate<(ValueDef<(number[]|null)>|ExprRef)>\": {\n \"anyOf\": [\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"test\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"expr\": {\n \"description\": \"Vega expression (which can refer to Vega-Lite parameters).\",\n \"type\": \"string\"\n },\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n }\n },\n \"required\": [\n \"expr\",\n \"test\"\n ],\n \"type\": \"object\"\n }\n ]\n },\n \"ConditionalPredicate<(ValueDef<(number|null)>|ExprRef)>\": {\n \"anyOf\": [\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n },\n \"value\": {\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\",\n \"type\": [\n \"number\",\n \"null\"\n ]\n }\n },\n \"required\": [\n \"test\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"expr\": {\n \"description\": \"Vega expression (which can refer to Vega-Lite parameters).\",\n \"type\": \"string\"\n },\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n }\n },\n \"required\": [\n \"expr\",\n \"test\"\n ],\n \"type\": \"object\"\n }\n ]\n },\n \"ConditionalPredicate<(ValueDef<(string|null)>|ExprRef)>\": {\n \"anyOf\": [\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n },\n \"value\": {\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\",\n \"type\": [\n \"string\",\n \"null\"\n ]\n }\n },\n \"required\": [\n \"test\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"expr\": {\n \"description\": \"Vega expression (which can refer to Vega-Lite parameters).\",\n \"type\": \"string\"\n },\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n }\n },\n \"required\": [\n \"expr\",\n \"test\"\n ],\n \"type\": \"object\"\n }\n ]\n },\n \"ConditionalPredicate\": {\n \"anyOf\": [\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"legend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Legend\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the legend. If `null`, the legend for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [legend properties](https://vega.github.io/vega-lite/docs/legend.html) are applied.\\n\\n__See also:__ [`legend`](https://vega.github.io/vega-lite/docs/legend.html) documentation.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"required\": [\n \"test\"\n ],\n \"type\": \"object\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"legend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Legend\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the legend. If `null`, the legend for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [legend properties](https://vega.github.io/vega-lite/docs/legend.html) are applied.\\n\\n__See also:__ [`legend`](https://vega.github.io/vega-lite/docs/legend.html) documentation.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"required\": [\n \"test\"\n ],\n \"type\": \"object\"\n }\n ]\n },\n \"ConditionalPredicate>\": {\n \"anyOf\": [\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"legend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Legend\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the legend. If `null`, the legend for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [legend properties](https://vega.github.io/vega-lite/docs/legend.html) are applied.\\n\\n__See also:__ [`legend`](https://vega.github.io/vega-lite/docs/legend.html) documentation.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/TypeForShape\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"required\": [\n \"test\"\n ],\n \"type\": \"object\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"legend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Legend\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the legend. If `null`, the legend for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [legend properties](https://vega.github.io/vega-lite/docs/legend.html) are applied.\\n\\n__See also:__ [`legend`](https://vega.github.io/vega-lite/docs/legend.html) documentation.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"required\": [\n \"test\"\n ],\n \"type\": \"object\"\n }\n ]\n },\n \"ConditionalPredicate\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"const\": \"binned\",\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"format\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Dict\"\n }\n ],\n \"description\": \"When used with the default `\\\"number\\\"` and `\\\"time\\\"` format type, the text formatting pattern for labels of guides (axes, legends, headers) and text marks.\\n\\n- If the format type is `\\\"number\\\"` (e.g., for quantitative fields), this is D3's [number format pattern](https://github.com/d3/d3-format#locale_format).\\n- If the format type is `\\\"time\\\"` (e.g., for temporal fields), this is D3's [time format pattern](https://github.com/d3/d3-time-format#locale_format).\\n\\nSee the [format documentation](https://vega.github.io/vega-lite/docs/format.html) for more examples.\\n\\nWhen used with a [custom `formatType`](https://vega.github.io/vega-lite/docs/config.html#custom-format-type), this value will be passed as `format` alongside `datum.value` to the registered function.\\n\\n__Default value:__ Derived from [numberFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for number format and from [timeFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for time format.\"\n },\n \"formatType\": {\n \"description\": \"The format type for labels. One of `\\\"number\\\"`, `\\\"time\\\"`, or a [registered custom format type](https://vega.github.io/vega-lite/docs/config.html#custom-format-type).\\n\\n__Default value:__\\n- `\\\"time\\\"` for temporal fields and ordinal and nominal fields with `timeUnit`.\\n- `\\\"number\\\"` for quantitative fields as well as ordinal and nominal fields without `timeUnit`.\",\n \"type\": \"string\"\n },\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"required\": [\n \"test\"\n ],\n \"type\": \"object\"\n },\n \"ConditionalPredicate>\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"test\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"ConditionalPredicate>\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"test\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"ConditionalPredicate>\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"test\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"ConditionalPredicate>\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"test\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"ConditionalPredicate>\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"test\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"ConditionalPredicate>\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"test\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"ConditionalPredicate>\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"test\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"Predicate for triggering the condition\"\n },\n \"value\": {\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\",\n \"type\": \"number\"\n }\n },\n \"required\": [\n \"test\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"Config\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"arc\": {\n \"$ref\": \"#/definitions/RectConfig\",\n \"description\": \"Arc-specific Config\"\n },\n \"area\": {\n \"$ref\": \"#/definitions/AreaConfig\",\n \"description\": \"Area-Specific Config\"\n },\n \"aria\": {\n \"description\": \"A boolean flag indicating if ARIA default attributes should be included for marks and guides (SVG output only). If false, the `\\\"aria-hidden\\\"` attribute will be set for all guides, removing them from the ARIA accessibility tree and Vega-Lite will not generate default descriptions for marks.\\n\\n__Default value:__ `true`.\",\n \"type\": \"boolean\"\n },\n \"autosize\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/AutosizeType\"\n },\n {\n \"$ref\": \"#/definitions/AutoSizeParams\"\n }\n ],\n \"description\": \"How the visualization size should be determined. If a string, should be one of `\\\"pad\\\"`, `\\\"fit\\\"` or `\\\"none\\\"`. Object values can additionally specify parameters for content sizing and automatic resizing.\\n\\n__Default value__: `pad`\"\n },\n \"axis\": {\n \"$ref\": \"#/definitions/AxisConfig\",\n \"description\": \"Axis configuration, which determines default properties for all `x` and `y` [axes](https://vega.github.io/vega-lite/docs/axis.html). For a full list of axis configuration options, please see the [corresponding section of the axis documentation](https://vega.github.io/vega-lite/docs/axis.html#config).\"\n },\n \"axisBand\": {\n \"$ref\": \"#/definitions/AxisConfig\",\n \"description\": \"Config for axes with \\\"band\\\" scales.\"\n },\n \"axisBottom\": {\n \"$ref\": \"#/definitions/AxisConfig\",\n \"description\": \"Config for x-axis along the bottom edge of the chart.\"\n },\n \"axisDiscrete\": {\n \"$ref\": \"#/definitions/AxisConfig\",\n \"description\": \"Config for axes with \\\"point\\\" or \\\"band\\\" scales.\"\n },\n \"axisLeft\": {\n \"$ref\": \"#/definitions/AxisConfig\",\n \"description\": \"Config for y-axis along the left edge of the chart.\"\n },\n \"axisPoint\": {\n \"$ref\": \"#/definitions/AxisConfig\",\n \"description\": \"Config for axes with \\\"point\\\" scales.\"\n },\n \"axisQuantitative\": {\n \"$ref\": \"#/definitions/AxisConfig\",\n \"description\": \"Config for quantitative axes.\"\n },\n \"axisRight\": {\n \"$ref\": \"#/definitions/AxisConfig\",\n \"description\": \"Config for y-axis along the right edge of the chart.\"\n },\n \"axisTemporal\": {\n \"$ref\": \"#/definitions/AxisConfig\",\n \"description\": \"Config for temporal axes.\"\n },\n \"axisTop\": {\n \"$ref\": \"#/definitions/AxisConfig\",\n \"description\": \"Config for x-axis along the top edge of the chart.\"\n },\n \"axisX\": {\n \"$ref\": \"#/definitions/AxisConfig\",\n \"description\": \"X-axis specific config.\"\n },\n \"axisXBand\": {\n \"$ref\": \"#/definitions/AxisConfig\",\n \"description\": \"Config for x-axes with \\\"band\\\" scales.\"\n },\n \"axisXDiscrete\": {\n \"$ref\": \"#/definitions/AxisConfig\",\n \"description\": \"Config for x-axes with \\\"point\\\" or \\\"band\\\" scales.\"\n },\n \"axisXPoint\": {\n \"$ref\": \"#/definitions/AxisConfig\",\n \"description\": \"Config for x-axes with \\\"point\\\" scales.\"\n },\n \"axisXQuantitative\": {\n \"$ref\": \"#/definitions/AxisConfig\",\n \"description\": \"Config for x-quantitative axes.\"\n },\n \"axisXTemporal\": {\n \"$ref\": \"#/definitions/AxisConfig\",\n \"description\": \"Config for x-temporal axes.\"\n },\n \"axisY\": {\n \"$ref\": \"#/definitions/AxisConfig\",\n \"description\": \"Y-axis specific config.\"\n },\n \"axisYBand\": {\n \"$ref\": \"#/definitions/AxisConfig\",\n \"description\": \"Config for y-axes with \\\"band\\\" scales.\"\n },\n \"axisYDiscrete\": {\n \"$ref\": \"#/definitions/AxisConfig\",\n \"description\": \"Config for y-axes with \\\"point\\\" or \\\"band\\\" scales.\"\n },\n \"axisYPoint\": {\n \"$ref\": \"#/definitions/AxisConfig\",\n \"description\": \"Config for y-axes with \\\"point\\\" scales.\"\n },\n \"axisYQuantitative\": {\n \"$ref\": \"#/definitions/AxisConfig\",\n \"description\": \"Config for y-quantitative axes.\"\n },\n \"axisYTemporal\": {\n \"$ref\": \"#/definitions/AxisConfig\",\n \"description\": \"Config for y-temporal axes.\"\n },\n \"background\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"CSS color property to use as the background of the entire view.\\n\\n__Default value:__ `\\\"white\\\"`\"\n },\n \"bar\": {\n \"$ref\": \"#/definitions/BarConfig\",\n \"description\": \"Bar-Specific Config\"\n },\n \"boxplot\": {\n \"$ref\": \"#/definitions/BoxPlotConfig\",\n \"description\": \"Box Config\"\n },\n \"circle\": {\n \"$ref\": \"#/definitions/MarkConfig\",\n \"description\": \"Circle-Specific Config\"\n },\n \"concat\": {\n \"$ref\": \"#/definitions/CompositionConfig\",\n \"description\": \"Default configuration for all concatenation and repeat view composition operators (`concat`, `hconcat`, `vconcat`, and `repeat`)\"\n },\n \"countTitle\": {\n \"description\": \"Default axis and legend title for count fields.\\n\\n__Default value:__ `'Count of Records`.\",\n \"type\": \"string\"\n },\n \"customFormatTypes\": {\n \"description\": \"Allow the `formatType` property for text marks and guides to accept a custom formatter function [registered as a Vega expression](https://vega.github.io/vega-lite/usage/compile.html#format-type).\",\n \"type\": \"boolean\"\n },\n \"errorband\": {\n \"$ref\": \"#/definitions/ErrorBandConfig\",\n \"description\": \"ErrorBand Config\"\n },\n \"errorbar\": {\n \"$ref\": \"#/definitions/ErrorBarConfig\",\n \"description\": \"ErrorBar Config\"\n },\n \"facet\": {\n \"$ref\": \"#/definitions/CompositionConfig\",\n \"description\": \"Default configuration for the `facet` view composition operator\"\n },\n \"fieldTitle\": {\n \"description\": \"Defines how Vega-Lite generates title for fields. There are three possible styles:\\n- `\\\"verbal\\\"` (Default) - displays function in a verbal style (e.g., \\\"Sum of field\\\", \\\"Year-month of date\\\", \\\"field (binned)\\\").\\n- `\\\"function\\\"` - displays function using parentheses and capitalized texts (e.g., \\\"SUM(field)\\\", \\\"YEARMONTH(date)\\\", \\\"BIN(field)\\\").\\n- `\\\"plain\\\"` - displays only the field name without functions (e.g., \\\"field\\\", \\\"date\\\", \\\"field\\\").\",\n \"enum\": [\n \"verbal\",\n \"functional\",\n \"plain\"\n ],\n \"type\": \"string\"\n },\n \"font\": {\n \"description\": \"Default font for all text marks, titles, and labels.\",\n \"type\": \"string\"\n },\n \"geoshape\": {\n \"$ref\": \"#/definitions/MarkConfig\",\n \"description\": \"Geoshape-Specific Config\"\n },\n \"header\": {\n \"$ref\": \"#/definitions/HeaderConfig\",\n \"description\": \"Header configuration, which determines default properties for all [headers](https://vega.github.io/vega-lite/docs/header.html).\\n\\nFor a full list of header configuration options, please see the [corresponding section of in the header documentation](https://vega.github.io/vega-lite/docs/header.html#config).\"\n },\n \"headerColumn\": {\n \"$ref\": \"#/definitions/HeaderConfig\",\n \"description\": \"Header configuration, which determines default properties for column [headers](https://vega.github.io/vega-lite/docs/header.html).\\n\\nFor a full list of header configuration options, please see the [corresponding section of in the header documentation](https://vega.github.io/vega-lite/docs/header.html#config).\"\n },\n \"headerFacet\": {\n \"$ref\": \"#/definitions/HeaderConfig\",\n \"description\": \"Header configuration, which determines default properties for non-row/column facet [headers](https://vega.github.io/vega-lite/docs/header.html).\\n\\nFor a full list of header configuration options, please see the [corresponding section of in the header documentation](https://vega.github.io/vega-lite/docs/header.html#config).\"\n },\n \"headerRow\": {\n \"$ref\": \"#/definitions/HeaderConfig\",\n \"description\": \"Header configuration, which determines default properties for row [headers](https://vega.github.io/vega-lite/docs/header.html).\\n\\nFor a full list of header configuration options, please see the [corresponding section of in the header documentation](https://vega.github.io/vega-lite/docs/header.html#config).\"\n },\n \"image\": {\n \"$ref\": \"#/definitions/RectConfig\",\n \"description\": \"Image-specific Config\"\n },\n \"legend\": {\n \"$ref\": \"#/definitions/LegendConfig\",\n \"description\": \"Legend configuration, which determines default properties for all [legends](https://vega.github.io/vega-lite/docs/legend.html). For a full list of legend configuration options, please see the [corresponding section of in the legend documentation](https://vega.github.io/vega-lite/docs/legend.html#config).\"\n },\n \"line\": {\n \"$ref\": \"#/definitions/LineConfig\",\n \"description\": \"Line-Specific Config\"\n },\n \"lineBreak\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A delimiter, such as a newline character, upon which to break text strings into multiple lines. This property provides a global default for text marks, which is overridden by mark or style config settings, and by the lineBreak mark encoding channel. If signal-valued, either string or regular expression (regexp) values are valid.\"\n },\n \"locale\": {\n \"$ref\": \"#/definitions/Locale\",\n \"description\": \"Locale definitions for string parsing and formatting of number and date values. The locale object should contain `number` and/or `time` properties with [locale definitions](https://vega.github.io/vega/docs/api/locale/). Locale definitions provided in the config block may be overridden by the View constructor locale option.\"\n },\n \"mark\": {\n \"$ref\": \"#/definitions/MarkConfig\",\n \"description\": \"Mark Config\"\n },\n \"normalizedNumberFormat\": {\n \"description\": \"If normalizedNumberFormatType is not specified, D3 number format for axis labels, text marks, and tooltips of normalized stacked fields (fields with `stack: \\\"normalize\\\"`). For example `\\\"s\\\"` for SI units. Use [D3's number format pattern](https://github.com/d3/d3-format#locale_format).\\n\\nIf `config.normalizedNumberFormatType` is specified and `config.customFormatTypes` is `true`, this value will be passed as `format` alongside `datum.value` to the `config.numberFormatType` function. __Default value:__ `%`\",\n \"type\": \"string\"\n },\n \"normalizedNumberFormatType\": {\n \"description\": \"[Custom format type](https://vega.github.io/vega-lite/docs/config.html#custom-format-type) for `config.normalizedNumberFormat`.\\n\\n__Default value:__ `undefined` -- This is equilvalent to call D3-format, which is exposed as [`format` in Vega-Expression](https://vega.github.io/vega/docs/expressions/#format). __Note:__ You must also set `customFormatTypes` to `true` to use this feature.\",\n \"type\": \"string\"\n },\n \"numberFormat\": {\n \"description\": \"If numberFormatType is not specified, D3 number format for guide labels, text marks, and tooltips of non-normalized fields (fields *without* `stack: \\\"normalize\\\"`). For example `\\\"s\\\"` for SI units. Use [D3's number format pattern](https://github.com/d3/d3-format#locale_format).\\n\\nIf `config.numberFormatType` is specified and `config.customFormatTypes` is `true`, this value will be passed as `format` alongside `datum.value` to the `config.numberFormatType` function.\",\n \"type\": \"string\"\n },\n \"numberFormatType\": {\n \"description\": \"[Custom format type](https://vega.github.io/vega-lite/docs/config.html#custom-format-type) for `config.numberFormat`.\\n\\n__Default value:__ `undefined` -- This is equilvalent to call D3-format, which is exposed as [`format` in Vega-Expression](https://vega.github.io/vega/docs/expressions/#format). __Note:__ You must also set `customFormatTypes` to `true` to use this feature.\",\n \"type\": \"string\"\n },\n \"padding\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Padding\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The default visualization padding, in pixels, from the edge of the visualization canvas to the data rectangle. If a number, specifies padding for all sides. If an object, the value should have the format `{\\\"left\\\": 5, \\\"top\\\": 5, \\\"right\\\": 5, \\\"bottom\\\": 5}` to specify padding for each side of the visualization.\\n\\n__Default value__: `5`\"\n },\n \"params\": {\n \"description\": \"Dynamic variables or selections that parameterize a visualization.\",\n \"items\": {\n \"$ref\": \"#/definitions/TopLevelParameter\"\n },\n \"type\": \"array\"\n },\n \"point\": {\n \"$ref\": \"#/definitions/MarkConfig\",\n \"description\": \"Point-Specific Config\"\n },\n \"projection\": {\n \"$ref\": \"#/definitions/ProjectionConfig\",\n \"description\": \"Projection configuration, which determines default properties for all [projections](https://vega.github.io/vega-lite/docs/projection.html). For a full list of projection configuration options, please see the [corresponding section of the projection documentation](https://vega.github.io/vega-lite/docs/projection.html#config).\"\n },\n \"range\": {\n \"$ref\": \"#/definitions/RangeConfig\",\n \"description\": \"An object hash that defines default range arrays or schemes for using with scales. For a full list of scale range configuration options, please see the [corresponding section of the scale documentation](https://vega.github.io/vega-lite/docs/scale.html#config).\"\n },\n \"rect\": {\n \"$ref\": \"#/definitions/RectConfig\",\n \"description\": \"Rect-Specific Config\"\n },\n \"rule\": {\n \"$ref\": \"#/definitions/MarkConfig\",\n \"description\": \"Rule-Specific Config\"\n },\n \"scale\": {\n \"$ref\": \"#/definitions/ScaleConfig\",\n \"description\": \"Scale configuration determines default properties for all [scales](https://vega.github.io/vega-lite/docs/scale.html). For a full list of scale configuration options, please see the [corresponding section of the scale documentation](https://vega.github.io/vega-lite/docs/scale.html#config).\"\n },\n \"selection\": {\n \"$ref\": \"#/definitions/SelectionConfig\",\n \"description\": \"An object hash for defining default properties for each type of selections.\"\n },\n \"square\": {\n \"$ref\": \"#/definitions/MarkConfig\",\n \"description\": \"Square-Specific Config\"\n },\n \"style\": {\n \"$ref\": \"#/definitions/StyleConfigIndex\",\n \"description\": \"An object hash that defines key-value mappings to determine default properties for marks with a given [style](https://vega.github.io/vega-lite/docs/mark.html#mark-def). The keys represent styles names; the values have to be valid [mark configuration objects](https://vega.github.io/vega-lite/docs/mark.html#config).\"\n },\n \"text\": {\n \"$ref\": \"#/definitions/MarkConfig\",\n \"description\": \"Text-Specific Config\"\n },\n \"tick\": {\n \"$ref\": \"#/definitions/TickConfig\",\n \"description\": \"Tick-Specific Config\"\n },\n \"timeFormat\": {\n \"description\": \"Default time format for raw time values (without time units) in text marks, legend labels and header labels.\\n\\n__Default value:__ `\\\"%b %d, %Y\\\"` __Note:__ Axes automatically determine the format for each label automatically so this config does not affect axes.\",\n \"type\": \"string\"\n },\n \"timeFormatType\": {\n \"description\": \"[Custom format type](https://vega.github.io/vega-lite/docs/config.html#custom-format-type) for `config.timeFormat`.\\n\\n__Default value:__ `undefined` -- This is equilvalent to call D3-time-format, which is exposed as [`timeFormat` in Vega-Expression](https://vega.github.io/vega/docs/expressions/#timeFormat). __Note:__ You must also set `customFormatTypes` to `true` and there must *not* be a `timeUnit` defined to use this feature.\",\n \"type\": \"string\"\n },\n \"title\": {\n \"$ref\": \"#/definitions/TitleConfig\",\n \"description\": \"Title configuration, which determines default properties for all [titles](https://vega.github.io/vega-lite/docs/title.html). For a full list of title configuration options, please see the [corresponding section of the title documentation](https://vega.github.io/vega-lite/docs/title.html#config).\"\n },\n \"tooltipFormat\": {\n \"$ref\": \"#/definitions/FormatConfig\",\n \"description\": \"Define [custom format configuration](https://vega.github.io/vega-lite/docs/config.html#format) for tooltips. If unspecified, default format config will be applied.\"\n },\n \"trail\": {\n \"$ref\": \"#/definitions/LineConfig\",\n \"description\": \"Trail-Specific Config\"\n },\n \"view\": {\n \"$ref\": \"#/definitions/ViewConfig\",\n \"description\": \"Default properties for [single view plots](https://vega.github.io/vega-lite/docs/spec.html#single).\"\n }\n },\n \"type\": \"object\"\n },\n \"CsvDataFormat\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"parse\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Parse\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"If set to `null`, disable type inference based on the spec and only use type inference based on the data. Alternatively, a parsing directive object can be provided for explicit data types. Each property of the object corresponds to a field name, and the value to the desired data type (one of `\\\"number\\\"`, `\\\"boolean\\\"`, `\\\"date\\\"`, or null (do not parse the field)). For example, `\\\"parse\\\": {\\\"modified_on\\\": \\\"date\\\"}` parses the `modified_on` field in each input record a Date value.\\n\\nFor `\\\"date\\\"`, we parse data based using JavaScript's [`Date.parse()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/parse). For Specific date formats can be provided (e.g., `{foo: \\\"date:'%m%d%Y'\\\"}`), using the [d3-time-format syntax](https://github.com/d3/d3-time-format#locale_format). UTC date format parsing is supported similarly (e.g., `{foo: \\\"utc:'%m%d%Y'\\\"}`). See more about [UTC time](https://vega.github.io/vega-lite/docs/timeunit.html#utc)\"\n },\n \"type\": {\n \"description\": \"Type of input data: `\\\"json\\\"`, `\\\"csv\\\"`, `\\\"tsv\\\"`, `\\\"dsv\\\"`.\\n\\n__Default value:__ The default format type is determined by the extension of the file URL. If no extension is detected, `\\\"json\\\"` will be used by default.\",\n \"enum\": [\n \"csv\",\n \"tsv\"\n ],\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"Cursor\": {\n \"enum\": [\n \"auto\",\n \"default\",\n \"none\",\n \"context-menu\",\n \"help\",\n \"pointer\",\n \"progress\",\n \"wait\",\n \"cell\",\n \"crosshair\",\n \"text\",\n \"vertical-text\",\n \"alias\",\n \"copy\",\n \"move\",\n \"no-drop\",\n \"not-allowed\",\n \"e-resize\",\n \"n-resize\",\n \"ne-resize\",\n \"nw-resize\",\n \"s-resize\",\n \"se-resize\",\n \"sw-resize\",\n \"w-resize\",\n \"ew-resize\",\n \"ns-resize\",\n \"nesw-resize\",\n \"nwse-resize\",\n \"col-resize\",\n \"row-resize\",\n \"all-scroll\",\n \"zoom-in\",\n \"zoom-out\",\n \"grab\",\n \"grabbing\"\n ],\n \"type\": \"string\"\n },\n \"Cyclical\": {\n \"enum\": [\n \"rainbow\",\n \"sinebow\"\n ],\n \"type\": \"string\"\n },\n \"Data\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/DataSource\"\n },\n {\n \"$ref\": \"#/definitions/Generator\"\n }\n ]\n },\n \"DataFormat\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/CsvDataFormat\"\n },\n {\n \"$ref\": \"#/definitions/DsvDataFormat\"\n },\n {\n \"$ref\": \"#/definitions/JsonDataFormat\"\n },\n {\n \"$ref\": \"#/definitions/TopoDataFormat\"\n }\n ]\n },\n \"DataSource\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/UrlData\"\n },\n {\n \"$ref\": \"#/definitions/InlineData\"\n },\n {\n \"$ref\": \"#/definitions/NamedData\"\n }\n ]\n },\n \"Datasets\": {\n \"$ref\": \"#/definitions/Dict\"\n },\n \"DateTime\": {\n \"additionalProperties\": false,\n \"description\": \"Object for defining datetime in Vega-Lite Filter. If both month and quarter are provided, month has higher precedence. `day` cannot be combined with other date. We accept string for month and day names.\",\n \"properties\": {\n \"date\": {\n \"description\": \"Integer value representing the date (day of the month) from 1-31.\",\n \"maximum\": 31,\n \"minimum\": 1,\n \"type\": \"number\"\n },\n \"day\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Day\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Value representing the day of a week. This can be one of: (1) integer value -- `1` represents Monday; (2) case-insensitive day name (e.g., `\\\"Monday\\\"`); (3) case-insensitive, 3-character short day name (e.g., `\\\"Mon\\\"`).\\n\\n**Warning:** A DateTime definition object with `day`** should not be combined with `year`, `quarter`, `month`, or `date`.\"\n },\n \"hours\": {\n \"description\": \"Integer value representing the hour of a day from 0-23.\",\n \"maximum\": 24,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"milliseconds\": {\n \"description\": \"Integer value representing the millisecond segment of time.\",\n \"maximum\": 1000,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"minutes\": {\n \"description\": \"Integer value representing the minute segment of time from 0-59.\",\n \"maximum\": 60,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"month\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Month\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"One of: (1) integer value representing the month from `1`-`12`. `1` represents January; (2) case-insensitive month name (e.g., `\\\"January\\\"`); (3) case-insensitive, 3-character short month name (e.g., `\\\"Jan\\\"`).\"\n },\n \"quarter\": {\n \"description\": \"Integer value representing the quarter of the year (from 1-4).\",\n \"maximum\": 4,\n \"minimum\": 1,\n \"type\": \"number\"\n },\n \"seconds\": {\n \"description\": \"Integer value representing the second segment (0-59) of a time value\",\n \"maximum\": 60,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"utc\": {\n \"description\": \"A boolean flag indicating if date time is in utc time. If false, the date time is in local time\",\n \"type\": \"boolean\"\n },\n \"year\": {\n \"description\": \"Integer value representing the year.\",\n \"type\": \"number\"\n }\n },\n \"type\": \"object\"\n },\n \"DatumDef\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"Day\": {\n \"maximum\": 7,\n \"minimum\": 1,\n \"type\": \"number\"\n },\n \"DensityTransform\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"as\": {\n \"description\": \"The output fields for the sample value and corresponding density estimate.\\n\\n__Default value:__ `[\\\"value\\\", \\\"density\\\"]`\",\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"maxItems\": 2,\n \"minItems\": 2,\n \"type\": \"array\"\n },\n \"bandwidth\": {\n \"description\": \"The bandwidth (standard deviation) of the Gaussian kernel. If unspecified or set to zero, the bandwidth value is automatically estimated from the input data using Scott’s rule.\",\n \"type\": \"number\"\n },\n \"counts\": {\n \"description\": \"A boolean flag indicating if the output values should be probability estimates (false) or smoothed counts (true).\\n\\n__Default value:__ `false`\",\n \"type\": \"boolean\"\n },\n \"cumulative\": {\n \"description\": \"A boolean flag indicating whether to produce density estimates (false) or cumulative density estimates (true).\\n\\n__Default value:__ `false`\",\n \"type\": \"boolean\"\n },\n \"density\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"The data field for which to perform density estimation.\"\n },\n \"extent\": {\n \"description\": \"A [min, max] domain from which to sample the distribution. If unspecified, the extent will be determined by the observed minimum and maximum values of the density value field.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"maxItems\": 2,\n \"minItems\": 2,\n \"type\": \"array\"\n },\n \"groupby\": {\n \"description\": \"The data fields to group by. If not specified, a single group containing all data objects will be used.\",\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"type\": \"array\"\n },\n \"maxsteps\": {\n \"description\": \"The maximum number of samples to take along the extent domain for plotting the density.\\n\\n__Default value:__ `200`\",\n \"type\": \"number\"\n },\n \"minsteps\": {\n \"description\": \"The minimum number of samples to take along the extent domain for plotting the density.\\n\\n__Default value:__ `25`\",\n \"type\": \"number\"\n },\n \"steps\": {\n \"description\": \"The exact number of samples to take along the extent domain for plotting the density. If specified, overrides both minsteps and maxsteps to set an exact number of uniform samples. Potentially useful in conjunction with a fixed extent to ensure consistent sample points for stacked densities.\",\n \"type\": \"number\"\n }\n },\n \"required\": [\n \"density\"\n ],\n \"type\": \"object\"\n },\n \"DerivedStream\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"between\": {\n \"items\": {\n \"$ref\": \"#/definitions/Stream\"\n },\n \"type\": \"array\"\n },\n \"consume\": {\n \"type\": \"boolean\"\n },\n \"debounce\": {\n \"type\": \"number\"\n },\n \"filter\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Expr\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/Expr\"\n },\n \"type\": \"array\"\n }\n ]\n },\n \"markname\": {\n \"type\": \"string\"\n },\n \"marktype\": {\n \"$ref\": \"#/definitions/MarkType\"\n },\n \"stream\": {\n \"$ref\": \"#/definitions/Stream\"\n },\n \"throttle\": {\n \"type\": \"number\"\n }\n },\n \"required\": [\n \"stream\"\n ],\n \"type\": \"object\"\n },\n \"Dict\": {\n \"additionalProperties\": {\n \"$ref\": \"#/definitions/InlineDataset\"\n },\n \"type\": \"object\"\n },\n \"Dict\": {\n \"additionalProperties\": {\n \"$ref\": \"#/definitions/SelectionInit\"\n },\n \"type\": \"object\"\n },\n \"Dict\": {\n \"additionalProperties\": {\n \"$ref\": \"#/definitions/SelectionInitInterval\"\n },\n \"type\": \"object\"\n },\n \"Dict\": {\n \"additionalProperties\": {},\n \"type\": \"object\"\n },\n \"Diverging\": {\n \"enum\": [\n \"blueorange\",\n \"blueorange-3\",\n \"blueorange-4\",\n \"blueorange-5\",\n \"blueorange-6\",\n \"blueorange-7\",\n \"blueorange-8\",\n \"blueorange-9\",\n \"blueorange-10\",\n \"blueorange-11\",\n \"brownbluegreen\",\n \"brownbluegreen-3\",\n \"brownbluegreen-4\",\n \"brownbluegreen-5\",\n \"brownbluegreen-6\",\n \"brownbluegreen-7\",\n \"brownbluegreen-8\",\n \"brownbluegreen-9\",\n \"brownbluegreen-10\",\n \"brownbluegreen-11\",\n \"purplegreen\",\n \"purplegreen-3\",\n \"purplegreen-4\",\n \"purplegreen-5\",\n \"purplegreen-6\",\n \"purplegreen-7\",\n \"purplegreen-8\",\n \"purplegreen-9\",\n \"purplegreen-10\",\n \"purplegreen-11\",\n \"pinkyellowgreen\",\n \"pinkyellowgreen-3\",\n \"pinkyellowgreen-4\",\n \"pinkyellowgreen-5\",\n \"pinkyellowgreen-6\",\n \"pinkyellowgreen-7\",\n \"pinkyellowgreen-8\",\n \"pinkyellowgreen-9\",\n \"pinkyellowgreen-10\",\n \"pinkyellowgreen-11\",\n \"purpleorange\",\n \"purpleorange-3\",\n \"purpleorange-4\",\n \"purpleorange-5\",\n \"purpleorange-6\",\n \"purpleorange-7\",\n \"purpleorange-8\",\n \"purpleorange-9\",\n \"purpleorange-10\",\n \"purpleorange-11\",\n \"redblue\",\n \"redblue-3\",\n \"redblue-4\",\n \"redblue-5\",\n \"redblue-6\",\n \"redblue-7\",\n \"redblue-8\",\n \"redblue-9\",\n \"redblue-10\",\n \"redblue-11\",\n \"redgrey\",\n \"redgrey-3\",\n \"redgrey-4\",\n \"redgrey-5\",\n \"redgrey-6\",\n \"redgrey-7\",\n \"redgrey-8\",\n \"redgrey-9\",\n \"redgrey-10\",\n \"redgrey-11\",\n \"redyellowblue\",\n \"redyellowblue-3\",\n \"redyellowblue-4\",\n \"redyellowblue-5\",\n \"redyellowblue-6\",\n \"redyellowblue-7\",\n \"redyellowblue-8\",\n \"redyellowblue-9\",\n \"redyellowblue-10\",\n \"redyellowblue-11\",\n \"redyellowgreen\",\n \"redyellowgreen-3\",\n \"redyellowgreen-4\",\n \"redyellowgreen-5\",\n \"redyellowgreen-6\",\n \"redyellowgreen-7\",\n \"redyellowgreen-8\",\n \"redyellowgreen-9\",\n \"redyellowgreen-10\",\n \"redyellowgreen-11\",\n \"spectral\",\n \"spectral-3\",\n \"spectral-4\",\n \"spectral-5\",\n \"spectral-6\",\n \"spectral-7\",\n \"spectral-8\",\n \"spectral-9\",\n \"spectral-10\",\n \"spectral-11\"\n ],\n \"type\": \"string\"\n },\n \"DomainUnionWith\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"unionWith\": {\n \"description\": \"Customized domain values to be union with the field's values or explicitly defined domain. Should be an array of valid scale domain values.\",\n \"items\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n }\n ]\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"unionWith\"\n ],\n \"type\": \"object\"\n },\n \"DsvDataFormat\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"delimiter\": {\n \"description\": \"The delimiter between records. The delimiter must be a single character (i.e., a single 16-bit code unit); so, ASCII delimiters are fine, but emoji delimiters are not.\",\n \"maxLength\": 1,\n \"minLength\": 1,\n \"type\": \"string\"\n },\n \"parse\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Parse\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"If set to `null`, disable type inference based on the spec and only use type inference based on the data. Alternatively, a parsing directive object can be provided for explicit data types. Each property of the object corresponds to a field name, and the value to the desired data type (one of `\\\"number\\\"`, `\\\"boolean\\\"`, `\\\"date\\\"`, or null (do not parse the field)). For example, `\\\"parse\\\": {\\\"modified_on\\\": \\\"date\\\"}` parses the `modified_on` field in each input record a Date value.\\n\\nFor `\\\"date\\\"`, we parse data based using JavaScript's [`Date.parse()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/parse). For Specific date formats can be provided (e.g., `{foo: \\\"date:'%m%d%Y'\\\"}`), using the [d3-time-format syntax](https://github.com/d3/d3-time-format#locale_format). UTC date format parsing is supported similarly (e.g., `{foo: \\\"utc:'%m%d%Y'\\\"}`). See more about [UTC time](https://vega.github.io/vega-lite/docs/timeunit.html#utc)\"\n },\n \"type\": {\n \"const\": \"dsv\",\n \"description\": \"Type of input data: `\\\"json\\\"`, `\\\"csv\\\"`, `\\\"tsv\\\"`, `\\\"dsv\\\"`.\\n\\n__Default value:__ The default format type is determined by the extension of the file URL. If no extension is detected, `\\\"json\\\"` will be used by default.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"delimiter\"\n ],\n \"type\": \"object\"\n },\n \"Element\": {\n \"type\": \"string\"\n },\n \"EncodingSortField\": {\n \"additionalProperties\": false,\n \"description\": \"A sort definition for sorting a discrete scale in an encoding field definition.\",\n \"properties\": {\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"The data [field](https://vega.github.io/vega-lite/docs/field.html) to sort by.\\n\\n__Default value:__ If unspecified, defaults to the field specified in the outer data reference.\"\n },\n \"op\": {\n \"$ref\": \"#/definitions/NonArgAggregateOp\",\n \"description\": \"An [aggregate operation](https://vega.github.io/vega-lite/docs/aggregate.html#ops) to perform on the field prior to sorting (e.g., `\\\"count\\\"`, `\\\"mean\\\"` and `\\\"median\\\"`). An aggregation is required when there are multiple values of the sort field for each encoded data field. The input data objects will be aggregated, grouped by the encoded data field.\\n\\nFor a full list of operations, please see the documentation for [aggregate](https://vega.github.io/vega-lite/docs/aggregate.html#ops).\\n\\n__Default value:__ `\\\"sum\\\"` for stacked plots. Otherwise, `\\\"min\\\"`.\"\n },\n \"order\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SortOrder\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"The sort order. One of `\\\"ascending\\\"` (default), `\\\"descending\\\"`, or `null` (no not sort).\"\n }\n },\n \"type\": \"object\"\n },\n \"ErrorBand\": {\n \"const\": \"errorband\",\n \"type\": \"string\"\n },\n \"ErrorBandConfig\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"band\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/AnyMarkConfig\"\n }\n ]\n },\n \"borders\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/AnyMarkConfig\"\n }\n ]\n },\n \"extent\": {\n \"$ref\": \"#/definitions/ErrorBarExtent\",\n \"description\": \"The extent of the band. Available options include:\\n- `\\\"ci\\\"`: Extend the band to the confidence interval of the mean.\\n- `\\\"stderr\\\"`: The size of band are set to the value of standard error, extending from the mean.\\n- `\\\"stdev\\\"`: The size of band are set to the value of standard deviation, extending from the mean.\\n- `\\\"iqr\\\"`: Extend the band to the q1 and q3.\\n\\n__Default value:__ `\\\"stderr\\\"`.\"\n },\n \"interpolate\": {\n \"$ref\": \"#/definitions/Interpolate\",\n \"description\": \"The line interpolation method for the error band. One of the following:\\n- `\\\"linear\\\"`: piecewise linear segments, as in a polyline.\\n- `\\\"linear-closed\\\"`: close the linear segments to form a polygon.\\n- `\\\"step\\\"`: a piecewise constant function (a step function) consisting of alternating horizontal and vertical lines. The y-value changes at the midpoint of each pair of adjacent x-values.\\n- `\\\"step-before\\\"`: a piecewise constant function (a step function) consisting of alternating horizontal and vertical lines. The y-value changes before the x-value.\\n- `\\\"step-after\\\"`: a piecewise constant function (a step function) consisting of alternating horizontal and vertical lines. The y-value changes after the x-value.\\n- `\\\"basis\\\"`: a B-spline, with control point duplication on the ends.\\n- `\\\"basis-open\\\"`: an open B-spline; may not intersect the start or end.\\n- `\\\"basis-closed\\\"`: a closed B-spline, as in a loop.\\n- `\\\"cardinal\\\"`: a Cardinal spline, with control point duplication on the ends.\\n- `\\\"cardinal-open\\\"`: an open Cardinal spline; may not intersect the start or end, but will intersect other control points.\\n- `\\\"cardinal-closed\\\"`: a closed Cardinal spline, as in a loop.\\n- `\\\"bundle\\\"`: equivalent to basis, except the tension parameter is used to straighten the spline.\\n- `\\\"monotone\\\"`: cubic interpolation that preserves monotonicity in y.\"\n },\n \"tension\": {\n \"description\": \"The tension parameter for the interpolation type of the error band.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n }\n },\n \"type\": \"object\"\n },\n \"ErrorBandDef\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"band\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/AnyMarkConfig\"\n }\n ]\n },\n \"borders\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/AnyMarkConfig\"\n }\n ]\n },\n \"clip\": {\n \"description\": \"Whether a composite mark be clipped to the enclosing group’s width and height.\",\n \"type\": \"boolean\"\n },\n \"color\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default color.\\n\\n__Default value:__ ■ `\\\"#4682b4\\\"`\\n\\n__Note:__\\n- This property cannot be used in a [style config](https://vega.github.io/vega-lite/docs/mark.html#style-config).\\n- The `fill` and `stroke` properties have higher precedence than `color` and will override `color`.\"\n },\n \"extent\": {\n \"$ref\": \"#/definitions/ErrorBarExtent\",\n \"description\": \"The extent of the band. Available options include:\\n- `\\\"ci\\\"`: Extend the band to the confidence interval of the mean.\\n- `\\\"stderr\\\"`: The size of band are set to the value of standard error, extending from the mean.\\n- `\\\"stdev\\\"`: The size of band are set to the value of standard deviation, extending from the mean.\\n- `\\\"iqr\\\"`: Extend the band to the q1 and q3.\\n\\n__Default value:__ `\\\"stderr\\\"`.\"\n },\n \"interpolate\": {\n \"$ref\": \"#/definitions/Interpolate\",\n \"description\": \"The line interpolation method for the error band. One of the following:\\n- `\\\"linear\\\"`: piecewise linear segments, as in a polyline.\\n- `\\\"linear-closed\\\"`: close the linear segments to form a polygon.\\n- `\\\"step\\\"`: a piecewise constant function (a step function) consisting of alternating horizontal and vertical lines. The y-value changes at the midpoint of each pair of adjacent x-values.\\n- `\\\"step-before\\\"`: a piecewise constant function (a step function) consisting of alternating horizontal and vertical lines. The y-value changes before the x-value.\\n- `\\\"step-after\\\"`: a piecewise constant function (a step function) consisting of alternating horizontal and vertical lines. The y-value changes after the x-value.\\n- `\\\"basis\\\"`: a B-spline, with control point duplication on the ends.\\n- `\\\"basis-open\\\"`: an open B-spline; may not intersect the start or end.\\n- `\\\"basis-closed\\\"`: a closed B-spline, as in a loop.\\n- `\\\"cardinal\\\"`: a Cardinal spline, with control point duplication on the ends.\\n- `\\\"cardinal-open\\\"`: an open Cardinal spline; may not intersect the start or end, but will intersect other control points.\\n- `\\\"cardinal-closed\\\"`: a closed Cardinal spline, as in a loop.\\n- `\\\"bundle\\\"`: equivalent to basis, except the tension parameter is used to straighten the spline.\\n- `\\\"monotone\\\"`: cubic interpolation that preserves monotonicity in y.\"\n },\n \"opacity\": {\n \"description\": \"The opacity (value between [0,1]) of the mark.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"orient\": {\n \"$ref\": \"#/definitions/Orientation\",\n \"description\": \"Orientation of the error band. This is normally automatically determined, but can be specified when the orientation is ambiguous and cannot be automatically determined.\"\n },\n \"tension\": {\n \"description\": \"The tension parameter for the interpolation type of the error band.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/ErrorBand\",\n \"description\": \"The mark type. This could a primitive mark type (one of `\\\"bar\\\"`, `\\\"circle\\\"`, `\\\"square\\\"`, `\\\"tick\\\"`, `\\\"line\\\"`, `\\\"area\\\"`, `\\\"point\\\"`, `\\\"geoshape\\\"`, `\\\"rule\\\"`, and `\\\"text\\\"`) or a composite mark type (`\\\"boxplot\\\"`, `\\\"errorband\\\"`, `\\\"errorbar\\\"`).\"\n }\n },\n \"required\": [\n \"type\"\n ],\n \"type\": \"object\"\n },\n \"ErrorBar\": {\n \"const\": \"errorbar\",\n \"type\": \"string\"\n },\n \"ErrorBarConfig\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"extent\": {\n \"$ref\": \"#/definitions/ErrorBarExtent\",\n \"description\": \"The extent of the rule. Available options include:\\n- `\\\"ci\\\"`: Extend the rule to the confidence interval of the mean.\\n- `\\\"stderr\\\"`: The size of rule are set to the value of standard error, extending from the mean.\\n- `\\\"stdev\\\"`: The size of rule are set to the value of standard deviation, extending from the mean.\\n- `\\\"iqr\\\"`: Extend the rule to the q1 and q3.\\n\\n__Default value:__ `\\\"stderr\\\"`.\"\n },\n \"rule\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/AnyMarkConfig\"\n }\n ]\n },\n \"size\": {\n \"description\": \"Size of the ticks of an error bar\",\n \"type\": \"number\"\n },\n \"thickness\": {\n \"description\": \"Thickness of the ticks and the bar of an error bar\",\n \"type\": \"number\"\n },\n \"ticks\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/AnyMarkConfig\"\n }\n ]\n }\n },\n \"type\": \"object\"\n },\n \"ErrorBarDef\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"clip\": {\n \"description\": \"Whether a composite mark be clipped to the enclosing group’s width and height.\",\n \"type\": \"boolean\"\n },\n \"color\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default color.\\n\\n__Default value:__ ■ `\\\"#4682b4\\\"`\\n\\n__Note:__\\n- This property cannot be used in a [style config](https://vega.github.io/vega-lite/docs/mark.html#style-config).\\n- The `fill` and `stroke` properties have higher precedence than `color` and will override `color`.\"\n },\n \"extent\": {\n \"$ref\": \"#/definitions/ErrorBarExtent\",\n \"description\": \"The extent of the rule. Available options include:\\n- `\\\"ci\\\"`: Extend the rule to the confidence interval of the mean.\\n- `\\\"stderr\\\"`: The size of rule are set to the value of standard error, extending from the mean.\\n- `\\\"stdev\\\"`: The size of rule are set to the value of standard deviation, extending from the mean.\\n- `\\\"iqr\\\"`: Extend the rule to the q1 and q3.\\n\\n__Default value:__ `\\\"stderr\\\"`.\"\n },\n \"opacity\": {\n \"description\": \"The opacity (value between [0,1]) of the mark.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"orient\": {\n \"$ref\": \"#/definitions/Orientation\",\n \"description\": \"Orientation of the error bar. This is normally automatically determined, but can be specified when the orientation is ambiguous and cannot be automatically determined.\"\n },\n \"rule\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/AnyMarkConfig\"\n }\n ]\n },\n \"size\": {\n \"description\": \"Size of the ticks of an error bar\",\n \"type\": \"number\"\n },\n \"thickness\": {\n \"description\": \"Thickness of the ticks and the bar of an error bar\",\n \"type\": \"number\"\n },\n \"ticks\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/AnyMarkConfig\"\n }\n ]\n },\n \"type\": {\n \"$ref\": \"#/definitions/ErrorBar\",\n \"description\": \"The mark type. This could a primitive mark type (one of `\\\"bar\\\"`, `\\\"circle\\\"`, `\\\"square\\\"`, `\\\"tick\\\"`, `\\\"line\\\"`, `\\\"area\\\"`, `\\\"point\\\"`, `\\\"geoshape\\\"`, `\\\"rule\\\"`, and `\\\"text\\\"`) or a composite mark type (`\\\"boxplot\\\"`, `\\\"errorband\\\"`, `\\\"errorbar\\\"`).\"\n }\n },\n \"required\": [\n \"type\"\n ],\n \"type\": \"object\"\n },\n \"ErrorBarExtent\": {\n \"enum\": [\n \"ci\",\n \"iqr\",\n \"stderr\",\n \"stdev\"\n ],\n \"type\": \"string\"\n },\n \"EventStream\": {\n \"anyOf\": [\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"between\": {\n \"items\": {\n \"$ref\": \"#/definitions/Stream\"\n },\n \"type\": \"array\"\n },\n \"consume\": {\n \"type\": \"boolean\"\n },\n \"debounce\": {\n \"type\": \"number\"\n },\n \"filter\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Expr\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/Expr\"\n },\n \"type\": \"array\"\n }\n ]\n },\n \"markname\": {\n \"type\": \"string\"\n },\n \"marktype\": {\n \"$ref\": \"#/definitions/MarkType\"\n },\n \"source\": {\n \"enum\": [\n \"view\",\n \"scope\"\n ],\n \"type\": \"string\"\n },\n \"throttle\": {\n \"type\": \"number\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/EventType\"\n }\n },\n \"required\": [\n \"type\"\n ],\n \"type\": \"object\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"between\": {\n \"items\": {\n \"$ref\": \"#/definitions/Stream\"\n },\n \"type\": \"array\"\n },\n \"consume\": {\n \"type\": \"boolean\"\n },\n \"debounce\": {\n \"type\": \"number\"\n },\n \"filter\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Expr\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/Expr\"\n },\n \"type\": \"array\"\n }\n ]\n },\n \"markname\": {\n \"type\": \"string\"\n },\n \"marktype\": {\n \"$ref\": \"#/definitions/MarkType\"\n },\n \"source\": {\n \"const\": \"window\",\n \"type\": \"string\"\n },\n \"throttle\": {\n \"type\": \"number\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/WindowEventType\"\n }\n },\n \"required\": [\n \"source\",\n \"type\"\n ],\n \"type\": \"object\"\n }\n ]\n },\n \"EventType\": {\n \"enum\": [\n \"click\",\n \"dblclick\",\n \"dragenter\",\n \"dragleave\",\n \"dragover\",\n \"keydown\",\n \"keypress\",\n \"keyup\",\n \"mousedown\",\n \"mousemove\",\n \"mouseout\",\n \"mouseover\",\n \"mouseup\",\n \"mousewheel\",\n \"pointerdown\",\n \"pointermove\",\n \"pointerout\",\n \"pointerover\",\n \"pointerup\",\n \"timer\",\n \"touchend\",\n \"touchmove\",\n \"touchstart\",\n \"wheel\"\n ],\n \"type\": \"string\"\n },\n \"Expr\": {\n \"type\": \"string\"\n },\n \"ExprRef\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"expr\": {\n \"description\": \"Vega expression (which can refer to Vega-Lite parameters).\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"expr\"\n ],\n \"type\": \"object\"\n },\n \"ExtentTransform\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"extent\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"The field of which to get the extent.\"\n },\n \"param\": {\n \"$ref\": \"#/definitions/ParameterName\",\n \"description\": \"The output parameter produced by the extent transform.\"\n }\n },\n \"required\": [\n \"extent\",\n \"param\"\n ],\n \"type\": \"object\"\n },\n \"FacetEncodingFieldDef\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"align\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LayoutAlign\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"The alignment to apply to grid rows and columns. The supported string values are `\\\"all\\\"`, `\\\"each\\\"`, and `\\\"none\\\"`.\\n\\n- For `\\\"none\\\"`, a flow layout will be used, in which adjacent subviews are simply placed one after the other.\\n- For `\\\"each\\\"`, subviews will be aligned into a clean grid structure, but each row or column may be of variable size.\\n- For `\\\"all\\\"`, subviews will be aligned and each row or column will be sized identically based on the maximum observed size. String values for this property will be applied to both grid rows and columns.\\n\\nAlternatively, an object value of the form `{\\\"row\\\": string, \\\"column\\\": string}` can be used to supply different alignments for rows and columns.\\n\\n__Default value:__ `\\\"all\\\"`.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"bounds\": {\n \"description\": \"The bounds calculation method to use for determining the extent of a sub-plot. One of `full` (the default) or `flush`.\\n\\n- If set to `full`, the entire calculated bounds (including axes, title, and legend) will be used.\\n- If set to `flush`, only the specified width and height values for the sub-view will be used. The `flush` setting can be useful when attempting to place sub-plots without axes or legends into a uniform grid structure.\\n\\n__Default value:__ `\\\"full\\\"`\",\n \"enum\": [\n \"full\",\n \"flush\"\n ],\n \"type\": \"string\"\n },\n \"center\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"Boolean flag indicating if subviews should be centered relative to their respective rows or columns.\\n\\nAn object value of the form `{\\\"row\\\": boolean, \\\"column\\\": boolean}` can be used to supply different centering values for rows and columns.\\n\\n__Default value:__ `false`\"\n },\n \"columns\": {\n \"description\": \"The number of columns to include in the view composition layout.\\n\\n__Default value__: `undefined` -- An infinite number of columns (a single row) will be assumed. This is equivalent to `hconcat` (for `concat`) and to using the `column` channel (for `facet` and `repeat`).\\n\\n__Note__:\\n\\n1) This property is only for:\\n- the general (wrappable) `concat` operator (not `hconcat`/`vconcat`)\\n- the `facet` and `repeat` operator with one field/repetition definition (without row/column nesting)\\n\\n2) Setting the `columns` to `1` is equivalent to `vconcat` (for `concat`) and to using the `row` channel (for `facet` and `repeat`).\",\n \"type\": \"number\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"header\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Header\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of a facet's header.\"\n },\n \"sort\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SortArray\"\n },\n {\n \"$ref\": \"#/definitions/SortOrder\"\n },\n {\n \"$ref\": \"#/definitions/EncodingSortField\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` is not supported for `row` and `column`.\"\n },\n \"spacing\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"The spacing in pixels between sub-views of the composition operator. An object of the form `{\\\"row\\\": number, \\\"column\\\": number}` can be used to set different spacing values for rows and columns.\\n\\n__Default value__: Depends on `\\\"spacing\\\"` property of [the view composition configuration](https://vega.github.io/vega-lite/docs/config.html#view-config) (`20` by default)\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"FacetFieldDef\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"header\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Header\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of a facet's header.\"\n },\n \"sort\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SortArray\"\n },\n {\n \"$ref\": \"#/definitions/SortOrder\"\n },\n {\n \"$ref\": \"#/definitions/EncodingSortField\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` is not supported for `row` and `column`.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"FacetMapping\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"column\": {\n \"$ref\": \"#/definitions/FacetFieldDef\",\n \"description\": \"A field definition for the horizontal facet of trellis plots.\"\n },\n \"row\": {\n \"$ref\": \"#/definitions/FacetFieldDef\",\n \"description\": \"A field definition for the vertical facet of trellis plots.\"\n }\n },\n \"type\": \"object\"\n },\n \"FacetedEncoding\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"angle\": {\n \"$ref\": \"#/definitions/NumericMarkPropDef\",\n \"description\": \"Rotation angle of point and text marks.\"\n },\n \"color\": {\n \"$ref\": \"#/definitions/ColorDef\",\n \"description\": \"Color of the marks – either fill or stroke color based on the `filled` property of mark definition. By default, `color` represents fill color for `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"tick\\\"`, `\\\"text\\\"`, `\\\"trail\\\"`, `\\\"circle\\\"`, and `\\\"square\\\"` / stroke color for `\\\"line\\\"` and `\\\"point\\\"`.\\n\\n__Default value:__ If undefined, the default color depends on [mark config](https://vega.github.io/vega-lite/docs/config.html#mark-config)'s `color` property.\\n\\n_Note:_ 1) For fine-grained control over both fill and stroke colors of the marks, please use the `fill` and `stroke` channels. The `fill` or `stroke` encodings have higher precedence than `color`, thus may override the `color` encoding if conflicting encodings are specified. 2) See the scale documentation for more information about customizing [color scheme](https://vega.github.io/vega-lite/docs/scale.html#scheme).\"\n },\n \"column\": {\n \"$ref\": \"#/definitions/RowColumnEncodingFieldDef\",\n \"description\": \"A field definition for the horizontal facet of trellis plots.\"\n },\n \"description\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StringFieldDefWithCondition\"\n },\n {\n \"$ref\": \"#/definitions/StringValueDefWithCondition\"\n }\n ],\n \"description\": \"A text description of this mark for ARIA accessibility (SVG output only). For SVG output the `\\\"aria-label\\\"` attribute will be set to this description.\"\n },\n \"detail\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FieldDefWithoutScale\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/FieldDefWithoutScale\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"Additional levels of detail for grouping data in aggregate views and in line, trail, and area marks without mapping data to a specific visual channel.\"\n },\n \"facet\": {\n \"$ref\": \"#/definitions/FacetEncodingFieldDef\",\n \"description\": \"A field definition for the (flexible) facet of trellis plots.\\n\\nIf either `row` or `column` is specified, this channel will be ignored.\"\n },\n \"fill\": {\n \"$ref\": \"#/definitions/ColorDef\",\n \"description\": \"Fill color of the marks. __Default value:__ If undefined, the default color depends on [mark config](https://vega.github.io/vega-lite/docs/config.html#mark-config)'s `color` property.\\n\\n_Note:_ The `fill` encoding has higher precedence than `color`, thus may override the `color` encoding if conflicting encodings are specified.\"\n },\n \"fillOpacity\": {\n \"$ref\": \"#/definitions/NumericMarkPropDef\",\n \"description\": \"Fill opacity of the marks.\\n\\n__Default value:__ If undefined, the default opacity depends on [mark config](https://vega.github.io/vega-lite/docs/config.html#mark-config)'s `fillOpacity` property.\"\n },\n \"href\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StringFieldDefWithCondition\"\n },\n {\n \"$ref\": \"#/definitions/StringValueDefWithCondition\"\n }\n ],\n \"description\": \"A URL to load upon mouse click.\"\n },\n \"key\": {\n \"$ref\": \"#/definitions/FieldDefWithoutScale\",\n \"description\": \"A data field to use as a unique key for data binding. When a visualization’s data is updated, the key value will be used to match data elements to existing mark instances. Use a key channel to enable object constancy for transitions over dynamic data.\"\n },\n \"latitude\": {\n \"$ref\": \"#/definitions/LatLongDef\",\n \"description\": \"Latitude position of geographically projected marks.\"\n },\n \"latitude2\": {\n \"$ref\": \"#/definitions/Position2Def\",\n \"description\": \"Latitude-2 position for geographically projected ranged `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"rect\\\"`, and `\\\"rule\\\"`.\"\n },\n \"longitude\": {\n \"$ref\": \"#/definitions/LatLongDef\",\n \"description\": \"Longitude position of geographically projected marks.\"\n },\n \"longitude2\": {\n \"$ref\": \"#/definitions/Position2Def\",\n \"description\": \"Longitude-2 position for geographically projected ranged `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"rect\\\"`, and `\\\"rule\\\"`.\"\n },\n \"opacity\": {\n \"$ref\": \"#/definitions/NumericMarkPropDef\",\n \"description\": \"Opacity of the marks.\\n\\n__Default value:__ If undefined, the default opacity depends on [mark config](https://vega.github.io/vega-lite/docs/config.html#mark-config)'s `opacity` property.\"\n },\n \"order\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/OrderFieldDef\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/OrderFieldDef\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/OrderValueDef\"\n },\n {\n \"$ref\": \"#/definitions/OrderOnlyDef\"\n }\n ],\n \"description\": \"Order of the marks.\\n- For stacked marks, this `order` channel encodes [stack order](https://vega.github.io/vega-lite/docs/stack.html#order).\\n- For line and trail marks, this `order` channel encodes order of data points in the lines. This can be useful for creating [a connected scatterplot](https://vega.github.io/vega-lite/examples/connected_scatterplot.html). Setting `order` to `{\\\"value\\\": null}` makes the line marks use the original order in the data sources.\\n- Otherwise, this `order` channel encodes layer order of the marks.\\n\\n__Note__: In aggregate plots, `order` field should be `aggregate`d to avoid creating additional aggregation grouping.\"\n },\n \"radius\": {\n \"$ref\": \"#/definitions/PolarDef\",\n \"description\": \"The outer radius in pixels of arc marks.\"\n },\n \"radius2\": {\n \"$ref\": \"#/definitions/Position2Def\",\n \"description\": \"The inner radius in pixels of arc marks.\"\n },\n \"row\": {\n \"$ref\": \"#/definitions/RowColumnEncodingFieldDef\",\n \"description\": \"A field definition for the vertical facet of trellis plots.\"\n },\n \"shape\": {\n \"$ref\": \"#/definitions/ShapeDef\",\n \"description\": \"Shape of the mark.\\n\\n1. For `point` marks the supported values include: - plotting shapes: `\\\"circle\\\"`, `\\\"square\\\"`, `\\\"cross\\\"`, `\\\"diamond\\\"`, `\\\"triangle-up\\\"`, `\\\"triangle-down\\\"`, `\\\"triangle-right\\\"`, or `\\\"triangle-left\\\"`. - the line symbol `\\\"stroke\\\"` - centered directional shapes `\\\"arrow\\\"`, `\\\"wedge\\\"`, or `\\\"triangle\\\"` - a custom [SVG path string](https://developer.mozilla.org/en-US/docs/Web/SVG/Tutorial/Paths) (For correct sizing, custom shape paths should be defined within a square bounding box with coordinates ranging from -1 to 1 along both the x and y dimensions.)\\n\\n2. For `geoshape` marks it should be a field definition of the geojson data\\n\\n__Default value:__ If undefined, the default shape depends on [mark config](https://vega.github.io/vega-lite/docs/config.html#point-config)'s `shape` property. (`\\\"circle\\\"` if unset.)\"\n },\n \"size\": {\n \"$ref\": \"#/definitions/NumericMarkPropDef\",\n \"description\": \"Size of the mark.\\n- For `\\\"point\\\"`, `\\\"square\\\"` and `\\\"circle\\\"`, – the symbol size, or pixel area of the mark.\\n- For `\\\"bar\\\"` and `\\\"tick\\\"` – the bar and tick's size.\\n- For `\\\"text\\\"` – the text's font size.\\n- Size is unsupported for `\\\"line\\\"`, `\\\"area\\\"`, and `\\\"rect\\\"`. (Use `\\\"trail\\\"` instead of line with varying size)\"\n },\n \"stroke\": {\n \"$ref\": \"#/definitions/ColorDef\",\n \"description\": \"Stroke color of the marks. __Default value:__ If undefined, the default color depends on [mark config](https://vega.github.io/vega-lite/docs/config.html#mark-config)'s `color` property.\\n\\n_Note:_ The `stroke` encoding has higher precedence than `color`, thus may override the `color` encoding if conflicting encodings are specified.\"\n },\n \"strokeDash\": {\n \"$ref\": \"#/definitions/NumericArrayMarkPropDef\",\n \"description\": \"Stroke dash of the marks.\\n\\n__Default value:__ `[1,0]` (No dash).\"\n },\n \"strokeOpacity\": {\n \"$ref\": \"#/definitions/NumericMarkPropDef\",\n \"description\": \"Stroke opacity of the marks.\\n\\n__Default value:__ If undefined, the default opacity depends on [mark config](https://vega.github.io/vega-lite/docs/config.html#mark-config)'s `strokeOpacity` property.\"\n },\n \"strokeWidth\": {\n \"$ref\": \"#/definitions/NumericMarkPropDef\",\n \"description\": \"Stroke width of the marks.\\n\\n__Default value:__ If undefined, the default stroke width depends on [mark config](https://vega.github.io/vega-lite/docs/config.html#mark-config)'s `strokeWidth` property.\"\n },\n \"text\": {\n \"$ref\": \"#/definitions/TextDef\",\n \"description\": \"Text of the `text` mark.\"\n },\n \"theta\": {\n \"$ref\": \"#/definitions/PolarDef\",\n \"description\": \"- For arc marks, the arc length in radians if theta2 is not specified, otherwise the start arc angle. (A value of 0 indicates up or “north”, increasing values proceed clockwise.)\\n\\n- For text marks, polar coordinate angle in radians.\"\n },\n \"theta2\": {\n \"$ref\": \"#/definitions/Position2Def\",\n \"description\": \"The end angle of arc marks in radians. A value of 0 indicates up or “north”, increasing values proceed clockwise.\"\n },\n \"tooltip\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StringFieldDefWithCondition\"\n },\n {\n \"$ref\": \"#/definitions/StringValueDefWithCondition\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/StringFieldDef\"\n },\n \"type\": \"array\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"The tooltip text to show upon mouse hover. Specifying `tooltip` encoding overrides [the `tooltip` property in the mark definition](https://vega.github.io/vega-lite/docs/mark.html#mark-def).\\n\\nSee the [`tooltip`](https://vega.github.io/vega-lite/docs/tooltip.html) documentation for a detailed discussion about tooltip in Vega-Lite.\"\n },\n \"url\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StringFieldDefWithCondition\"\n },\n {\n \"$ref\": \"#/definitions/StringValueDefWithCondition\"\n }\n ],\n \"description\": \"The URL of an image mark.\"\n },\n \"x\": {\n \"$ref\": \"#/definitions/PositionDef\",\n \"description\": \"X coordinates of the marks, or width of horizontal `\\\"bar\\\"` and `\\\"area\\\"` without specified `x2` or `width`.\\n\\nThe `value` of this channel can be a number or a string `\\\"width\\\"` for the width of the plot.\"\n },\n \"x2\": {\n \"$ref\": \"#/definitions/Position2Def\",\n \"description\": \"X2 coordinates for ranged `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"rect\\\"`, and `\\\"rule\\\"`.\\n\\nThe `value` of this channel can be a number or a string `\\\"width\\\"` for the width of the plot.\"\n },\n \"xError\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SecondaryFieldDef\"\n },\n {\n \"$ref\": \"#/definitions/ValueDef\"\n }\n ],\n \"description\": \"Error value of x coordinates for error specified `\\\"errorbar\\\"` and `\\\"errorband\\\"`.\"\n },\n \"xError2\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SecondaryFieldDef\"\n },\n {\n \"$ref\": \"#/definitions/ValueDef\"\n }\n ],\n \"description\": \"Secondary error value of x coordinates for error specified `\\\"errorbar\\\"` and `\\\"errorband\\\"`.\"\n },\n \"xOffset\": {\n \"$ref\": \"#/definitions/OffsetDef\",\n \"description\": \"Offset of x-position of the marks\"\n },\n \"y\": {\n \"$ref\": \"#/definitions/PositionDef\",\n \"description\": \"Y coordinates of the marks, or height of vertical `\\\"bar\\\"` and `\\\"area\\\"` without specified `y2` or `height`.\\n\\nThe `value` of this channel can be a number or a string `\\\"height\\\"` for the height of the plot.\"\n },\n \"y2\": {\n \"$ref\": \"#/definitions/Position2Def\",\n \"description\": \"Y2 coordinates for ranged `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"rect\\\"`, and `\\\"rule\\\"`.\\n\\nThe `value` of this channel can be a number or a string `\\\"height\\\"` for the height of the plot.\"\n },\n \"yError\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SecondaryFieldDef\"\n },\n {\n \"$ref\": \"#/definitions/ValueDef\"\n }\n ],\n \"description\": \"Error value of y coordinates for error specified `\\\"errorbar\\\"` and `\\\"errorband\\\"`.\"\n },\n \"yError2\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SecondaryFieldDef\"\n },\n {\n \"$ref\": \"#/definitions/ValueDef\"\n }\n ],\n \"description\": \"Secondary error value of y coordinates for error specified `\\\"errorbar\\\"` and `\\\"errorband\\\"`.\"\n },\n \"yOffset\": {\n \"$ref\": \"#/definitions/OffsetDef\",\n \"description\": \"Offset of y-position of the marks\"\n }\n },\n \"type\": \"object\"\n },\n \"FacetedUnitSpec\": {\n \"additionalProperties\": false,\n \"description\": \"Unit spec that can have a composite mark and row or column channels (shorthand for a facet spec).\",\n \"properties\": {\n \"align\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LayoutAlign\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"The alignment to apply to grid rows and columns. The supported string values are `\\\"all\\\"`, `\\\"each\\\"`, and `\\\"none\\\"`.\\n\\n- For `\\\"none\\\"`, a flow layout will be used, in which adjacent subviews are simply placed one after the other.\\n- For `\\\"each\\\"`, subviews will be aligned into a clean grid structure, but each row or column may be of variable size.\\n- For `\\\"all\\\"`, subviews will be aligned and each row or column will be sized identically based on the maximum observed size. String values for this property will be applied to both grid rows and columns.\\n\\nAlternatively, an object value of the form `{\\\"row\\\": string, \\\"column\\\": string}` can be used to supply different alignments for rows and columns.\\n\\n__Default value:__ `\\\"all\\\"`.\"\n },\n \"bounds\": {\n \"description\": \"The bounds calculation method to use for determining the extent of a sub-plot. One of `full` (the default) or `flush`.\\n\\n- If set to `full`, the entire calculated bounds (including axes, title, and legend) will be used.\\n- If set to `flush`, only the specified width and height values for the sub-view will be used. The `flush` setting can be useful when attempting to place sub-plots without axes or legends into a uniform grid structure.\\n\\n__Default value:__ `\\\"full\\\"`\",\n \"enum\": [\n \"full\",\n \"flush\"\n ],\n \"type\": \"string\"\n },\n \"center\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"Boolean flag indicating if subviews should be centered relative to their respective rows or columns.\\n\\nAn object value of the form `{\\\"row\\\": boolean, \\\"column\\\": boolean}` can be used to supply different centering values for rows and columns.\\n\\n__Default value:__ `false`\"\n },\n \"data\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Data\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object describing the data source. Set to `null` to ignore the parent's data source. If no data is set, it is derived from the parent.\"\n },\n \"description\": {\n \"description\": \"Description of this mark for commenting purpose.\",\n \"type\": \"string\"\n },\n \"encoding\": {\n \"$ref\": \"#/definitions/FacetedEncoding\",\n \"description\": \"A key-value mapping between encoding channels and definition of fields.\"\n },\n \"height\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"container\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Step\"\n }\n ],\n \"description\": \"The height of a visualization.\\n\\n- For a plot with a continuous y-field, height should be a number.\\n- For a plot with either a discrete y-field or no y-field, height can be either a number indicating a fixed height or an object in the form of `{step: number}` defining the height per discrete step. (No y-field is equivalent to having one discrete step.)\\n- To enable responsive sizing on height, it should be set to `\\\"container\\\"`.\\n\\n__Default value:__ Based on `config.view.continuousHeight` for a plot with a continuous y-field and `config.view.discreteHeight` otherwise.\\n\\n__Note:__ For plots with [`row` and `column` channels](https://vega.github.io/vega-lite/docs/encoding.html#facet), this represents the height of a single view and the `\\\"container\\\"` option cannot be used.\\n\\n__See also:__ [`height`](https://vega.github.io/vega-lite/docs/size.html) documentation.\"\n },\n \"mark\": {\n \"$ref\": \"#/definitions/AnyMark\",\n \"description\": \"A string describing the mark type (one of `\\\"bar\\\"`, `\\\"circle\\\"`, `\\\"square\\\"`, `\\\"tick\\\"`, `\\\"line\\\"`, `\\\"area\\\"`, `\\\"point\\\"`, `\\\"rule\\\"`, `\\\"geoshape\\\"`, and `\\\"text\\\"`) or a [mark definition object](https://vega.github.io/vega-lite/docs/mark.html#mark-def).\"\n },\n \"name\": {\n \"description\": \"Name of the visualization for later reference.\",\n \"type\": \"string\"\n },\n \"params\": {\n \"description\": \"An array of parameters that may either be simple variables, or more complex selections that map user input to data queries.\",\n \"items\": {\n \"$ref\": \"#/definitions/SelectionParameter\"\n },\n \"type\": \"array\"\n },\n \"projection\": {\n \"$ref\": \"#/definitions/Projection\",\n \"description\": \"An object defining properties of geographic projection, which will be applied to `shape` path for `\\\"geoshape\\\"` marks and to `latitude` and `\\\"longitude\\\"` channels for other marks.\"\n },\n \"resolve\": {\n \"$ref\": \"#/definitions/Resolve\",\n \"description\": \"Scale, axis, and legend resolutions for view composition specifications.\"\n },\n \"spacing\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"The spacing in pixels between sub-views of the composition operator. An object of the form `{\\\"row\\\": number, \\\"column\\\": number}` can be used to set different spacing values for rows and columns.\\n\\n__Default value__: Depends on `\\\"spacing\\\"` property of [the view composition configuration](https://vega.github.io/vega-lite/docs/config.html#view-config) (`20` by default)\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"$ref\": \"#/definitions/TitleParams\"\n }\n ],\n \"description\": \"Title for the plot.\"\n },\n \"transform\": {\n \"description\": \"An array of data transformations such as filter and new field calculation.\",\n \"items\": {\n \"$ref\": \"#/definitions/Transform\"\n },\n \"type\": \"array\"\n },\n \"view\": {\n \"$ref\": \"#/definitions/ViewBackground\",\n \"description\": \"An object defining the view background's fill and stroke.\\n\\n__Default value:__ none (transparent)\"\n },\n \"width\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"container\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Step\"\n }\n ],\n \"description\": \"The width of a visualization.\\n\\n- For a plot with a continuous x-field, width should be a number.\\n- For a plot with either a discrete x-field or no x-field, width can be either a number indicating a fixed width or an object in the form of `{step: number}` defining the width per discrete step. (No x-field is equivalent to having one discrete step.)\\n- To enable responsive sizing on width, it should be set to `\\\"container\\\"`.\\n\\n__Default value:__ Based on `config.view.continuousWidth` for a plot with a continuous x-field and `config.view.discreteWidth` otherwise.\\n\\n__Note:__ For plots with [`row` and `column` channels](https://vega.github.io/vega-lite/docs/encoding.html#facet), this represents the width of a single view and the `\\\"container\\\"` option cannot be used.\\n\\n__See also:__ [`width`](https://vega.github.io/vega-lite/docs/size.html) documentation.\"\n }\n },\n \"required\": [\n \"mark\"\n ],\n \"type\": \"object\"\n },\n \"Feature\": {\n \"additionalProperties\": false,\n \"description\": \"A feature object which contains a geometry and associated properties. https://tools.ietf.org/html/rfc7946#section-3.2\",\n \"properties\": {\n \"bbox\": {\n \"$ref\": \"#/definitions/BBox\",\n \"description\": \"Bounding box of the coordinate range of the object's Geometries, Features, or Feature Collections. https://tools.ietf.org/html/rfc7946#section-5\"\n },\n \"geometry\": {\n \"$ref\": \"#/definitions/Geometry\",\n \"description\": \"The feature's geometry\"\n },\n \"id\": {\n \"description\": \"A value that uniquely identifies this feature in a https://tools.ietf.org/html/rfc7946#section-3.2.\",\n \"type\": [\n \"string\",\n \"number\"\n ]\n },\n \"properties\": {\n \"$ref\": \"#/definitions/GeoJsonProperties\",\n \"description\": \"Properties associated with this feature.\"\n },\n \"type\": {\n \"const\": \"Feature\",\n \"description\": \"Specifies the type of GeoJSON object.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"geometry\",\n \"properties\",\n \"type\"\n ],\n \"type\": \"object\"\n },\n \"Feature\": {\n \"additionalProperties\": false,\n \"description\": \"A feature object which contains a geometry and associated properties. https://tools.ietf.org/html/rfc7946#section-3.2\",\n \"properties\": {\n \"bbox\": {\n \"$ref\": \"#/definitions/BBox\",\n \"description\": \"Bounding box of the coordinate range of the object's Geometries, Features, or Feature Collections. https://tools.ietf.org/html/rfc7946#section-5\"\n },\n \"geometry\": {\n \"$ref\": \"#/definitions/Geometry\",\n \"description\": \"The feature's geometry\"\n },\n \"id\": {\n \"description\": \"A value that uniquely identifies this feature in a https://tools.ietf.org/html/rfc7946#section-3.2.\",\n \"type\": [\n \"string\",\n \"number\"\n ]\n },\n \"properties\": {\n \"$ref\": \"#/definitions/GeoJsonProperties\",\n \"description\": \"Properties associated with this feature.\"\n },\n \"type\": {\n \"const\": \"Feature\",\n \"description\": \"Specifies the type of GeoJSON object.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"geometry\",\n \"properties\",\n \"type\"\n ],\n \"type\": \"object\"\n },\n \"FeatureCollection\": {\n \"additionalProperties\": false,\n \"description\": \"A collection of feature objects. https://tools.ietf.org/html/rfc7946#section-3.3\",\n \"properties\": {\n \"bbox\": {\n \"$ref\": \"#/definitions/BBox\",\n \"description\": \"Bounding box of the coordinate range of the object's Geometries, Features, or Feature Collections. https://tools.ietf.org/html/rfc7946#section-5\"\n },\n \"features\": {\n \"items\": {\n \"$ref\": \"#/definitions/Feature\"\n },\n \"type\": \"array\"\n },\n \"type\": {\n \"const\": \"FeatureCollection\",\n \"description\": \"Specifies the type of GeoJSON object.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"features\",\n \"type\"\n ],\n \"type\": \"object\"\n },\n \"Field\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FieldName\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ]\n },\n \"FieldDefWithoutScale\": {\n \"$ref\": \"#/definitions/TypedFieldDef\",\n \"description\": \"Field Def without scale (and without bin: \\\"binned\\\" support).\"\n },\n \"FieldEqualPredicate\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"equal\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"number\"\n },\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The value that the field should be equal to.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"Field to be tested.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit for the field to be tested.\"\n }\n },\n \"required\": [\n \"equal\",\n \"field\"\n ],\n \"type\": \"object\"\n },\n \"FieldGTEPredicate\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"field\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"Field to be tested.\"\n },\n \"gte\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The value that the field should be greater than or equals to.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit for the field to be tested.\"\n }\n },\n \"required\": [\n \"field\",\n \"gte\"\n ],\n \"type\": \"object\"\n },\n \"FieldGTPredicate\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"field\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"Field to be tested.\"\n },\n \"gt\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The value that the field should be greater than.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit for the field to be tested.\"\n }\n },\n \"required\": [\n \"field\",\n \"gt\"\n ],\n \"type\": \"object\"\n },\n \"FieldLTEPredicate\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"field\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"Field to be tested.\"\n },\n \"lte\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The value that the field should be less than or equals to.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit for the field to be tested.\"\n }\n },\n \"required\": [\n \"field\",\n \"lte\"\n ],\n \"type\": \"object\"\n },\n \"FieldLTPredicate\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"field\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"Field to be tested.\"\n },\n \"lt\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The value that the field should be less than.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit for the field to be tested.\"\n }\n },\n \"required\": [\n \"field\",\n \"lt\"\n ],\n \"type\": \"object\"\n },\n \"FieldName\": {\n \"type\": \"string\"\n },\n \"FieldOneOfPredicate\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"field\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"Field to be tested.\"\n },\n \"oneOf\": {\n \"anyOf\": [\n {\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n {\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"items\": {\n \"type\": \"boolean\"\n },\n \"type\": \"array\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/DateTime\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A set of values that the `field`'s value should be a member of, for a data item included in the filtered data.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit for the field to be tested.\"\n }\n },\n \"required\": [\n \"field\",\n \"oneOf\"\n ],\n \"type\": \"object\"\n },\n \"FieldOrDatumDefWithCondition\": {\n \"additionalProperties\": false,\n \"description\": \"A FieldDef with Condition { condition: {value: ...}, field: ..., ... }\",\n \"properties\": {\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Gradient|string|null|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Gradient|string|null|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"FieldOrDatumDefWithCondition\": {\n \"additionalProperties\": false,\n \"description\": \"A FieldDef with Condition { condition: {value: ...}, field: ..., ... }\",\n \"properties\": {\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|null|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|null|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"FieldOrDatumDefWithCondition\": {\n \"additionalProperties\": false,\n \"description\": \"A FieldDef with Condition { condition: {value: ...}, field: ..., ... }\",\n \"properties\": {\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"FieldOrDatumDefWithCondition\": {\n \"additionalProperties\": false,\n \"description\": \"A FieldDef with Condition { condition: {value: ...}, field: ..., ... }\",\n \"properties\": {\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number[]|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number[]|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"FieldOrDatumDefWithCondition\": {\n \"additionalProperties\": false,\n \"description\": \"A FieldDef with Condition { condition: {value: ...}, field: ..., ... }\",\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Gradient|string|null|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Gradient|string|null|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"legend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Legend\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the legend. If `null`, the legend for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [legend properties](https://vega.github.io/vega-lite/docs/legend.html) are applied.\\n\\n__See also:__ [`legend`](https://vega.github.io/vega-lite/docs/legend.html) documentation.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"FieldOrDatumDefWithCondition\": {\n \"additionalProperties\": false,\n \"description\": \"A FieldDef with Condition { condition: {value: ...}, field: ..., ... }\",\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"legend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Legend\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the legend. If `null`, the legend for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [legend properties](https://vega.github.io/vega-lite/docs/legend.html) are applied.\\n\\n__See also:__ [`legend`](https://vega.github.io/vega-lite/docs/legend.html) documentation.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"FieldOrDatumDefWithCondition\": {\n \"additionalProperties\": false,\n \"description\": \"A FieldDef with Condition { condition: {value: ...}, field: ..., ... }\",\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number[]|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number[]|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"legend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Legend\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the legend. If `null`, the legend for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [legend properties](https://vega.github.io/vega-lite/docs/legend.html) are applied.\\n\\n__See also:__ [`legend`](https://vega.github.io/vega-lite/docs/legend.html) documentation.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"FieldOrDatumDefWithCondition,(string|null)>\": {\n \"additionalProperties\": false,\n \"description\": \"A FieldDef with Condition { condition: {value: ...}, field: ..., ... }\",\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|null|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|null|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"legend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Legend\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the legend. If `null`, the legend for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [legend properties](https://vega.github.io/vega-lite/docs/legend.html) are applied.\\n\\n__See also:__ [`legend`](https://vega.github.io/vega-lite/docs/legend.html) documentation.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/TypeForShape\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"FieldOrDatumDefWithCondition\": {\n \"additionalProperties\": false,\n \"description\": \"A FieldDef with Condition { condition: {value: ...}, field: ..., ... }\",\n \"properties\": {\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Text|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Text|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"format\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Dict\"\n }\n ],\n \"description\": \"When used with the default `\\\"number\\\"` and `\\\"time\\\"` format type, the text formatting pattern for labels of guides (axes, legends, headers) and text marks.\\n\\n- If the format type is `\\\"number\\\"` (e.g., for quantitative fields), this is D3's [number format pattern](https://github.com/d3/d3-format#locale_format).\\n- If the format type is `\\\"time\\\"` (e.g., for temporal fields), this is D3's [time format pattern](https://github.com/d3/d3-time-format#locale_format).\\n\\nSee the [format documentation](https://vega.github.io/vega-lite/docs/format.html) for more examples.\\n\\nWhen used with a [custom `formatType`](https://vega.github.io/vega-lite/docs/config.html#custom-format-type), this value will be passed as `format` alongside `datum.value` to the registered function.\\n\\n__Default value:__ Derived from [numberFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for number format and from [timeFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for time format.\"\n },\n \"formatType\": {\n \"description\": \"The format type for labels. One of `\\\"number\\\"`, `\\\"time\\\"`, or a [registered custom format type](https://vega.github.io/vega-lite/docs/config.html#custom-format-type).\\n\\n__Default value:__\\n- `\\\"time\\\"` for temporal fields and ordinal and nominal fields with `timeUnit`.\\n- `\\\"number\\\"` for quantitative fields as well as ordinal and nominal fields without `timeUnit`.\",\n \"type\": \"string\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"FieldOrDatumDefWithCondition\": {\n \"additionalProperties\": false,\n \"description\": \"A FieldDef with Condition { condition: {value: ...}, field: ..., ... }\",\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"const\": \"binned\",\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Text|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Text|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"format\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Dict\"\n }\n ],\n \"description\": \"When used with the default `\\\"number\\\"` and `\\\"time\\\"` format type, the text formatting pattern for labels of guides (axes, legends, headers) and text marks.\\n\\n- If the format type is `\\\"number\\\"` (e.g., for quantitative fields), this is D3's [number format pattern](https://github.com/d3/d3-format#locale_format).\\n- If the format type is `\\\"time\\\"` (e.g., for temporal fields), this is D3's [time format pattern](https://github.com/d3/d3-time-format#locale_format).\\n\\nSee the [format documentation](https://vega.github.io/vega-lite/docs/format.html) for more examples.\\n\\nWhen used with a [custom `formatType`](https://vega.github.io/vega-lite/docs/config.html#custom-format-type), this value will be passed as `format` alongside `datum.value` to the registered function.\\n\\n__Default value:__ Derived from [numberFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for number format and from [timeFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for time format.\"\n },\n \"formatType\": {\n \"description\": \"The format type for labels. One of `\\\"number\\\"`, `\\\"time\\\"`, or a [registered custom format type](https://vega.github.io/vega-lite/docs/config.html#custom-format-type).\\n\\n__Default value:__\\n- `\\\"time\\\"` for temporal fields and ordinal and nominal fields with `timeUnit`.\\n- `\\\"number\\\"` for quantitative fields as well as ordinal and nominal fields without `timeUnit`.\",\n \"type\": \"string\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"FieldOrDatumDefWithCondition\": {\n \"additionalProperties\": false,\n \"description\": \"A FieldDef with Condition { condition: {value: ...}, field: ..., ... }\",\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"const\": \"binned\",\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"format\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Dict\"\n }\n ],\n \"description\": \"When used with the default `\\\"number\\\"` and `\\\"time\\\"` format type, the text formatting pattern for labels of guides (axes, legends, headers) and text marks.\\n\\n- If the format type is `\\\"number\\\"` (e.g., for quantitative fields), this is D3's [number format pattern](https://github.com/d3/d3-format#locale_format).\\n- If the format type is `\\\"time\\\"` (e.g., for temporal fields), this is D3's [time format pattern](https://github.com/d3/d3-time-format#locale_format).\\n\\nSee the [format documentation](https://vega.github.io/vega-lite/docs/format.html) for more examples.\\n\\nWhen used with a [custom `formatType`](https://vega.github.io/vega-lite/docs/config.html#custom-format-type), this value will be passed as `format` alongside `datum.value` to the registered function.\\n\\n__Default value:__ Derived from [numberFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for number format and from [timeFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for time format.\"\n },\n \"formatType\": {\n \"description\": \"The format type for labels. One of `\\\"number\\\"`, `\\\"time\\\"`, or a [registered custom format type](https://vega.github.io/vega-lite/docs/config.html#custom-format-type).\\n\\n__Default value:__\\n- `\\\"time\\\"` for temporal fields and ordinal and nominal fields with `timeUnit`.\\n- `\\\"number\\\"` for quantitative fields as well as ordinal and nominal fields without `timeUnit`.\",\n \"type\": \"string\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"FieldRange\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"field\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"field\"\n ],\n \"type\": \"object\"\n },\n \"FieldRangePredicate\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"field\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"Field to be tested.\"\n },\n \"range\": {\n \"anyOf\": [\n {\n \"items\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"An array of inclusive minimum and maximum values for a field value of a data item to be included in the filtered data.\",\n \"maxItems\": 2,\n \"minItems\": 2\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit for the field to be tested.\"\n }\n },\n \"required\": [\n \"field\",\n \"range\"\n ],\n \"type\": \"object\"\n },\n \"FieldValidPredicate\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"field\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"Field to be tested.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit for the field to be tested.\"\n },\n \"valid\": {\n \"description\": \"If set to true the field's value has to be valid, meaning both not `null` and not [`NaN`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/NaN).\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"field\",\n \"valid\"\n ],\n \"type\": \"object\"\n },\n \"FilterTransform\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"filter\": {\n \"$ref\": \"#/definitions/PredicateComposition\",\n \"description\": \"The `filter` property must be a predication definition, which can take one of the following forms:\\n\\n1) an [expression](https://vega.github.io/vega-lite/docs/types.html#expression) string, where `datum` can be used to refer to the current data object. For example, `{filter: \\\"datum.b2 > 60\\\"}` would make the output data includes only items that have values in the field `b2` over 60.\\n\\n2) one of the [field predicates](https://vega.github.io/vega-lite/docs/predicate.html#field-predicate): [`equal`](https://vega.github.io/vega-lite/docs/predicate.html#field-equal-predicate), [`lt`](https://vega.github.io/vega-lite/docs/predicate.html#lt-predicate), [`lte`](https://vega.github.io/vega-lite/docs/predicate.html#lte-predicate), [`gt`](https://vega.github.io/vega-lite/docs/predicate.html#gt-predicate), [`gte`](https://vega.github.io/vega-lite/docs/predicate.html#gte-predicate), [`range`](https://vega.github.io/vega-lite/docs/predicate.html#range-predicate), [`oneOf`](https://vega.github.io/vega-lite/docs/predicate.html#one-of-predicate), or [`valid`](https://vega.github.io/vega-lite/docs/predicate.html#valid-predicate),\\n\\n3) a [selection predicate](https://vega.github.io/vega-lite/docs/predicate.html#selection-predicate), which define the names of a selection that the data point should belong to (or a logical composition of selections).\\n\\n4) a [logical composition](https://vega.github.io/vega-lite/docs/predicate.html#composition) of (1), (2), or (3).\"\n }\n },\n \"required\": [\n \"filter\"\n ],\n \"type\": \"object\"\n },\n \"Fit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/GeoJsonFeature\"\n },\n {\n \"$ref\": \"#/definitions/GeoJsonFeatureCollection\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/GeoJsonFeature\"\n },\n \"type\": \"array\"\n }\n ]\n },\n \"FlattenTransform\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"as\": {\n \"description\": \"The output field names for extracted array values.\\n\\n__Default value:__ The field name of the corresponding array field\",\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"type\": \"array\"\n },\n \"flatten\": {\n \"description\": \"An array of one or more data fields containing arrays to flatten. If multiple fields are specified, their array values should have a parallel structure, ideally with the same length. If the lengths of parallel arrays do not match, the longest array will be used with `null` values added for missing entries.\",\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"flatten\"\n ],\n \"type\": \"object\"\n },\n \"FoldTransform\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"as\": {\n \"description\": \"The output field names for the key and value properties produced by the fold transform. __Default value:__ `[\\\"key\\\", \\\"value\\\"]`\",\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"maxItems\": 2,\n \"minItems\": 2,\n \"type\": \"array\"\n },\n \"fold\": {\n \"description\": \"An array of data fields indicating the properties to fold.\",\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"fold\"\n ],\n \"type\": \"object\"\n },\n \"FontStyle\": {\n \"type\": \"string\"\n },\n \"FontWeight\": {\n \"enum\": [\n \"normal\",\n \"bold\",\n \"lighter\",\n \"bolder\",\n 100,\n 200,\n 300,\n 400,\n 500,\n 600,\n 700,\n 800,\n 900\n ],\n \"type\": [\n \"string\",\n \"number\"\n ]\n },\n \"FormatConfig\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"normalizedNumberFormat\": {\n \"description\": \"If normalizedNumberFormatType is not specified, D3 number format for axis labels, text marks, and tooltips of normalized stacked fields (fields with `stack: \\\"normalize\\\"`). For example `\\\"s\\\"` for SI units. Use [D3's number format pattern](https://github.com/d3/d3-format#locale_format).\\n\\nIf `config.normalizedNumberFormatType` is specified and `config.customFormatTypes` is `true`, this value will be passed as `format` alongside `datum.value` to the `config.numberFormatType` function. __Default value:__ `%`\",\n \"type\": \"string\"\n },\n \"normalizedNumberFormatType\": {\n \"description\": \"[Custom format type](https://vega.github.io/vega-lite/docs/config.html#custom-format-type) for `config.normalizedNumberFormat`.\\n\\n__Default value:__ `undefined` -- This is equilvalent to call D3-format, which is exposed as [`format` in Vega-Expression](https://vega.github.io/vega/docs/expressions/#format). __Note:__ You must also set `customFormatTypes` to `true` to use this feature.\",\n \"type\": \"string\"\n },\n \"numberFormat\": {\n \"description\": \"If numberFormatType is not specified, D3 number format for guide labels, text marks, and tooltips of non-normalized fields (fields *without* `stack: \\\"normalize\\\"`). For example `\\\"s\\\"` for SI units. Use [D3's number format pattern](https://github.com/d3/d3-format#locale_format).\\n\\nIf `config.numberFormatType` is specified and `config.customFormatTypes` is `true`, this value will be passed as `format` alongside `datum.value` to the `config.numberFormatType` function.\",\n \"type\": \"string\"\n },\n \"numberFormatType\": {\n \"description\": \"[Custom format type](https://vega.github.io/vega-lite/docs/config.html#custom-format-type) for `config.numberFormat`.\\n\\n__Default value:__ `undefined` -- This is equilvalent to call D3-format, which is exposed as [`format` in Vega-Expression](https://vega.github.io/vega/docs/expressions/#format). __Note:__ You must also set `customFormatTypes` to `true` to use this feature.\",\n \"type\": \"string\"\n },\n \"timeFormat\": {\n \"description\": \"Default time format for raw time values (without time units) in text marks, legend labels and header labels.\\n\\n__Default value:__ `\\\"%b %d, %Y\\\"` __Note:__ Axes automatically determine the format for each label automatically so this config does not affect axes.\",\n \"type\": \"string\"\n },\n \"timeFormatType\": {\n \"description\": \"[Custom format type](https://vega.github.io/vega-lite/docs/config.html#custom-format-type) for `config.timeFormat`.\\n\\n__Default value:__ `undefined` -- This is equilvalent to call D3-time-format, which is exposed as [`timeFormat` in Vega-Expression](https://vega.github.io/vega/docs/expressions/#timeFormat). __Note:__ You must also set `customFormatTypes` to `true` and there must *not* be a `timeUnit` defined to use this feature.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"Generator\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SequenceGenerator\"\n },\n {\n \"$ref\": \"#/definitions/SphereGenerator\"\n },\n {\n \"$ref\": \"#/definitions/GraticuleGenerator\"\n }\n ]\n },\n \"ConcatSpec\": {\n \"additionalProperties\": false,\n \"description\": \"Base interface for a generalized concatenation specification.\",\n \"properties\": {\n \"align\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LayoutAlign\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"The alignment to apply to grid rows and columns. The supported string values are `\\\"all\\\"`, `\\\"each\\\"`, and `\\\"none\\\"`.\\n\\n- For `\\\"none\\\"`, a flow layout will be used, in which adjacent subviews are simply placed one after the other.\\n- For `\\\"each\\\"`, subviews will be aligned into a clean grid structure, but each row or column may be of variable size.\\n- For `\\\"all\\\"`, subviews will be aligned and each row or column will be sized identically based on the maximum observed size. String values for this property will be applied to both grid rows and columns.\\n\\nAlternatively, an object value of the form `{\\\"row\\\": string, \\\"column\\\": string}` can be used to supply different alignments for rows and columns.\\n\\n__Default value:__ `\\\"all\\\"`.\"\n },\n \"bounds\": {\n \"description\": \"The bounds calculation method to use for determining the extent of a sub-plot. One of `full` (the default) or `flush`.\\n\\n- If set to `full`, the entire calculated bounds (including axes, title, and legend) will be used.\\n- If set to `flush`, only the specified width and height values for the sub-view will be used. The `flush` setting can be useful when attempting to place sub-plots without axes or legends into a uniform grid structure.\\n\\n__Default value:__ `\\\"full\\\"`\",\n \"enum\": [\n \"full\",\n \"flush\"\n ],\n \"type\": \"string\"\n },\n \"center\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"Boolean flag indicating if subviews should be centered relative to their respective rows or columns.\\n\\nAn object value of the form `{\\\"row\\\": boolean, \\\"column\\\": boolean}` can be used to supply different centering values for rows and columns.\\n\\n__Default value:__ `false`\"\n },\n \"columns\": {\n \"description\": \"The number of columns to include in the view composition layout.\\n\\n__Default value__: `undefined` -- An infinite number of columns (a single row) will be assumed. This is equivalent to `hconcat` (for `concat`) and to using the `column` channel (for `facet` and `repeat`).\\n\\n__Note__:\\n\\n1) This property is only for:\\n- the general (wrappable) `concat` operator (not `hconcat`/`vconcat`)\\n- the `facet` and `repeat` operator with one field/repetition definition (without row/column nesting)\\n\\n2) Setting the `columns` to `1` is equivalent to `vconcat` (for `concat`) and to using the `row` channel (for `facet` and `repeat`).\",\n \"type\": \"number\"\n },\n \"concat\": {\n \"description\": \"A list of views to be concatenated.\",\n \"items\": {\n \"$ref\": \"#/definitions/Spec\"\n },\n \"type\": \"array\"\n },\n \"data\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Data\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object describing the data source. Set to `null` to ignore the parent's data source. If no data is set, it is derived from the parent.\"\n },\n \"description\": {\n \"description\": \"Description of this mark for commenting purpose.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the visualization for later reference.\",\n \"type\": \"string\"\n },\n \"resolve\": {\n \"$ref\": \"#/definitions/Resolve\",\n \"description\": \"Scale, axis, and legend resolutions for view composition specifications.\"\n },\n \"spacing\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"The spacing in pixels between sub-views of the composition operator. An object of the form `{\\\"row\\\": number, \\\"column\\\": number}` can be used to set different spacing values for rows and columns.\\n\\n__Default value__: Depends on `\\\"spacing\\\"` property of [the view composition configuration](https://vega.github.io/vega-lite/docs/config.html#view-config) (`20` by default)\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"$ref\": \"#/definitions/TitleParams\"\n }\n ],\n \"description\": \"Title for the plot.\"\n },\n \"transform\": {\n \"description\": \"An array of data transformations such as filter and new field calculation.\",\n \"items\": {\n \"$ref\": \"#/definitions/Transform\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"concat\"\n ],\n \"type\": \"object\"\n },\n \"FacetSpec\": {\n \"additionalProperties\": false,\n \"description\": \"Base interface for a facet specification.\",\n \"properties\": {\n \"align\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LayoutAlign\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"The alignment to apply to grid rows and columns. The supported string values are `\\\"all\\\"`, `\\\"each\\\"`, and `\\\"none\\\"`.\\n\\n- For `\\\"none\\\"`, a flow layout will be used, in which adjacent subviews are simply placed one after the other.\\n- For `\\\"each\\\"`, subviews will be aligned into a clean grid structure, but each row or column may be of variable size.\\n- For `\\\"all\\\"`, subviews will be aligned and each row or column will be sized identically based on the maximum observed size. String values for this property will be applied to both grid rows and columns.\\n\\nAlternatively, an object value of the form `{\\\"row\\\": string, \\\"column\\\": string}` can be used to supply different alignments for rows and columns.\\n\\n__Default value:__ `\\\"all\\\"`.\"\n },\n \"bounds\": {\n \"description\": \"The bounds calculation method to use for determining the extent of a sub-plot. One of `full` (the default) or `flush`.\\n\\n- If set to `full`, the entire calculated bounds (including axes, title, and legend) will be used.\\n- If set to `flush`, only the specified width and height values for the sub-view will be used. The `flush` setting can be useful when attempting to place sub-plots without axes or legends into a uniform grid structure.\\n\\n__Default value:__ `\\\"full\\\"`\",\n \"enum\": [\n \"full\",\n \"flush\"\n ],\n \"type\": \"string\"\n },\n \"center\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"Boolean flag indicating if subviews should be centered relative to their respective rows or columns.\\n\\nAn object value of the form `{\\\"row\\\": boolean, \\\"column\\\": boolean}` can be used to supply different centering values for rows and columns.\\n\\n__Default value:__ `false`\"\n },\n \"columns\": {\n \"description\": \"The number of columns to include in the view composition layout.\\n\\n__Default value__: `undefined` -- An infinite number of columns (a single row) will be assumed. This is equivalent to `hconcat` (for `concat`) and to using the `column` channel (for `facet` and `repeat`).\\n\\n__Note__:\\n\\n1) This property is only for:\\n- the general (wrappable) `concat` operator (not `hconcat`/`vconcat`)\\n- the `facet` and `repeat` operator with one field/repetition definition (without row/column nesting)\\n\\n2) Setting the `columns` to `1` is equivalent to `vconcat` (for `concat`) and to using the `row` channel (for `facet` and `repeat`).\",\n \"type\": \"number\"\n },\n \"data\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Data\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object describing the data source. Set to `null` to ignore the parent's data source. If no data is set, it is derived from the parent.\"\n },\n \"description\": {\n \"description\": \"Description of this mark for commenting purpose.\",\n \"type\": \"string\"\n },\n \"facet\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FacetFieldDef\"\n },\n {\n \"$ref\": \"#/definitions/FacetMapping\"\n }\n ],\n \"description\": \"Definition for how to facet the data. One of: 1) [a field definition for faceting the plot by one field](https://vega.github.io/vega-lite/docs/facet.html#field-def) 2) [An object that maps `row` and `column` channels to their field definitions](https://vega.github.io/vega-lite/docs/facet.html#mapping)\"\n },\n \"name\": {\n \"description\": \"Name of the visualization for later reference.\",\n \"type\": \"string\"\n },\n \"resolve\": {\n \"$ref\": \"#/definitions/Resolve\",\n \"description\": \"Scale, axis, and legend resolutions for view composition specifications.\"\n },\n \"spacing\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"The spacing in pixels between sub-views of the composition operator. An object of the form `{\\\"row\\\": number, \\\"column\\\": number}` can be used to set different spacing values for rows and columns.\\n\\n__Default value__: Depends on `\\\"spacing\\\"` property of [the view composition configuration](https://vega.github.io/vega-lite/docs/config.html#view-config) (`20` by default)\"\n },\n \"spec\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LayerSpec\"\n },\n {\n \"$ref\": \"#/definitions/FacetedUnitSpec\"\n }\n ],\n \"description\": \"A specification of the view that gets faceted.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"$ref\": \"#/definitions/TitleParams\"\n }\n ],\n \"description\": \"Title for the plot.\"\n },\n \"transform\": {\n \"description\": \"An array of data transformations such as filter and new field calculation.\",\n \"items\": {\n \"$ref\": \"#/definitions/Transform\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"facet\",\n \"spec\"\n ],\n \"type\": \"object\"\n },\n \"HConcatSpec\": {\n \"additionalProperties\": false,\n \"description\": \"Base interface for a horizontal concatenation specification.\",\n \"properties\": {\n \"bounds\": {\n \"description\": \"The bounds calculation method to use for determining the extent of a sub-plot. One of `full` (the default) or `flush`.\\n\\n- If set to `full`, the entire calculated bounds (including axes, title, and legend) will be used.\\n- If set to `flush`, only the specified width and height values for the sub-view will be used. The `flush` setting can be useful when attempting to place sub-plots without axes or legends into a uniform grid structure.\\n\\n__Default value:__ `\\\"full\\\"`\",\n \"enum\": [\n \"full\",\n \"flush\"\n ],\n \"type\": \"string\"\n },\n \"center\": {\n \"description\": \"Boolean flag indicating if subviews should be centered relative to their respective rows or columns.\\n\\n__Default value:__ `false`\",\n \"type\": \"boolean\"\n },\n \"data\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Data\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object describing the data source. Set to `null` to ignore the parent's data source. If no data is set, it is derived from the parent.\"\n },\n \"description\": {\n \"description\": \"Description of this mark for commenting purpose.\",\n \"type\": \"string\"\n },\n \"hconcat\": {\n \"description\": \"A list of views to be concatenated and put into a row.\",\n \"items\": {\n \"$ref\": \"#/definitions/Spec\"\n },\n \"type\": \"array\"\n },\n \"name\": {\n \"description\": \"Name of the visualization for later reference.\",\n \"type\": \"string\"\n },\n \"resolve\": {\n \"$ref\": \"#/definitions/Resolve\",\n \"description\": \"Scale, axis, and legend resolutions for view composition specifications.\"\n },\n \"spacing\": {\n \"description\": \"The spacing in pixels between sub-views of the concat operator.\\n\\n__Default value__: `10`\",\n \"type\": \"number\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"$ref\": \"#/definitions/TitleParams\"\n }\n ],\n \"description\": \"Title for the plot.\"\n },\n \"transform\": {\n \"description\": \"An array of data transformations such as filter and new field calculation.\",\n \"items\": {\n \"$ref\": \"#/definitions/Transform\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"hconcat\"\n ],\n \"type\": \"object\"\n },\n \"Spec\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FacetedUnitSpec\"\n },\n {\n \"$ref\": \"#/definitions/LayerSpec\"\n },\n {\n \"$ref\": \"#/definitions/RepeatSpec\"\n },\n {\n \"$ref\": \"#/definitions/FacetSpec\"\n },\n {\n \"$ref\": \"#/definitions/ConcatSpec\"\n },\n {\n \"$ref\": \"#/definitions/VConcatSpec\"\n },\n {\n \"$ref\": \"#/definitions/HConcatSpec\"\n }\n ],\n \"description\": \"Any specification in Vega-Lite.\"\n },\n \"GenericUnitSpec\": {\n \"additionalProperties\": false,\n \"description\": \"Base interface for a unit (single-view) specification.\",\n \"properties\": {\n \"data\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Data\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object describing the data source. Set to `null` to ignore the parent's data source. If no data is set, it is derived from the parent.\"\n },\n \"description\": {\n \"description\": \"Description of this mark for commenting purpose.\",\n \"type\": \"string\"\n },\n \"encoding\": {\n \"$ref\": \"#/definitions/Encoding\",\n \"description\": \"A key-value mapping between encoding channels and definition of fields.\"\n },\n \"mark\": {\n \"$ref\": \"#/definitions/AnyMark\",\n \"description\": \"A string describing the mark type (one of `\\\"bar\\\"`, `\\\"circle\\\"`, `\\\"square\\\"`, `\\\"tick\\\"`, `\\\"line\\\"`, `\\\"area\\\"`, `\\\"point\\\"`, `\\\"rule\\\"`, `\\\"geoshape\\\"`, and `\\\"text\\\"`) or a [mark definition object](https://vega.github.io/vega-lite/docs/mark.html#mark-def).\"\n },\n \"name\": {\n \"description\": \"Name of the visualization for later reference.\",\n \"type\": \"string\"\n },\n \"params\": {\n \"description\": \"An array of parameters that may either be simple variables, or more complex selections that map user input to data queries.\",\n \"items\": {\n \"$ref\": \"#/definitions/SelectionParameter\"\n },\n \"type\": \"array\"\n },\n \"projection\": {\n \"$ref\": \"#/definitions/Projection\",\n \"description\": \"An object defining properties of geographic projection, which will be applied to `shape` path for `\\\"geoshape\\\"` marks and to `latitude` and `\\\"longitude\\\"` channels for other marks.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"$ref\": \"#/definitions/TitleParams\"\n }\n ],\n \"description\": \"Title for the plot.\"\n },\n \"transform\": {\n \"description\": \"An array of data transformations such as filter and new field calculation.\",\n \"items\": {\n \"$ref\": \"#/definitions/Transform\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"mark\"\n ],\n \"type\": \"object\"\n },\n \"VConcatSpec\": {\n \"additionalProperties\": false,\n \"description\": \"Base interface for a vertical concatenation specification.\",\n \"properties\": {\n \"bounds\": {\n \"description\": \"The bounds calculation method to use for determining the extent of a sub-plot. One of `full` (the default) or `flush`.\\n\\n- If set to `full`, the entire calculated bounds (including axes, title, and legend) will be used.\\n- If set to `flush`, only the specified width and height values for the sub-view will be used. The `flush` setting can be useful when attempting to place sub-plots without axes or legends into a uniform grid structure.\\n\\n__Default value:__ `\\\"full\\\"`\",\n \"enum\": [\n \"full\",\n \"flush\"\n ],\n \"type\": \"string\"\n },\n \"center\": {\n \"description\": \"Boolean flag indicating if subviews should be centered relative to their respective rows or columns.\\n\\n__Default value:__ `false`\",\n \"type\": \"boolean\"\n },\n \"data\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Data\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object describing the data source. Set to `null` to ignore the parent's data source. If no data is set, it is derived from the parent.\"\n },\n \"description\": {\n \"description\": \"Description of this mark for commenting purpose.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the visualization for later reference.\",\n \"type\": \"string\"\n },\n \"resolve\": {\n \"$ref\": \"#/definitions/Resolve\",\n \"description\": \"Scale, axis, and legend resolutions for view composition specifications.\"\n },\n \"spacing\": {\n \"description\": \"The spacing in pixels between sub-views of the concat operator.\\n\\n__Default value__: `10`\",\n \"type\": \"number\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"$ref\": \"#/definitions/TitleParams\"\n }\n ],\n \"description\": \"Title for the plot.\"\n },\n \"transform\": {\n \"description\": \"An array of data transformations such as filter and new field calculation.\",\n \"items\": {\n \"$ref\": \"#/definitions/Transform\"\n },\n \"type\": \"array\"\n },\n \"vconcat\": {\n \"description\": \"A list of views to be concatenated and put into a column.\",\n \"items\": {\n \"$ref\": \"#/definitions/Spec\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"vconcat\"\n ],\n \"type\": \"object\"\n },\n \"GeoJsonFeature\": {\n \"$ref\": \"#/definitions/Feature\"\n },\n \"GeoJsonFeatureCollection\": {\n \"$ref\": \"#/definitions/FeatureCollection\"\n },\n \"GeoJsonProperties\": {\n \"anyOf\": [\n {\n \"type\": \"object\"\n },\n {\n \"type\": \"null\"\n }\n ]\n },\n \"Geometry\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Point\"\n },\n {\n \"$ref\": \"#/definitions/MultiPoint\"\n },\n {\n \"$ref\": \"#/definitions/LineString\"\n },\n {\n \"$ref\": \"#/definitions/MultiLineString\"\n },\n {\n \"$ref\": \"#/definitions/Polygon\"\n },\n {\n \"$ref\": \"#/definitions/MultiPolygon\"\n },\n {\n \"$ref\": \"#/definitions/GeometryCollection\"\n }\n ],\n \"description\": \"Union of geometry objects. https://tools.ietf.org/html/rfc7946#section-3\"\n },\n \"GeometryCollection\": {\n \"additionalProperties\": false,\n \"description\": \"Geometry Collection https://tools.ietf.org/html/rfc7946#section-3.1.8\",\n \"properties\": {\n \"bbox\": {\n \"$ref\": \"#/definitions/BBox\",\n \"description\": \"Bounding box of the coordinate range of the object's Geometries, Features, or Feature Collections. https://tools.ietf.org/html/rfc7946#section-5\"\n },\n \"geometries\": {\n \"items\": {\n \"$ref\": \"#/definitions/Geometry\"\n },\n \"type\": \"array\"\n },\n \"type\": {\n \"const\": \"GeometryCollection\",\n \"description\": \"Specifies the type of GeoJSON object.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"geometries\",\n \"type\"\n ],\n \"type\": \"object\"\n },\n \"Gradient\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LinearGradient\"\n },\n {\n \"$ref\": \"#/definitions/RadialGradient\"\n }\n ]\n },\n \"GradientStop\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"color\": {\n \"$ref\": \"#/definitions/Color\",\n \"description\": \"The color value at this point in the gradient.\"\n },\n \"offset\": {\n \"description\": \"The offset fraction for the color stop, indicating its position within the gradient.\",\n \"type\": \"number\"\n }\n },\n \"required\": [\n \"offset\",\n \"color\"\n ],\n \"type\": \"object\"\n },\n \"GraticuleGenerator\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"graticule\": {\n \"anyOf\": [\n {\n \"const\": true,\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/GraticuleParams\"\n }\n ],\n \"description\": \"Generate graticule GeoJSON data for geographic reference lines.\"\n },\n \"name\": {\n \"description\": \"Provide a placeholder name and bind data at runtime.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"graticule\"\n ],\n \"type\": \"object\"\n },\n \"GraticuleParams\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"extent\": {\n \"$ref\": \"#/definitions/Vector2>\",\n \"description\": \"Sets both the major and minor extents to the same values.\"\n },\n \"extentMajor\": {\n \"$ref\": \"#/definitions/Vector2>\",\n \"description\": \"The major extent of the graticule as a two-element array of coordinates.\"\n },\n \"extentMinor\": {\n \"$ref\": \"#/definitions/Vector2>\",\n \"description\": \"The minor extent of the graticule as a two-element array of coordinates.\"\n },\n \"precision\": {\n \"description\": \"The precision of the graticule in degrees.\\n\\n__Default value:__ `2.5`\",\n \"type\": \"number\"\n },\n \"step\": {\n \"$ref\": \"#/definitions/Vector2\",\n \"description\": \"Sets both the major and minor step angles to the same values.\"\n },\n \"stepMajor\": {\n \"$ref\": \"#/definitions/Vector2\",\n \"description\": \"The major step angles of the graticule.\\n\\n\\n__Default value:__ `[90, 360]`\"\n },\n \"stepMinor\": {\n \"$ref\": \"#/definitions/Vector2\",\n \"description\": \"The minor step angles of the graticule.\\n\\n__Default value:__ `[10, 10]`\"\n }\n },\n \"type\": \"object\"\n },\n \"Header\": {\n \"additionalProperties\": false,\n \"description\": \"Headers of row / column channels for faceted plots.\",\n \"properties\": {\n \"format\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Dict\"\n }\n ],\n \"description\": \"When used with the default `\\\"number\\\"` and `\\\"time\\\"` format type, the text formatting pattern for labels of guides (axes, legends, headers) and text marks.\\n\\n- If the format type is `\\\"number\\\"` (e.g., for quantitative fields), this is D3's [number format pattern](https://github.com/d3/d3-format#locale_format).\\n- If the format type is `\\\"time\\\"` (e.g., for temporal fields), this is D3's [time format pattern](https://github.com/d3/d3-time-format#locale_format).\\n\\nSee the [format documentation](https://vega.github.io/vega-lite/docs/format.html) for more examples.\\n\\nWhen used with a [custom `formatType`](https://vega.github.io/vega-lite/docs/config.html#custom-format-type), this value will be passed as `format` alongside `datum.value` to the registered function.\\n\\n__Default value:__ Derived from [numberFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for number format and from [timeFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for time format.\"\n },\n \"formatType\": {\n \"description\": \"The format type for labels. One of `\\\"number\\\"`, `\\\"time\\\"`, or a [registered custom format type](https://vega.github.io/vega-lite/docs/config.html#custom-format-type).\\n\\n__Default value:__\\n- `\\\"time\\\"` for temporal fields and ordinal and nominal fields with `timeUnit`.\\n- `\\\"number\\\"` for quantitative fields as well as ordinal and nominal fields without `timeUnit`.\",\n \"type\": \"string\"\n },\n \"labelAlign\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Align\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Horizontal text alignment of header labels. One of `\\\"left\\\"`, `\\\"center\\\"`, or `\\\"right\\\"`.\"\n },\n \"labelAnchor\": {\n \"$ref\": \"#/definitions/TitleAnchor\",\n \"description\": \"The anchor position for placing the labels. One of `\\\"start\\\"`, `\\\"middle\\\"`, or `\\\"end\\\"`. For example, with a label orientation of top these anchor positions map to a left-, center-, or right-aligned label.\"\n },\n \"labelAngle\": {\n \"description\": \"The rotation angle of the header labels.\\n\\n__Default value:__ `0` for column header, `-90` for row header.\",\n \"maximum\": 360,\n \"minimum\": -360,\n \"type\": \"number\"\n },\n \"labelBaseline\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextBaseline\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The vertical text baseline for the header labels. One of `\\\"alphabetic\\\"` (default), `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`, `\\\"line-top\\\"`, or `\\\"line-bottom\\\"`. The `\\\"line-top\\\"` and `\\\"line-bottom\\\"` values operate similarly to `\\\"top\\\"` and `\\\"bottom\\\"`, but are calculated relative to the `titleLineHeight` rather than `titleFontSize` alone.\"\n },\n \"labelColor\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The color of the header label, can be in hex color code or regular color name.\"\n },\n \"labelExpr\": {\n \"description\": \"[Vega expression](https://vega.github.io/vega/docs/expressions/) for customizing labels.\\n\\n__Note:__ The label text and value can be assessed via the `label` and `value` properties of the header's backing `datum` object.\",\n \"type\": \"string\"\n },\n \"labelFont\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The font of the header label.\"\n },\n \"labelFontSize\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The font size of the header label, in pixels.\",\n \"minimum\": 0\n },\n \"labelFontStyle\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The font style of the header label.\"\n },\n \"labelFontWeight\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The font weight of the header label.\"\n },\n \"labelLimit\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The maximum length of the header label in pixels. The text value will be automatically truncated if the rendered size exceeds the limit.\\n\\n__Default value:__ `0`, indicating no limit\"\n },\n \"labelLineHeight\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Line height in pixels for multi-line header labels or title text with `\\\"line-top\\\"` or `\\\"line-bottom\\\"` baseline.\"\n },\n \"labelOrient\": {\n \"$ref\": \"#/definitions/Orient\",\n \"description\": \"The orientation of the header label. One of `\\\"top\\\"`, `\\\"bottom\\\"`, `\\\"left\\\"` or `\\\"right\\\"`.\"\n },\n \"labelPadding\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The padding, in pixel, between facet header's label and the plot.\\n\\n__Default value:__ `10`\"\n },\n \"labels\": {\n \"description\": \"A boolean flag indicating if labels should be included as part of the header.\\n\\n__Default value:__ `true`.\",\n \"type\": \"boolean\"\n },\n \"orient\": {\n \"$ref\": \"#/definitions/Orient\",\n \"description\": \"Shortcut for setting both labelOrient and titleOrient.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"titleAlign\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Align\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Horizontal text alignment (to the anchor) of header titles.\"\n },\n \"titleAnchor\": {\n \"$ref\": \"#/definitions/TitleAnchor\",\n \"description\": \"The anchor position for placing the title. One of `\\\"start\\\"`, `\\\"middle\\\"`, or `\\\"end\\\"`. For example, with an orientation of top these anchor positions map to a left-, center-, or right-aligned title.\"\n },\n \"titleAngle\": {\n \"description\": \"The rotation angle of the header title.\\n\\n__Default value:__ `0`.\",\n \"maximum\": 360,\n \"minimum\": -360,\n \"type\": \"number\"\n },\n \"titleBaseline\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextBaseline\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The vertical text baseline for the header title. One of `\\\"alphabetic\\\"` (default), `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`, `\\\"line-top\\\"`, or `\\\"line-bottom\\\"`. The `\\\"line-top\\\"` and `\\\"line-bottom\\\"` values operate similarly to `\\\"top\\\"` and `\\\"bottom\\\"`, but are calculated relative to the `titleLineHeight` rather than `titleFontSize` alone.\\n\\n__Default value:__ `\\\"middle\\\"`\"\n },\n \"titleColor\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Color of the header title, can be in hex color code or regular color name.\"\n },\n \"titleFont\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Font of the header title. (e.g., `\\\"Helvetica Neue\\\"`).\"\n },\n \"titleFontSize\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Font size of the header title.\",\n \"minimum\": 0\n },\n \"titleFontStyle\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The font style of the header title.\"\n },\n \"titleFontWeight\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Font weight of the header title. This can be either a string (e.g `\\\"bold\\\"`, `\\\"normal\\\"`) or a number (`100`, `200`, `300`, ..., `900` where `\\\"normal\\\"` = `400` and `\\\"bold\\\"` = `700`).\"\n },\n \"titleLimit\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The maximum length of the header title in pixels. The text value will be automatically truncated if the rendered size exceeds the limit.\\n\\n__Default value:__ `0`, indicating no limit\"\n },\n \"titleLineHeight\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Line height in pixels for multi-line header title text or title text with `\\\"line-top\\\"` or `\\\"line-bottom\\\"` baseline.\"\n },\n \"titleOrient\": {\n \"$ref\": \"#/definitions/Orient\",\n \"description\": \"The orientation of the header title. One of `\\\"top\\\"`, `\\\"bottom\\\"`, `\\\"left\\\"` or `\\\"right\\\"`.\"\n },\n \"titlePadding\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The padding, in pixel, between facet header's title and the label.\\n\\n__Default value:__ `10`\"\n }\n },\n \"type\": \"object\"\n },\n \"HeaderConfig\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"format\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Dict\"\n }\n ],\n \"description\": \"When used with the default `\\\"number\\\"` and `\\\"time\\\"` format type, the text formatting pattern for labels of guides (axes, legends, headers) and text marks.\\n\\n- If the format type is `\\\"number\\\"` (e.g., for quantitative fields), this is D3's [number format pattern](https://github.com/d3/d3-format#locale_format).\\n- If the format type is `\\\"time\\\"` (e.g., for temporal fields), this is D3's [time format pattern](https://github.com/d3/d3-time-format#locale_format).\\n\\nSee the [format documentation](https://vega.github.io/vega-lite/docs/format.html) for more examples.\\n\\nWhen used with a [custom `formatType`](https://vega.github.io/vega-lite/docs/config.html#custom-format-type), this value will be passed as `format` alongside `datum.value` to the registered function.\\n\\n__Default value:__ Derived from [numberFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for number format and from [timeFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for time format.\"\n },\n \"formatType\": {\n \"description\": \"The format type for labels. One of `\\\"number\\\"`, `\\\"time\\\"`, or a [registered custom format type](https://vega.github.io/vega-lite/docs/config.html#custom-format-type).\\n\\n__Default value:__\\n- `\\\"time\\\"` for temporal fields and ordinal and nominal fields with `timeUnit`.\\n- `\\\"number\\\"` for quantitative fields as well as ordinal and nominal fields without `timeUnit`.\",\n \"type\": \"string\"\n },\n \"labelAlign\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Align\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Horizontal text alignment of header labels. One of `\\\"left\\\"`, `\\\"center\\\"`, or `\\\"right\\\"`.\"\n },\n \"labelAnchor\": {\n \"$ref\": \"#/definitions/TitleAnchor\",\n \"description\": \"The anchor position for placing the labels. One of `\\\"start\\\"`, `\\\"middle\\\"`, or `\\\"end\\\"`. For example, with a label orientation of top these anchor positions map to a left-, center-, or right-aligned label.\"\n },\n \"labelAngle\": {\n \"description\": \"The rotation angle of the header labels.\\n\\n__Default value:__ `0` for column header, `-90` for row header.\",\n \"maximum\": 360,\n \"minimum\": -360,\n \"type\": \"number\"\n },\n \"labelBaseline\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextBaseline\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The vertical text baseline for the header labels. One of `\\\"alphabetic\\\"` (default), `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`, `\\\"line-top\\\"`, or `\\\"line-bottom\\\"`. The `\\\"line-top\\\"` and `\\\"line-bottom\\\"` values operate similarly to `\\\"top\\\"` and `\\\"bottom\\\"`, but are calculated relative to the `titleLineHeight` rather than `titleFontSize` alone.\"\n },\n \"labelColor\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The color of the header label, can be in hex color code or regular color name.\"\n },\n \"labelExpr\": {\n \"description\": \"[Vega expression](https://vega.github.io/vega/docs/expressions/) for customizing labels.\\n\\n__Note:__ The label text and value can be assessed via the `label` and `value` properties of the header's backing `datum` object.\",\n \"type\": \"string\"\n },\n \"labelFont\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The font of the header label.\"\n },\n \"labelFontSize\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The font size of the header label, in pixels.\",\n \"minimum\": 0\n },\n \"labelFontStyle\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The font style of the header label.\"\n },\n \"labelFontWeight\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The font weight of the header label.\"\n },\n \"labelLimit\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The maximum length of the header label in pixels. The text value will be automatically truncated if the rendered size exceeds the limit.\\n\\n__Default value:__ `0`, indicating no limit\"\n },\n \"labelLineHeight\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Line height in pixels for multi-line header labels or title text with `\\\"line-top\\\"` or `\\\"line-bottom\\\"` baseline.\"\n },\n \"labelOrient\": {\n \"$ref\": \"#/definitions/Orient\",\n \"description\": \"The orientation of the header label. One of `\\\"top\\\"`, `\\\"bottom\\\"`, `\\\"left\\\"` or `\\\"right\\\"`.\"\n },\n \"labelPadding\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The padding, in pixel, between facet header's label and the plot.\\n\\n__Default value:__ `10`\"\n },\n \"labels\": {\n \"description\": \"A boolean flag indicating if labels should be included as part of the header.\\n\\n__Default value:__ `true`.\",\n \"type\": \"boolean\"\n },\n \"orient\": {\n \"$ref\": \"#/definitions/Orient\",\n \"description\": \"Shortcut for setting both labelOrient and titleOrient.\"\n },\n \"title\": {\n \"description\": \"Set to null to disable title for the axis, legend, or header.\",\n \"type\": \"null\"\n },\n \"titleAlign\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Align\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Horizontal text alignment (to the anchor) of header titles.\"\n },\n \"titleAnchor\": {\n \"$ref\": \"#/definitions/TitleAnchor\",\n \"description\": \"The anchor position for placing the title. One of `\\\"start\\\"`, `\\\"middle\\\"`, or `\\\"end\\\"`. For example, with an orientation of top these anchor positions map to a left-, center-, or right-aligned title.\"\n },\n \"titleAngle\": {\n \"description\": \"The rotation angle of the header title.\\n\\n__Default value:__ `0`.\",\n \"maximum\": 360,\n \"minimum\": -360,\n \"type\": \"number\"\n },\n \"titleBaseline\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextBaseline\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The vertical text baseline for the header title. One of `\\\"alphabetic\\\"` (default), `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`, `\\\"line-top\\\"`, or `\\\"line-bottom\\\"`. The `\\\"line-top\\\"` and `\\\"line-bottom\\\"` values operate similarly to `\\\"top\\\"` and `\\\"bottom\\\"`, but are calculated relative to the `titleLineHeight` rather than `titleFontSize` alone.\\n\\n__Default value:__ `\\\"middle\\\"`\"\n },\n \"titleColor\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Color of the header title, can be in hex color code or regular color name.\"\n },\n \"titleFont\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Font of the header title. (e.g., `\\\"Helvetica Neue\\\"`).\"\n },\n \"titleFontSize\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Font size of the header title.\",\n \"minimum\": 0\n },\n \"titleFontStyle\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The font style of the header title.\"\n },\n \"titleFontWeight\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Font weight of the header title. This can be either a string (e.g `\\\"bold\\\"`, `\\\"normal\\\"`) or a number (`100`, `200`, `300`, ..., `900` where `\\\"normal\\\"` = `400` and `\\\"bold\\\"` = `700`).\"\n },\n \"titleLimit\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The maximum length of the header title in pixels. The text value will be automatically truncated if the rendered size exceeds the limit.\\n\\n__Default value:__ `0`, indicating no limit\"\n },\n \"titleLineHeight\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Line height in pixels for multi-line header title text or title text with `\\\"line-top\\\"` or `\\\"line-bottom\\\"` baseline.\"\n },\n \"titleOrient\": {\n \"$ref\": \"#/definitions/Orient\",\n \"description\": \"The orientation of the header title. One of `\\\"top\\\"`, `\\\"bottom\\\"`, `\\\"left\\\"` or `\\\"right\\\"`.\"\n },\n \"titlePadding\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The padding, in pixel, between facet header's title and the label.\\n\\n__Default value:__ `10`\"\n }\n },\n \"type\": \"object\"\n },\n \"HexColor\": {\n \"format\": \"color-hex\",\n \"type\": \"string\"\n },\n \"ImputeMethod\": {\n \"enum\": [\n \"value\",\n \"median\",\n \"max\",\n \"min\",\n \"mean\"\n ],\n \"type\": \"string\"\n },\n \"ImputeParams\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"frame\": {\n \"description\": \"A frame specification as a two-element array used to control the window over which the specified method is applied. The array entries should either be a number indicating the offset from the current data object, or null to indicate unbounded rows preceding or following the current data object. For example, the value `[-5, 5]` indicates that the window should include five objects preceding and five objects following the current object.\\n\\n__Default value:__: `[null, null]` indicating that the window includes all objects.\",\n \"items\": {\n \"type\": [\n \"null\",\n \"number\"\n ]\n },\n \"maxItems\": 2,\n \"minItems\": 2,\n \"type\": \"array\"\n },\n \"keyvals\": {\n \"anyOf\": [\n {\n \"items\": {},\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ImputeSequence\"\n }\n ],\n \"description\": \"Defines the key values that should be considered for imputation. An array of key values or an object defining a [number sequence](https://vega.github.io/vega-lite/docs/impute.html#sequence-def).\\n\\nIf provided, this will be used in addition to the key values observed within the input data. If not provided, the values will be derived from all unique values of the `key` field. For `impute` in `encoding`, the key field is the x-field if the y-field is imputed, or vice versa.\\n\\nIf there is no impute grouping, this property _must_ be specified.\"\n },\n \"method\": {\n \"$ref\": \"#/definitions/ImputeMethod\",\n \"description\": \"The imputation method to use for the field value of imputed data objects. One of `\\\"value\\\"`, `\\\"mean\\\"`, `\\\"median\\\"`, `\\\"max\\\"` or `\\\"min\\\"`.\\n\\n__Default value:__ `\\\"value\\\"`\"\n },\n \"value\": {\n \"description\": \"The field value to use when the imputation `method` is `\\\"value\\\"`.\"\n }\n },\n \"type\": \"object\"\n },\n \"ImputeSequence\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"start\": {\n \"description\": \"The starting value of the sequence. __Default value:__ `0`\",\n \"type\": \"number\"\n },\n \"step\": {\n \"description\": \"The step value between sequence entries. __Default value:__ `1` or `-1` if `stop < start`\",\n \"type\": \"number\"\n },\n \"stop\": {\n \"description\": \"The ending value(exclusive) of the sequence.\",\n \"type\": \"number\"\n }\n },\n \"required\": [\n \"stop\"\n ],\n \"type\": \"object\"\n },\n \"ImputeTransform\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"frame\": {\n \"description\": \"A frame specification as a two-element array used to control the window over which the specified method is applied. The array entries should either be a number indicating the offset from the current data object, or null to indicate unbounded rows preceding or following the current data object. For example, the value `[-5, 5]` indicates that the window should include five objects preceding and five objects following the current object.\\n\\n__Default value:__: `[null, null]` indicating that the window includes all objects.\",\n \"items\": {\n \"type\": [\n \"null\",\n \"number\"\n ]\n },\n \"maxItems\": 2,\n \"minItems\": 2,\n \"type\": \"array\"\n },\n \"groupby\": {\n \"description\": \"An optional array of fields by which to group the values. Imputation will then be performed on a per-group basis.\",\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"type\": \"array\"\n },\n \"impute\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"The data field for which the missing values should be imputed.\"\n },\n \"key\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"A key field that uniquely identifies data objects within a group. Missing key values (those occurring in the data but not in the current group) will be imputed.\"\n },\n \"keyvals\": {\n \"anyOf\": [\n {\n \"items\": {},\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ImputeSequence\"\n }\n ],\n \"description\": \"Defines the key values that should be considered for imputation. An array of key values or an object defining a [number sequence](https://vega.github.io/vega-lite/docs/impute.html#sequence-def).\\n\\nIf provided, this will be used in addition to the key values observed within the input data. If not provided, the values will be derived from all unique values of the `key` field. For `impute` in `encoding`, the key field is the x-field if the y-field is imputed, or vice versa.\\n\\nIf there is no impute grouping, this property _must_ be specified.\"\n },\n \"method\": {\n \"$ref\": \"#/definitions/ImputeMethod\",\n \"description\": \"The imputation method to use for the field value of imputed data objects. One of `\\\"value\\\"`, `\\\"mean\\\"`, `\\\"median\\\"`, `\\\"max\\\"` or `\\\"min\\\"`.\\n\\n__Default value:__ `\\\"value\\\"`\"\n },\n \"value\": {\n \"description\": \"The field value to use when the imputation `method` is `\\\"value\\\"`.\"\n }\n },\n \"required\": [\n \"impute\",\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"InlineData\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"format\": {\n \"$ref\": \"#/definitions/DataFormat\",\n \"description\": \"An object that specifies the format for parsing the data.\"\n },\n \"name\": {\n \"description\": \"Provide a placeholder name and bind data at runtime.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"$ref\": \"#/definitions/InlineDataset\",\n \"description\": \"The full data set, included inline. This can be an array of objects or primitive values, an object, or a string. Arrays of primitive values are ingested as objects with a `data` property. Strings are parsed according to the specified format type.\"\n }\n },\n \"required\": [\n \"values\"\n ],\n \"type\": \"object\"\n },\n \"InlineDataset\": {\n \"anyOf\": [\n {\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n {\n \"items\": {\n \"type\": \"boolean\"\n },\n \"type\": \"array\"\n },\n {\n \"items\": {\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"object\"\n }\n ]\n },\n \"Interpolate\": {\n \"enum\": [\n \"basis\",\n \"basis-open\",\n \"basis-closed\",\n \"bundle\",\n \"cardinal\",\n \"cardinal-open\",\n \"cardinal-closed\",\n \"catmull-rom\",\n \"linear\",\n \"linear-closed\",\n \"monotone\",\n \"natural\",\n \"step\",\n \"step-before\",\n \"step-after\"\n ],\n \"type\": \"string\"\n },\n \"IntervalSelectionConfig\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"clear\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Stream\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"boolean\"\n }\n ],\n \"description\": \"Clears the selection, emptying it of all values. This property can be a [Event Stream](https://vega.github.io/vega/docs/event-streams/) or `false` to disable clear.\\n\\n__Default value:__ `dblclick`.\\n\\n__See also:__ [`clear` examples ](https://vega.github.io/vega-lite/docs/selection.html#clear) in the documentation.\"\n },\n \"encodings\": {\n \"description\": \"An array of encoding channels. The corresponding data field values must match for a data tuple to fall within the selection.\\n\\n__See also:__ The [projection with `encodings` and `fields` section](https://vega.github.io/vega-lite/docs/selection.html#project) in the documentation.\",\n \"items\": {\n \"$ref\": \"#/definitions/SingleDefUnitChannel\"\n },\n \"type\": \"array\"\n },\n \"fields\": {\n \"description\": \"An array of field names whose values must match for a data tuple to fall within the selection.\\n\\n__See also:__ The [projection with `encodings` and `fields` section](https://vega.github.io/vega-lite/docs/selection.html#project) in the documentation.\",\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"type\": \"array\"\n },\n \"mark\": {\n \"$ref\": \"#/definitions/BrushConfig\",\n \"description\": \"An interval selection also adds a rectangle mark to depict the extents of the interval. The `mark` property can be used to customize the appearance of the mark.\\n\\n__See also:__ [`mark` examples](https://vega.github.io/vega-lite/docs/selection.html#mark) in the documentation.\"\n },\n \"on\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Stream\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"A [Vega event stream](https://vega.github.io/vega/docs/event-streams/) (object or selector) that triggers the selection. For interval selections, the event stream must specify a [start and end](https://vega.github.io/vega/docs/event-streams/#between-filters).\\n\\n__See also:__ [`on` examples](https://vega.github.io/vega-lite/docs/selection.html#on) in the documentation.\"\n },\n \"resolve\": {\n \"$ref\": \"#/definitions/SelectionResolution\",\n \"description\": \"With layered and multi-view displays, a strategy that determines how selections' data queries are resolved when applied in a filter transform, conditional encoding rule, or scale domain.\\n\\nOne of:\\n- `\\\"global\\\"` -- only one brush exists for the entire SPLOM. When the user begins to drag, any previous brushes are cleared, and a new one is constructed.\\n- `\\\"union\\\"` -- each cell contains its own brush, and points are highlighted if they lie within _any_ of these individual brushes.\\n- `\\\"intersect\\\"` -- each cell contains its own brush, and points are highlighted only if they fall within _all_ of these individual brushes.\\n\\n__Default value:__ `global`.\\n\\n__See also:__ [`resolve` examples](https://vega.github.io/vega-lite/docs/selection.html#resolve) in the documentation.\"\n },\n \"translate\": {\n \"description\": \"When truthy, allows a user to interactively move an interval selection back-and-forth. Can be `true`, `false` (to disable panning), or a [Vega event stream definition](https://vega.github.io/vega/docs/event-streams/) which must include a start and end event to trigger continuous panning. Discrete panning (e.g., pressing the left/right arrow keys) will be supported in future versions.\\n\\n__Default value:__ `true`, which corresponds to `[pointerdown, window:pointerup] > window:pointermove!`. This default allows users to clicks and drags within an interval selection to reposition it.\\n\\n__See also:__ [`translate` examples](https://vega.github.io/vega-lite/docs/selection.html#translate) in the documentation.\",\n \"type\": [\n \"string\",\n \"boolean\"\n ]\n },\n \"type\": {\n \"const\": \"interval\",\n \"description\": \"Determines the default event processing and data query for the selection. Vega-Lite currently supports two selection types:\\n\\n- `\\\"point\\\"` -- to select multiple discrete data values; the first value is selected on `click` and additional values toggled on shift-click.\\n- `\\\"interval\\\"` -- to select a continuous range of data values on `drag`.\",\n \"type\": \"string\"\n },\n \"zoom\": {\n \"description\": \"When truthy, allows a user to interactively resize an interval selection. Can be `true`, `false` (to disable zooming), or a [Vega event stream definition](https://vega.github.io/vega/docs/event-streams/). Currently, only `wheel` events are supported, but custom event streams can still be used to specify filters, debouncing, and throttling. Future versions will expand the set of events that can trigger this transformation.\\n\\n__Default value:__ `true`, which corresponds to `wheel!`. This default allows users to use the mouse wheel to resize an interval selection.\\n\\n__See also:__ [`zoom` examples](https://vega.github.io/vega-lite/docs/selection.html#zoom) in the documentation.\",\n \"type\": [\n \"string\",\n \"boolean\"\n ]\n }\n },\n \"required\": [\n \"type\"\n ],\n \"type\": \"object\"\n },\n \"IntervalSelectionConfigWithoutType\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"clear\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Stream\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"boolean\"\n }\n ],\n \"description\": \"Clears the selection, emptying it of all values. This property can be a [Event Stream](https://vega.github.io/vega/docs/event-streams/) or `false` to disable clear.\\n\\n__Default value:__ `dblclick`.\\n\\n__See also:__ [`clear` examples ](https://vega.github.io/vega-lite/docs/selection.html#clear) in the documentation.\"\n },\n \"encodings\": {\n \"description\": \"An array of encoding channels. The corresponding data field values must match for a data tuple to fall within the selection.\\n\\n__See also:__ The [projection with `encodings` and `fields` section](https://vega.github.io/vega-lite/docs/selection.html#project) in the documentation.\",\n \"items\": {\n \"$ref\": \"#/definitions/SingleDefUnitChannel\"\n },\n \"type\": \"array\"\n },\n \"fields\": {\n \"description\": \"An array of field names whose values must match for a data tuple to fall within the selection.\\n\\n__See also:__ The [projection with `encodings` and `fields` section](https://vega.github.io/vega-lite/docs/selection.html#project) in the documentation.\",\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"type\": \"array\"\n },\n \"mark\": {\n \"$ref\": \"#/definitions/BrushConfig\",\n \"description\": \"An interval selection also adds a rectangle mark to depict the extents of the interval. The `mark` property can be used to customize the appearance of the mark.\\n\\n__See also:__ [`mark` examples](https://vega.github.io/vega-lite/docs/selection.html#mark) in the documentation.\"\n },\n \"on\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Stream\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"A [Vega event stream](https://vega.github.io/vega/docs/event-streams/) (object or selector) that triggers the selection. For interval selections, the event stream must specify a [start and end](https://vega.github.io/vega/docs/event-streams/#between-filters).\\n\\n__See also:__ [`on` examples](https://vega.github.io/vega-lite/docs/selection.html#on) in the documentation.\"\n },\n \"resolve\": {\n \"$ref\": \"#/definitions/SelectionResolution\",\n \"description\": \"With layered and multi-view displays, a strategy that determines how selections' data queries are resolved when applied in a filter transform, conditional encoding rule, or scale domain.\\n\\nOne of:\\n- `\\\"global\\\"` -- only one brush exists for the entire SPLOM. When the user begins to drag, any previous brushes are cleared, and a new one is constructed.\\n- `\\\"union\\\"` -- each cell contains its own brush, and points are highlighted if they lie within _any_ of these individual brushes.\\n- `\\\"intersect\\\"` -- each cell contains its own brush, and points are highlighted only if they fall within _all_ of these individual brushes.\\n\\n__Default value:__ `global`.\\n\\n__See also:__ [`resolve` examples](https://vega.github.io/vega-lite/docs/selection.html#resolve) in the documentation.\"\n },\n \"translate\": {\n \"description\": \"When truthy, allows a user to interactively move an interval selection back-and-forth. Can be `true`, `false` (to disable panning), or a [Vega event stream definition](https://vega.github.io/vega/docs/event-streams/) which must include a start and end event to trigger continuous panning. Discrete panning (e.g., pressing the left/right arrow keys) will be supported in future versions.\\n\\n__Default value:__ `true`, which corresponds to `[pointerdown, window:pointerup] > window:pointermove!`. This default allows users to clicks and drags within an interval selection to reposition it.\\n\\n__See also:__ [`translate` examples](https://vega.github.io/vega-lite/docs/selection.html#translate) in the documentation.\",\n \"type\": [\n \"string\",\n \"boolean\"\n ]\n },\n \"zoom\": {\n \"description\": \"When truthy, allows a user to interactively resize an interval selection. Can be `true`, `false` (to disable zooming), or a [Vega event stream definition](https://vega.github.io/vega/docs/event-streams/). Currently, only `wheel` events are supported, but custom event streams can still be used to specify filters, debouncing, and throttling. Future versions will expand the set of events that can trigger this transformation.\\n\\n__Default value:__ `true`, which corresponds to `wheel!`. This default allows users to use the mouse wheel to resize an interval selection.\\n\\n__See also:__ [`zoom` examples](https://vega.github.io/vega-lite/docs/selection.html#zoom) in the documentation.\",\n \"type\": [\n \"string\",\n \"boolean\"\n ]\n }\n },\n \"type\": \"object\"\n },\n \"JoinAggregateFieldDef\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"as\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"The output name for the join aggregate operation.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"The data field for which to compute the aggregate function. This can be omitted for functions that do not operate over a field such as `\\\"count\\\"`.\"\n },\n \"op\": {\n \"$ref\": \"#/definitions/AggregateOp\",\n \"description\": \"The aggregation operation to apply (e.g., `\\\"sum\\\"`, `\\\"average\\\"` or `\\\"count\\\"`). See the list of all supported operations [here](https://vega.github.io/vega-lite/docs/aggregate.html#ops).\"\n }\n },\n \"required\": [\n \"op\",\n \"as\"\n ],\n \"type\": \"object\"\n },\n \"JoinAggregateTransform\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"groupby\": {\n \"description\": \"The data fields for partitioning the data objects into separate groups. If unspecified, all data points will be in a single group.\",\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"type\": \"array\"\n },\n \"joinaggregate\": {\n \"description\": \"The definition of the fields in the join aggregate, and what calculations to use.\",\n \"items\": {\n \"$ref\": \"#/definitions/JoinAggregateFieldDef\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"joinaggregate\"\n ],\n \"type\": \"object\"\n },\n \"JsonDataFormat\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"parse\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Parse\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"If set to `null`, disable type inference based on the spec and only use type inference based on the data. Alternatively, a parsing directive object can be provided for explicit data types. Each property of the object corresponds to a field name, and the value to the desired data type (one of `\\\"number\\\"`, `\\\"boolean\\\"`, `\\\"date\\\"`, or null (do not parse the field)). For example, `\\\"parse\\\": {\\\"modified_on\\\": \\\"date\\\"}` parses the `modified_on` field in each input record a Date value.\\n\\nFor `\\\"date\\\"`, we parse data based using JavaScript's [`Date.parse()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/parse). For Specific date formats can be provided (e.g., `{foo: \\\"date:'%m%d%Y'\\\"}`), using the [d3-time-format syntax](https://github.com/d3/d3-time-format#locale_format). UTC date format parsing is supported similarly (e.g., `{foo: \\\"utc:'%m%d%Y'\\\"}`). See more about [UTC time](https://vega.github.io/vega-lite/docs/timeunit.html#utc)\"\n },\n \"property\": {\n \"description\": \"The JSON property containing the desired data. This parameter can be used when the loaded JSON file may have surrounding structure or meta-data. For example `\\\"property\\\": \\\"values.features\\\"` is equivalent to retrieving `json.values.features` from the loaded JSON object.\",\n \"type\": \"string\"\n },\n \"type\": {\n \"const\": \"json\",\n \"description\": \"Type of input data: `\\\"json\\\"`, `\\\"csv\\\"`, `\\\"tsv\\\"`, `\\\"dsv\\\"`.\\n\\n__Default value:__ The default format type is determined by the extension of the file URL. If no extension is detected, `\\\"json\\\"` will be used by default.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"LabelOverlap\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"const\": \"parity\",\n \"type\": \"string\"\n },\n {\n \"const\": \"greedy\",\n \"type\": \"string\"\n }\n ]\n },\n \"LatLongDef\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LatLongFieldDef\"\n },\n {\n \"$ref\": \"#/definitions/DatumDef\"\n }\n ]\n },\n \"LatLongFieldDef\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\",\n \"type\": \"null\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"const\": \"quantitative\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"LayerRepeatMapping\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"column\": {\n \"description\": \"An array of fields to be repeated horizontally.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"layer\": {\n \"description\": \"An array of fields to be repeated as layers.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"row\": {\n \"description\": \"An array of fields to be repeated vertically.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"layer\"\n ],\n \"type\": \"object\"\n },\n \"LayerRepeatSpec\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"align\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LayoutAlign\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"The alignment to apply to grid rows and columns. The supported string values are `\\\"all\\\"`, `\\\"each\\\"`, and `\\\"none\\\"`.\\n\\n- For `\\\"none\\\"`, a flow layout will be used, in which adjacent subviews are simply placed one after the other.\\n- For `\\\"each\\\"`, subviews will be aligned into a clean grid structure, but each row or column may be of variable size.\\n- For `\\\"all\\\"`, subviews will be aligned and each row or column will be sized identically based on the maximum observed size. String values for this property will be applied to both grid rows and columns.\\n\\nAlternatively, an object value of the form `{\\\"row\\\": string, \\\"column\\\": string}` can be used to supply different alignments for rows and columns.\\n\\n__Default value:__ `\\\"all\\\"`.\"\n },\n \"bounds\": {\n \"description\": \"The bounds calculation method to use for determining the extent of a sub-plot. One of `full` (the default) or `flush`.\\n\\n- If set to `full`, the entire calculated bounds (including axes, title, and legend) will be used.\\n- If set to `flush`, only the specified width and height values for the sub-view will be used. The `flush` setting can be useful when attempting to place sub-plots without axes or legends into a uniform grid structure.\\n\\n__Default value:__ `\\\"full\\\"`\",\n \"enum\": [\n \"full\",\n \"flush\"\n ],\n \"type\": \"string\"\n },\n \"center\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"Boolean flag indicating if subviews should be centered relative to their respective rows or columns.\\n\\nAn object value of the form `{\\\"row\\\": boolean, \\\"column\\\": boolean}` can be used to supply different centering values for rows and columns.\\n\\n__Default value:__ `false`\"\n },\n \"columns\": {\n \"description\": \"The number of columns to include in the view composition layout.\\n\\n__Default value__: `undefined` -- An infinite number of columns (a single row) will be assumed. This is equivalent to `hconcat` (for `concat`) and to using the `column` channel (for `facet` and `repeat`).\\n\\n__Note__:\\n\\n1) This property is only for:\\n- the general (wrappable) `concat` operator (not `hconcat`/`vconcat`)\\n- the `facet` and `repeat` operator with one field/repetition definition (without row/column nesting)\\n\\n2) Setting the `columns` to `1` is equivalent to `vconcat` (for `concat`) and to using the `row` channel (for `facet` and `repeat`).\",\n \"type\": \"number\"\n },\n \"data\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Data\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object describing the data source. Set to `null` to ignore the parent's data source. If no data is set, it is derived from the parent.\"\n },\n \"description\": {\n \"description\": \"Description of this mark for commenting purpose.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the visualization for later reference.\",\n \"type\": \"string\"\n },\n \"repeat\": {\n \"$ref\": \"#/definitions/LayerRepeatMapping\",\n \"description\": \"Definition for fields to be repeated. One of: 1) An array of fields to be repeated. If `\\\"repeat\\\"` is an array, the field can be referred to as `{\\\"repeat\\\": \\\"repeat\\\"}`. The repeated views are laid out in a wrapped row. You can set the number of columns to control the wrapping. 2) An object that maps `\\\"row\\\"` and/or `\\\"column\\\"` to the listed fields to be repeated along the particular orientations. The objects `{\\\"repeat\\\": \\\"row\\\"}` and `{\\\"repeat\\\": \\\"column\\\"}` can be used to refer to the repeated field respectively.\"\n },\n \"resolve\": {\n \"$ref\": \"#/definitions/Resolve\",\n \"description\": \"Scale, axis, and legend resolutions for view composition specifications.\"\n },\n \"spacing\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"The spacing in pixels between sub-views of the composition operator. An object of the form `{\\\"row\\\": number, \\\"column\\\": number}` can be used to set different spacing values for rows and columns.\\n\\n__Default value__: Depends on `\\\"spacing\\\"` property of [the view composition configuration](https://vega.github.io/vega-lite/docs/config.html#view-config) (`20` by default)\"\n },\n \"spec\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LayerSpec\"\n },\n {\n \"$ref\": \"#/definitions/UnitSpecWithFrame\"\n }\n ],\n \"description\": \"A specification of the view that gets repeated.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"$ref\": \"#/definitions/TitleParams\"\n }\n ],\n \"description\": \"Title for the plot.\"\n },\n \"transform\": {\n \"description\": \"An array of data transformations such as filter and new field calculation.\",\n \"items\": {\n \"$ref\": \"#/definitions/Transform\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"repeat\",\n \"spec\"\n ],\n \"type\": \"object\"\n },\n \"LayerSpec\": {\n \"additionalProperties\": false,\n \"description\": \"A full layered plot specification, which may contains `encoding` and `projection` properties that will be applied to underlying unit (single-view) specifications.\",\n \"properties\": {\n \"data\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Data\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object describing the data source. Set to `null` to ignore the parent's data source. If no data is set, it is derived from the parent.\"\n },\n \"description\": {\n \"description\": \"Description of this mark for commenting purpose.\",\n \"type\": \"string\"\n },\n \"encoding\": {\n \"$ref\": \"#/definitions/SharedEncoding\",\n \"description\": \"A shared key-value mapping between encoding channels and definition of fields in the underlying layers.\"\n },\n \"height\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"container\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Step\"\n }\n ],\n \"description\": \"The height of a visualization.\\n\\n- For a plot with a continuous y-field, height should be a number.\\n- For a plot with either a discrete y-field or no y-field, height can be either a number indicating a fixed height or an object in the form of `{step: number}` defining the height per discrete step. (No y-field is equivalent to having one discrete step.)\\n- To enable responsive sizing on height, it should be set to `\\\"container\\\"`.\\n\\n__Default value:__ Based on `config.view.continuousHeight` for a plot with a continuous y-field and `config.view.discreteHeight` otherwise.\\n\\n__Note:__ For plots with [`row` and `column` channels](https://vega.github.io/vega-lite/docs/encoding.html#facet), this represents the height of a single view and the `\\\"container\\\"` option cannot be used.\\n\\n__See also:__ [`height`](https://vega.github.io/vega-lite/docs/size.html) documentation.\"\n },\n \"layer\": {\n \"description\": \"Layer or single view specifications to be layered.\\n\\n__Note__: Specifications inside `layer` cannot use `row` and `column` channels as layering facet specifications is not allowed. Instead, use the [facet operator](https://vega.github.io/vega-lite/docs/facet.html) and place a layer inside a facet.\",\n \"items\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LayerSpec\"\n },\n {\n \"$ref\": \"#/definitions/UnitSpec\"\n }\n ]\n },\n \"type\": \"array\"\n },\n \"name\": {\n \"description\": \"Name of the visualization for later reference.\",\n \"type\": \"string\"\n },\n \"projection\": {\n \"$ref\": \"#/definitions/Projection\",\n \"description\": \"An object defining properties of the geographic projection shared by underlying layers.\"\n },\n \"resolve\": {\n \"$ref\": \"#/definitions/Resolve\",\n \"description\": \"Scale, axis, and legend resolutions for view composition specifications.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"$ref\": \"#/definitions/TitleParams\"\n }\n ],\n \"description\": \"Title for the plot.\"\n },\n \"transform\": {\n \"description\": \"An array of data transformations such as filter and new field calculation.\",\n \"items\": {\n \"$ref\": \"#/definitions/Transform\"\n },\n \"type\": \"array\"\n },\n \"view\": {\n \"$ref\": \"#/definitions/ViewBackground\",\n \"description\": \"An object defining the view background's fill and stroke.\\n\\n__Default value:__ none (transparent)\"\n },\n \"width\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"container\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Step\"\n }\n ],\n \"description\": \"The width of a visualization.\\n\\n- For a plot with a continuous x-field, width should be a number.\\n- For a plot with either a discrete x-field or no x-field, width can be either a number indicating a fixed width or an object in the form of `{step: number}` defining the width per discrete step. (No x-field is equivalent to having one discrete step.)\\n- To enable responsive sizing on width, it should be set to `\\\"container\\\"`.\\n\\n__Default value:__ Based on `config.view.continuousWidth` for a plot with a continuous x-field and `config.view.discreteWidth` otherwise.\\n\\n__Note:__ For plots with [`row` and `column` channels](https://vega.github.io/vega-lite/docs/encoding.html#facet), this represents the width of a single view and the `\\\"container\\\"` option cannot be used.\\n\\n__See also:__ [`width`](https://vega.github.io/vega-lite/docs/size.html) documentation.\"\n }\n },\n \"required\": [\n \"layer\"\n ],\n \"type\": \"object\"\n },\n \"LayoutAlign\": {\n \"enum\": [\n \"all\",\n \"each\",\n \"none\"\n ],\n \"type\": \"string\"\n },\n \"Legend\": {\n \"additionalProperties\": false,\n \"description\": \"Properties of a legend or boolean flag for determining whether to show it.\",\n \"properties\": {\n \"aria\": {\n \"anyOf\": [\n {\n \"description\": \"A boolean flag indicating if [ARIA attributes](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) should be included (SVG output only). If `false`, the \\\"aria-hidden\\\" attribute will be set on the output SVG group, removing the legend from the ARIA accessibility tree.\\n\\n__Default value:__ `true`\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"clipHeight\": {\n \"anyOf\": [\n {\n \"description\": \"The height in pixels to clip symbol legend entries and limit their size.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"columnPadding\": {\n \"anyOf\": [\n {\n \"description\": \"The horizontal padding in pixels between symbol legend entries.\\n\\n__Default value:__ `10`.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"columns\": {\n \"anyOf\": [\n {\n \"description\": \"The number of columns in which to arrange symbol legend entries. A value of `0` or lower indicates a single row with one column per entry.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadius\": {\n \"anyOf\": [\n {\n \"description\": \"Corner radius for the full legend.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"description\": {\n \"anyOf\": [\n {\n \"description\": \"A text description of this legend for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If the `aria` property is true, for SVG output the [\\\"aria-label\\\" attribute](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/ARIA_Techniques/Using_the_aria-label_attribute) will be set to this description. If the description is unspecified it will be automatically generated.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"direction\": {\n \"$ref\": \"#/definitions/Orientation\",\n \"description\": \"The direction of the legend, one of `\\\"vertical\\\"` or `\\\"horizontal\\\"`.\\n\\n__Default value:__\\n- For top-/bottom-`orient`ed legends, `\\\"horizontal\\\"`\\n- For left-/right-`orient`ed legends, `\\\"vertical\\\"`\\n- For top/bottom-left/right-`orient`ed legends, `\\\"horizontal\\\"` for gradient legends and `\\\"vertical\\\"` for symbol legends.\"\n },\n \"fillColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"Background fill color for the full legend.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"format\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Dict\"\n }\n ],\n \"description\": \"When used with the default `\\\"number\\\"` and `\\\"time\\\"` format type, the text formatting pattern for labels of guides (axes, legends, headers) and text marks.\\n\\n- If the format type is `\\\"number\\\"` (e.g., for quantitative fields), this is D3's [number format pattern](https://github.com/d3/d3-format#locale_format).\\n- If the format type is `\\\"time\\\"` (e.g., for temporal fields), this is D3's [time format pattern](https://github.com/d3/d3-time-format#locale_format).\\n\\nSee the [format documentation](https://vega.github.io/vega-lite/docs/format.html) for more examples.\\n\\nWhen used with a [custom `formatType`](https://vega.github.io/vega-lite/docs/config.html#custom-format-type), this value will be passed as `format` alongside `datum.value` to the registered function.\\n\\n__Default value:__ Derived from [numberFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for number format and from [timeFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for time format.\"\n },\n \"formatType\": {\n \"description\": \"The format type for labels. One of `\\\"number\\\"`, `\\\"time\\\"`, or a [registered custom format type](https://vega.github.io/vega-lite/docs/config.html#custom-format-type).\\n\\n__Default value:__\\n- `\\\"time\\\"` for temporal fields and ordinal and nominal fields with `timeUnit`.\\n- `\\\"number\\\"` for quantitative fields as well as ordinal and nominal fields without `timeUnit`.\",\n \"type\": \"string\"\n },\n \"gradientLength\": {\n \"anyOf\": [\n {\n \"description\": \"The length in pixels of the primary axis of a color gradient. This value corresponds to the height of a vertical gradient or the width of a horizontal gradient.\\n\\n__Default value:__ `200`.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"gradientOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"Opacity of the color gradient.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"gradientStrokeColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"The color of the gradient stroke, can be in hex color code or regular color name.\\n\\n__Default value:__ `\\\"lightGray\\\"`.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"gradientStrokeWidth\": {\n \"anyOf\": [\n {\n \"description\": \"The width of the gradient stroke, in pixels.\\n\\n__Default value:__ `0`.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"gradientThickness\": {\n \"anyOf\": [\n {\n \"description\": \"The thickness in pixels of the color gradient. This value corresponds to the width of a vertical gradient or the height of a horizontal gradient.\\n\\n__Default value:__ `16`.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"gridAlign\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LayoutAlign\",\n \"description\": \"The alignment to apply to symbol legends rows and columns. The supported string values are `\\\"all\\\"`, `\\\"each\\\"` (the default), and `none`. For more information, see the [grid layout documentation](https://vega.github.io/vega/docs/layout).\\n\\n__Default value:__ `\\\"each\\\"`.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelAlign\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Align\",\n \"description\": \"The alignment of the legend label, can be left, center, or right.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelBaseline\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextBaseline\",\n \"description\": \"The position of the baseline of legend label, can be `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`, or `\\\"alphabetic\\\"`.\\n\\n__Default value:__ `\\\"middle\\\"`.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"The color of the legend label, can be in hex color code or regular color name.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelExpr\": {\n \"description\": \"[Vega expression](https://vega.github.io/vega/docs/expressions/) for customizing labels.\\n\\n__Note:__ The label text and value can be assessed via the `label` and `value` properties of the legend's backing `datum` object.\",\n \"type\": \"string\"\n },\n \"labelFont\": {\n \"anyOf\": [\n {\n \"description\": \"The font of the legend label.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelFontSize\": {\n \"anyOf\": [\n {\n \"description\": \"The font size of legend label.\\n\\n__Default value:__ `10`.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelFontStyle\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\",\n \"description\": \"The font style of legend label.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelFontWeight\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\",\n \"description\": \"The font weight of legend label.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelLimit\": {\n \"anyOf\": [\n {\n \"description\": \"Maximum allowed pixel width of legend tick labels.\\n\\n__Default value:__ `160`.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The offset of the legend label.\\n\\n__Default value:__ `4`.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"Opacity of labels.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelOverlap\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LabelOverlap\",\n \"description\": \"The strategy to use for resolving overlap of labels in gradient legends. If `false`, no overlap reduction is attempted. If set to `true` (default) or `\\\"parity\\\"`, a strategy of removing every other label is used. If set to `\\\"greedy\\\"`, a linear scan of the labels is performed, removing any label that overlaps with the last visible label (this often works better for log-scaled axes).\\n\\n__Default value:__ `true`.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelPadding\": {\n \"anyOf\": [\n {\n \"description\": \"Padding in pixels between the legend and legend labels.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelSeparation\": {\n \"anyOf\": [\n {\n \"description\": \"The minimum separation that must be between label bounding boxes for them to be considered non-overlapping (default `0`). This property is ignored if *labelOverlap* resolution is not enabled.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"legendX\": {\n \"anyOf\": [\n {\n \"description\": \"Custom x-position for legend with orient \\\"none\\\".\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"legendY\": {\n \"anyOf\": [\n {\n \"description\": \"Custom y-position for legend with orient \\\"none\\\".\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"offset\": {\n \"anyOf\": [\n {\n \"description\": \"The offset in pixels by which to displace the legend from the data rectangle and axes.\\n\\n__Default value:__ `18`.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"orient\": {\n \"$ref\": \"#/definitions/LegendOrient\",\n \"description\": \"The orientation of the legend, which determines how the legend is positioned within the scene. One of `\\\"left\\\"`, `\\\"right\\\"`, `\\\"top\\\"`, `\\\"bottom\\\"`, `\\\"top-left\\\"`, `\\\"top-right\\\"`, `\\\"bottom-left\\\"`, `\\\"bottom-right\\\"`, `\\\"none\\\"`.\\n\\n__Default value:__ `\\\"right\\\"`\"\n },\n \"padding\": {\n \"anyOf\": [\n {\n \"description\": \"The padding between the border and content of the legend group.\\n\\n__Default value:__ `0`.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"rowPadding\": {\n \"anyOf\": [\n {\n \"description\": \"The vertical padding in pixels between symbol legend entries.\\n\\n__Default value:__ `2`.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"Border stroke color for the full legend.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"symbolDash\": {\n \"anyOf\": [\n {\n \"description\": \"An array of alternating [stroke, space] lengths for dashed symbol strokes.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"symbolDashOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The pixel offset at which to start drawing with the symbol stroke dash array.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"symbolFillColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"The color of the legend symbol,\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"symbolLimit\": {\n \"anyOf\": [\n {\n \"description\": \"The maximum number of allowed entries for a symbol legend. Additional entries will be dropped.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"symbolOffset\": {\n \"anyOf\": [\n {\n \"description\": \"Horizontal pixel offset for legend symbols.\\n\\n__Default value:__ `0`.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"symbolOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"Opacity of the legend symbols.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"symbolSize\": {\n \"anyOf\": [\n {\n \"description\": \"The size of the legend symbol, in pixels.\\n\\n__Default value:__ `100`.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"symbolStrokeColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"Stroke color for legend symbols.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"symbolStrokeWidth\": {\n \"anyOf\": [\n {\n \"description\": \"The width of the symbol's stroke.\\n\\n__Default value:__ `1.5`.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"symbolType\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SymbolShape\",\n \"description\": \"The symbol shape. One of the plotting shapes `circle` (default), `square`, `cross`, `diamond`, `triangle-up`, `triangle-down`, `triangle-right`, or `triangle-left`, the line symbol `stroke`, or one of the centered directional shapes `arrow`, `wedge`, or `triangle`. Alternatively, a custom [SVG path string](https://developer.mozilla.org/en-US/docs/Web/SVG/Tutorial/Paths) can be provided. For correct sizing, custom shape paths should be defined within a square bounding box with coordinates ranging from -1 to 1 along both the x and y dimensions.\\n\\n__Default value:__ `\\\"circle\\\"`.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"tickCount\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TickCount\",\n \"description\": \"The desired number of tick values for quantitative legends.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"tickMinStep\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The minimum desired step between legend ticks, in terms of scale domain values. For example, a value of `1` indicates that ticks should not be less than 1 unit apart. If `tickMinStep` is specified, the `tickCount` value will be adjusted, if necessary, to enforce the minimum step value.\\n\\n__Default value__: `undefined`\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"titleAlign\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Align\",\n \"description\": \"Horizontal text alignment for legend titles.\\n\\n__Default value:__ `\\\"left\\\"`.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleAnchor\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TitleAnchor\",\n \"description\": \"Text anchor position for placing legend titles.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleBaseline\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextBaseline\",\n \"description\": \"Vertical text baseline for legend titles. One of `\\\"alphabetic\\\"` (default), `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`, `\\\"line-top\\\"`, or `\\\"line-bottom\\\"`. The `\\\"line-top\\\"` and `\\\"line-bottom\\\"` values operate similarly to `\\\"top\\\"` and `\\\"bottom\\\"`, but are calculated relative to the *lineHeight* rather than *fontSize* alone.\\n\\n__Default value:__ `\\\"top\\\"`.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"The color of the legend title, can be in hex color code or regular color name.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleFont\": {\n \"anyOf\": [\n {\n \"description\": \"The font of the legend title.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleFontSize\": {\n \"anyOf\": [\n {\n \"description\": \"The font size of the legend title.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleFontStyle\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\",\n \"description\": \"The font style of the legend title.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleFontWeight\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\",\n \"description\": \"The font weight of the legend title. This can be either a string (e.g `\\\"bold\\\"`, `\\\"normal\\\"`) or a number (`100`, `200`, `300`, ..., `900` where `\\\"normal\\\"` = `400` and `\\\"bold\\\"` = `700`).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleLimit\": {\n \"anyOf\": [\n {\n \"description\": \"Maximum allowed pixel width of legend titles.\\n\\n__Default value:__ `180`.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleLineHeight\": {\n \"anyOf\": [\n {\n \"description\": \"Line height in pixels for multi-line title text or title text with `\\\"line-top\\\"` or `\\\"line-bottom\\\"` baseline.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"Opacity of the legend title.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleOrient\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Orient\",\n \"description\": \"Orientation of the legend title.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titlePadding\": {\n \"anyOf\": [\n {\n \"description\": \"The padding, in pixels, between title and legend.\\n\\n__Default value:__ `5`.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"type\": {\n \"description\": \"The type of the legend. Use `\\\"symbol\\\"` to create a discrete legend and `\\\"gradient\\\"` for a continuous color gradient.\\n\\n__Default value:__ `\\\"gradient\\\"` for non-binned quantitative fields and temporal fields; `\\\"symbol\\\"` otherwise.\",\n \"enum\": [\n \"symbol\",\n \"gradient\"\n ],\n \"type\": \"string\"\n },\n \"values\": {\n \"anyOf\": [\n {\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n {\n \"items\": {\n \"type\": \"boolean\"\n },\n \"type\": \"array\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/DateTime\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Explicitly set the visible legend values.\"\n },\n \"zindex\": {\n \"description\": \"A non-negative integer indicating the z-index of the legend. If zindex is 0, legend should be drawn behind all chart elements. To put them in front, use zindex = 1.\",\n \"minimum\": 0,\n \"type\": \"number\"\n }\n },\n \"type\": \"object\"\n },\n \"LegendBinding\": {\n \"anyOf\": [\n {\n \"const\": \"legend\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/LegendStreamBinding\"\n }\n ]\n },\n \"LegendConfig\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aria\": {\n \"anyOf\": [\n {\n \"description\": \"A boolean flag indicating if [ARIA attributes](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) should be included (SVG output only). If `false`, the \\\"aria-hidden\\\" attribute will be set on the output SVG group, removing the legend from the ARIA accessibility tree.\\n\\n__Default value:__ `true`\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"clipHeight\": {\n \"anyOf\": [\n {\n \"description\": \"The height in pixels to clip symbol legend entries and limit their size.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"columnPadding\": {\n \"anyOf\": [\n {\n \"description\": \"The horizontal padding in pixels between symbol legend entries.\\n\\n__Default value:__ `10`.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"columns\": {\n \"anyOf\": [\n {\n \"description\": \"The number of columns in which to arrange symbol legend entries. A value of `0` or lower indicates a single row with one column per entry.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadius\": {\n \"anyOf\": [\n {\n \"description\": \"Corner radius for the full legend.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"description\": {\n \"anyOf\": [\n {\n \"description\": \"A text description of this legend for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If the `aria` property is true, for SVG output the [\\\"aria-label\\\" attribute](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/ARIA_Techniques/Using_the_aria-label_attribute) will be set to this description. If the description is unspecified it will be automatically generated.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"direction\": {\n \"$ref\": \"#/definitions/Orientation\",\n \"description\": \"The direction of the legend, one of `\\\"vertical\\\"` or `\\\"horizontal\\\"`.\\n\\n__Default value:__\\n- For top-/bottom-`orient`ed legends, `\\\"horizontal\\\"`\\n- For left-/right-`orient`ed legends, `\\\"vertical\\\"`\\n- For top/bottom-left/right-`orient`ed legends, `\\\"horizontal\\\"` for gradient legends and `\\\"vertical\\\"` for symbol legends.\"\n },\n \"disable\": {\n \"description\": \"Disable legend by default\",\n \"type\": \"boolean\"\n },\n \"fillColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"Background fill color for the full legend.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"gradientDirection\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Orientation\",\n \"description\": \"The default direction (`\\\"horizontal\\\"` or `\\\"vertical\\\"`) for gradient legends.\\n\\n__Default value:__ `\\\"vertical\\\"`.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"gradientHorizontalMaxLength\": {\n \"description\": \"Max legend length for a horizontal gradient when `config.legend.gradientLength` is undefined.\\n\\n__Default value:__ `200`\",\n \"type\": \"number\"\n },\n \"gradientHorizontalMinLength\": {\n \"description\": \"Min legend length for a horizontal gradient when `config.legend.gradientLength` is undefined.\\n\\n__Default value:__ `100`\",\n \"type\": \"number\"\n },\n \"gradientLabelLimit\": {\n \"anyOf\": [\n {\n \"description\": \"The maximum allowed length in pixels of color ramp gradient labels.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"gradientLabelOffset\": {\n \"anyOf\": [\n {\n \"description\": \"Vertical offset in pixels for color ramp gradient labels.\\n\\n__Default value:__ `2`.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"gradientLength\": {\n \"anyOf\": [\n {\n \"description\": \"The length in pixels of the primary axis of a color gradient. This value corresponds to the height of a vertical gradient or the width of a horizontal gradient.\\n\\n__Default value:__ `200`.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"gradientOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"Opacity of the color gradient.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"gradientStrokeColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"The color of the gradient stroke, can be in hex color code or regular color name.\\n\\n__Default value:__ `\\\"lightGray\\\"`.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"gradientStrokeWidth\": {\n \"anyOf\": [\n {\n \"description\": \"The width of the gradient stroke, in pixels.\\n\\n__Default value:__ `0`.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"gradientThickness\": {\n \"anyOf\": [\n {\n \"description\": \"The thickness in pixels of the color gradient. This value corresponds to the width of a vertical gradient or the height of a horizontal gradient.\\n\\n__Default value:__ `16`.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"gradientVerticalMaxLength\": {\n \"description\": \"Max legend length for a vertical gradient when `config.legend.gradientLength` is undefined.\\n\\n__Default value:__ `200`\",\n \"type\": \"number\"\n },\n \"gradientVerticalMinLength\": {\n \"description\": \"Min legend length for a vertical gradient when `config.legend.gradientLength` is undefined.\\n\\n__Default value:__ `100`\",\n \"type\": \"number\"\n },\n \"gridAlign\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LayoutAlign\",\n \"description\": \"The alignment to apply to symbol legends rows and columns. The supported string values are `\\\"all\\\"`, `\\\"each\\\"` (the default), and `none`. For more information, see the [grid layout documentation](https://vega.github.io/vega/docs/layout).\\n\\n__Default value:__ `\\\"each\\\"`.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelAlign\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Align\",\n \"description\": \"The alignment of the legend label, can be left, center, or right.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelBaseline\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextBaseline\",\n \"description\": \"The position of the baseline of legend label, can be `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`, or `\\\"alphabetic\\\"`.\\n\\n__Default value:__ `\\\"middle\\\"`.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"The color of the legend label, can be in hex color code or regular color name.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelFont\": {\n \"anyOf\": [\n {\n \"description\": \"The font of the legend label.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelFontSize\": {\n \"anyOf\": [\n {\n \"description\": \"The font size of legend label.\\n\\n__Default value:__ `10`.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelFontStyle\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\",\n \"description\": \"The font style of legend label.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelFontWeight\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\",\n \"description\": \"The font weight of legend label.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelLimit\": {\n \"anyOf\": [\n {\n \"description\": \"Maximum allowed pixel width of legend tick labels.\\n\\n__Default value:__ `160`.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The offset of the legend label.\\n\\n__Default value:__ `4`.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"Opacity of labels.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelOverlap\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LabelOverlap\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The strategy to use for resolving overlap of labels in gradient legends. If `false`, no overlap reduction is attempted. If set to `true` or `\\\"parity\\\"`, a strategy of removing every other label is used. If set to `\\\"greedy\\\"`, a linear scan of the labels is performed, removing any label that overlaps with the last visible label (this often works better for log-scaled axes).\\n\\n__Default value:__ `\\\"greedy\\\"` for `log scales otherwise `true`.\"\n },\n \"labelPadding\": {\n \"anyOf\": [\n {\n \"description\": \"Padding in pixels between the legend and legend labels.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"labelSeparation\": {\n \"anyOf\": [\n {\n \"description\": \"The minimum separation that must be between label bounding boxes for them to be considered non-overlapping (default `0`). This property is ignored if *labelOverlap* resolution is not enabled.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"layout\": {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n \"legendX\": {\n \"anyOf\": [\n {\n \"description\": \"Custom x-position for legend with orient \\\"none\\\".\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"legendY\": {\n \"anyOf\": [\n {\n \"description\": \"Custom y-position for legend with orient \\\"none\\\".\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"offset\": {\n \"anyOf\": [\n {\n \"description\": \"The offset in pixels by which to displace the legend from the data rectangle and axes.\\n\\n__Default value:__ `18`.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"orient\": {\n \"$ref\": \"#/definitions/LegendOrient\",\n \"description\": \"The orientation of the legend, which determines how the legend is positioned within the scene. One of `\\\"left\\\"`, `\\\"right\\\"`, `\\\"top\\\"`, `\\\"bottom\\\"`, `\\\"top-left\\\"`, `\\\"top-right\\\"`, `\\\"bottom-left\\\"`, `\\\"bottom-right\\\"`, `\\\"none\\\"`.\\n\\n__Default value:__ `\\\"right\\\"`\"\n },\n \"padding\": {\n \"anyOf\": [\n {\n \"description\": \"The padding between the border and content of the legend group.\\n\\n__Default value:__ `0`.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"rowPadding\": {\n \"anyOf\": [\n {\n \"description\": \"The vertical padding in pixels between symbol legend entries.\\n\\n__Default value:__ `2`.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"Border stroke color for the full legend.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeDash\": {\n \"anyOf\": [\n {\n \"description\": \"Border stroke dash pattern for the full legend.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeWidth\": {\n \"anyOf\": [\n {\n \"description\": \"Border stroke width for the full legend.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"symbolBaseFillColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"Default fill color for legend symbols. Only applied if there is no `\\\"fill\\\"` scale color encoding for the legend.\\n\\n__Default value:__ `\\\"transparent\\\"`.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"symbolBaseStrokeColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"Default stroke color for legend symbols. Only applied if there is no `\\\"fill\\\"` scale color encoding for the legend.\\n\\n__Default value:__ `\\\"gray\\\"`.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"symbolDash\": {\n \"anyOf\": [\n {\n \"description\": \"An array of alternating [stroke, space] lengths for dashed symbol strokes.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"symbolDashOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The pixel offset at which to start drawing with the symbol stroke dash array.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"symbolDirection\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Orientation\",\n \"description\": \"The default direction (`\\\"horizontal\\\"` or `\\\"vertical\\\"`) for symbol legends.\\n\\n__Default value:__ `\\\"vertical\\\"`.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"symbolFillColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"The color of the legend symbol,\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"symbolLimit\": {\n \"anyOf\": [\n {\n \"description\": \"The maximum number of allowed entries for a symbol legend. Additional entries will be dropped.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"symbolOffset\": {\n \"anyOf\": [\n {\n \"description\": \"Horizontal pixel offset for legend symbols.\\n\\n__Default value:__ `0`.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"symbolOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"Opacity of the legend symbols.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"symbolSize\": {\n \"anyOf\": [\n {\n \"description\": \"The size of the legend symbol, in pixels.\\n\\n__Default value:__ `100`.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"symbolStrokeColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"Stroke color for legend symbols.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"symbolStrokeWidth\": {\n \"anyOf\": [\n {\n \"description\": \"The width of the symbol's stroke.\\n\\n__Default value:__ `1.5`.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"symbolType\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SymbolShape\",\n \"description\": \"The symbol shape. One of the plotting shapes `circle` (default), `square`, `cross`, `diamond`, `triangle-up`, `triangle-down`, `triangle-right`, or `triangle-left`, the line symbol `stroke`, or one of the centered directional shapes `arrow`, `wedge`, or `triangle`. Alternatively, a custom [SVG path string](https://developer.mozilla.org/en-US/docs/Web/SVG/Tutorial/Paths) can be provided. For correct sizing, custom shape paths should be defined within a square bounding box with coordinates ranging from -1 to 1 along both the x and y dimensions.\\n\\n__Default value:__ `\\\"circle\\\"`.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"tickCount\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TickCount\",\n \"description\": \"The desired number of tick values for quantitative legends.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"title\": {\n \"description\": \"Set to null to disable title for the axis, legend, or header.\",\n \"type\": \"null\"\n },\n \"titleAlign\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Align\",\n \"description\": \"Horizontal text alignment for legend titles.\\n\\n__Default value:__ `\\\"left\\\"`.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleAnchor\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TitleAnchor\",\n \"description\": \"Text anchor position for placing legend titles.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleBaseline\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextBaseline\",\n \"description\": \"Vertical text baseline for legend titles. One of `\\\"alphabetic\\\"` (default), `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`, `\\\"line-top\\\"`, or `\\\"line-bottom\\\"`. The `\\\"line-top\\\"` and `\\\"line-bottom\\\"` values operate similarly to `\\\"top\\\"` and `\\\"bottom\\\"`, but are calculated relative to the *lineHeight* rather than *fontSize* alone.\\n\\n__Default value:__ `\\\"top\\\"`.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"The color of the legend title, can be in hex color code or regular color name.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleFont\": {\n \"anyOf\": [\n {\n \"description\": \"The font of the legend title.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleFontSize\": {\n \"anyOf\": [\n {\n \"description\": \"The font size of the legend title.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleFontStyle\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\",\n \"description\": \"The font style of the legend title.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleFontWeight\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\",\n \"description\": \"The font weight of the legend title. This can be either a string (e.g `\\\"bold\\\"`, `\\\"normal\\\"`) or a number (`100`, `200`, `300`, ..., `900` where `\\\"normal\\\"` = `400` and `\\\"bold\\\"` = `700`).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleLimit\": {\n \"anyOf\": [\n {\n \"description\": \"Maximum allowed pixel width of legend titles.\\n\\n__Default value:__ `180`.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleLineHeight\": {\n \"anyOf\": [\n {\n \"description\": \"Line height in pixels for multi-line title text or title text with `\\\"line-top\\\"` or `\\\"line-bottom\\\"` baseline.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"Opacity of the legend title.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titleOrient\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Orient\",\n \"description\": \"Orientation of the legend title.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"titlePadding\": {\n \"anyOf\": [\n {\n \"description\": \"The padding, in pixels, between title and legend.\\n\\n__Default value:__ `5`.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"unselectedOpacity\": {\n \"description\": \"The opacity of unselected legend entries.\\n\\n__Default value:__ 0.35.\",\n \"type\": \"number\"\n },\n \"zindex\": {\n \"anyOf\": [\n {\n \"description\": \"The integer z-index indicating the layering of the legend group relative to other axis, mark, and legend groups.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n }\n },\n \"type\": \"object\"\n },\n \"LegendOrient\": {\n \"enum\": [\n \"none\",\n \"left\",\n \"right\",\n \"top\",\n \"bottom\",\n \"top-left\",\n \"top-right\",\n \"bottom-left\",\n \"bottom-right\"\n ],\n \"type\": \"string\"\n },\n \"LegendResolveMap\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"angle\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"color\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"fill\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"fillOpacity\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"opacity\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"shape\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"size\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"stroke\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"strokeDash\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"strokeOpacity\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"strokeWidth\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n }\n },\n \"type\": \"object\"\n },\n \"LegendStreamBinding\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"legend\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Stream\"\n }\n ]\n }\n },\n \"required\": [\n \"legend\"\n ],\n \"type\": \"object\"\n },\n \"LineConfig\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"align\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Align\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The horizontal alignment of the text or ranged marks (area, bar, image, rect, rule). One of `\\\"left\\\"`, `\\\"right\\\"`, `\\\"center\\\"`.\\n\\n__Note:__ Expression reference is *not* supported for range marks.\"\n },\n \"angle\": {\n \"anyOf\": [\n {\n \"description\": \"The rotation angle of the text, in degrees.\",\n \"maximum\": 360,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"aria\": {\n \"anyOf\": [\n {\n \"description\": \"A boolean flag indicating if [ARIA attributes](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) should be included (SVG output only). If `false`, the \\\"aria-hidden\\\" attribute will be set on the output SVG element, removing the mark item from the ARIA accessibility tree.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"ariaRole\": {\n \"anyOf\": [\n {\n \"description\": \"Sets the type of user interface element of the mark item for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If specified, this property determines the \\\"role\\\" attribute. Warning: this property is experimental and may be changed in the future.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"ariaRoleDescription\": {\n \"anyOf\": [\n {\n \"description\": \"A human-readable, author-localized description for the role of the mark item for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If specified, this property determines the \\\"aria-roledescription\\\" attribute. Warning: this property is experimental and may be changed in the future.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"aspect\": {\n \"anyOf\": [\n {\n \"description\": \"Whether to keep aspect ratio of image marks.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"baseline\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextBaseline\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"For text marks, the vertical text baseline. One of `\\\"alphabetic\\\"` (default), `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`, `\\\"line-top\\\"`, `\\\"line-bottom\\\"`, or an expression reference that provides one of the valid values. The `\\\"line-top\\\"` and `\\\"line-bottom\\\"` values operate similarly to `\\\"top\\\"` and `\\\"bottom\\\"`, but are calculated relative to the `lineHeight` rather than `fontSize` alone.\\n\\nFor range marks, the vertical alignment of the marks. One of `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`.\\n\\n__Note:__ Expression reference is *not* supported for range marks.\"\n },\n \"blend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Blend\",\n \"description\": \"The color blend mode for drawing an item on its current background. Any valid [CSS mix-blend-mode](https://developer.mozilla.org/en-US/docs/Web/CSS/mix-blend-mode) value can be used.\\n\\n__Default value: `\\\"source-over\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"color\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default color.\\n\\n__Default value:__ ■ `\\\"#4682b4\\\"`\\n\\n__Note:__\\n- This property cannot be used in a [style config](https://vega.github.io/vega-lite/docs/mark.html#style-config).\\n- The `fill` and `stroke` properties have higher precedence than `color` and will override `color`.\"\n },\n \"cornerRadius\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles or arcs' corners.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusBottomLeft\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' bottom left corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusBottomRight\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' bottom right corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusTopLeft\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' top right corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusTopRight\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' top left corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cursor\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Cursor\",\n \"description\": \"The mouse cursor used over the mark. Any valid [CSS cursor type](https://developer.mozilla.org/en-US/docs/Web/CSS/cursor#Values) can be used.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"description\": {\n \"anyOf\": [\n {\n \"description\": \"A text description of the mark item for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If specified, this property determines the [\\\"aria-label\\\" attribute](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/ARIA_Techniques/Using_the_aria-label_attribute).\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"dir\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextDirection\",\n \"description\": \"The direction of the text. One of `\\\"ltr\\\"` (left-to-right) or `\\\"rtl\\\"` (right-to-left). This property determines on which side is truncated in response to the limit parameter.\\n\\n__Default value:__ `\\\"ltr\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"dx\": {\n \"anyOf\": [\n {\n \"description\": \"The horizontal offset, in pixels, between the text label and its anchor point. The offset is applied after rotation by the _angle_ property.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"dy\": {\n \"anyOf\": [\n {\n \"description\": \"The vertical offset, in pixels, between the text label and its anchor point. The offset is applied after rotation by the _angle_ property.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"ellipsis\": {\n \"anyOf\": [\n {\n \"description\": \"The ellipsis string for text truncated in response to the limit parameter.\\n\\n__Default value:__ `\\\"…\\\"`\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"endAngle\": {\n \"anyOf\": [\n {\n \"description\": \"The end angle in radians for arc marks. A value of `0` indicates up (north), increasing values proceed clockwise.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fill\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default fill color. This property has higher precedence than `config.color`. Set to `null` to remove fill.\\n\\n__Default value:__ (None)\"\n },\n \"fillOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"The fill opacity (value between [0,1]).\\n\\n__Default value:__ `1`\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"filled\": {\n \"description\": \"Whether the mark's color should be used as fill color instead of stroke color.\\n\\n__Default value:__ `false` for all `point`, `line`, and `rule` marks as well as `geoshape` marks for [`graticule`](https://vega.github.io/vega-lite/docs/data.html#graticule) data sources; otherwise, `true`.\\n\\n__Note:__ This property cannot be used in a [style config](https://vega.github.io/vega-lite/docs/mark.html#style-config).\",\n \"type\": \"boolean\"\n },\n \"font\": {\n \"anyOf\": [\n {\n \"description\": \"The typeface to set the text in (e.g., `\\\"Helvetica Neue\\\"`).\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontSize\": {\n \"anyOf\": [\n {\n \"description\": \"The font size, in pixels.\\n\\n__Default value:__ `11`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontStyle\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\",\n \"description\": \"The font style (e.g., `\\\"italic\\\"`).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontWeight\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\",\n \"description\": \"The font weight. This can be either a string (e.g `\\\"bold\\\"`, `\\\"normal\\\"`) or a number (`100`, `200`, `300`, ..., `900` where `\\\"normal\\\"` = `400` and `\\\"bold\\\"` = `700`).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"height\": {\n \"anyOf\": [\n {\n \"description\": \"Height of the marks.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"href\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/URI\",\n \"description\": \"A URL to load upon mouse click. If defined, the mark acts as a hyperlink.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"innerRadius\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The inner radius in pixels of arc marks. `innerRadius` is an alias for `radius2`.\\n\\n__Default value:__ `0`\",\n \"minimum\": 0\n },\n \"interpolate\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Interpolate\",\n \"description\": \"The line interpolation method to use for line and area marks. One of the following:\\n- `\\\"linear\\\"`: piecewise linear segments, as in a polyline.\\n- `\\\"linear-closed\\\"`: close the linear segments to form a polygon.\\n- `\\\"step\\\"`: alternate between horizontal and vertical segments, as in a step function.\\n- `\\\"step-before\\\"`: alternate between vertical and horizontal segments, as in a step function.\\n- `\\\"step-after\\\"`: alternate between horizontal and vertical segments, as in a step function.\\n- `\\\"basis\\\"`: a B-spline, with control point duplication on the ends.\\n- `\\\"basis-open\\\"`: an open B-spline; may not intersect the start or end.\\n- `\\\"basis-closed\\\"`: a closed B-spline, as in a loop.\\n- `\\\"cardinal\\\"`: a Cardinal spline, with control point duplication on the ends.\\n- `\\\"cardinal-open\\\"`: an open Cardinal spline; may not intersect the start or end, but will intersect other control points.\\n- `\\\"cardinal-closed\\\"`: a closed Cardinal spline, as in a loop.\\n- `\\\"bundle\\\"`: equivalent to basis, except the tension parameter is used to straighten the spline.\\n- `\\\"monotone\\\"`: cubic interpolation that preserves monotonicity in y.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"invalid\": {\n \"description\": \"Defines how Vega-Lite should handle marks for invalid values (`null` and `NaN`).\\n- If set to `\\\"filter\\\"` (default), all data items with null values will be skipped (for line, trail, and area marks) or filtered (for other marks).\\n- If `null`, all data items are included. In this case, invalid values will be interpreted as zeroes.\",\n \"enum\": [\n \"filter\",\n null\n ],\n \"type\": [\n \"string\",\n \"null\"\n ]\n },\n \"limit\": {\n \"anyOf\": [\n {\n \"description\": \"The maximum length of the text mark in pixels. The text value will be automatically truncated if the rendered size exceeds the limit.\\n\\n__Default value:__ `0` -- indicating no limit\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"lineBreak\": {\n \"anyOf\": [\n {\n \"description\": \"A delimiter, such as a newline character, upon which to break text strings into multiple lines. This property is ignored if the text is array-valued.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"lineHeight\": {\n \"anyOf\": [\n {\n \"description\": \"The line height in pixels (the spacing between subsequent lines of text) for multi-line text marks.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"opacity\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The overall opacity (value between [0,1]).\\n\\n__Default value:__ `0.7` for non-aggregate plots with `point`, `tick`, `circle`, or `square` marks or layered `bar` charts and `1` otherwise.\",\n \"maximum\": 1,\n \"minimum\": 0\n },\n \"order\": {\n \"description\": \"For line and trail marks, this `order` property can be set to `null` or `false` to make the lines use the original order in the data sources.\",\n \"type\": [\n \"null\",\n \"boolean\"\n ]\n },\n \"orient\": {\n \"$ref\": \"#/definitions/Orientation\",\n \"description\": \"The orientation of a non-stacked bar, tick, area, and line charts. The value is either horizontal (default) or vertical.\\n- For bar, rule and tick, this determines whether the size of the bar and tick should be applied to x or y dimension.\\n- For area, this property determines the orient property of the Vega output.\\n- For line and trail marks, this property determines the sort order of the points in the line if `config.sortLineBy` is not specified. For stacked charts, this is always determined by the orientation of the stack; therefore explicitly specified value will be ignored.\"\n },\n \"outerRadius\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The outer radius in pixels of arc marks. `outerRadius` is an alias for `radius`.\\n\\n__Default value:__ `0`\",\n \"minimum\": 0\n },\n \"padAngle\": {\n \"anyOf\": [\n {\n \"description\": \"The angular padding applied to sides of the arc, in radians.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"point\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/OverlayMarkDef\"\n },\n {\n \"const\": \"transparent\",\n \"type\": \"string\"\n }\n ],\n \"description\": \"A flag for overlaying points on top of line or area marks, or an object defining the properties of the overlayed points.\\n\\n- If this property is `\\\"transparent\\\"`, transparent points will be used (for enhancing tooltips and selections).\\n\\n- If this property is an empty object (`{}`) or `true`, filled points with default properties will be used.\\n\\n- If this property is `false`, no points would be automatically added to line or area marks.\\n\\n__Default value:__ `false`.\"\n },\n \"radius\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"For arc mark, the primary (outer) radius in pixels.\\n\\nFor text marks, polar coordinate radial offset, in pixels, of the text from the origin determined by the `x` and `y` properties.\\n\\n__Default value:__ `min(plot_width, plot_height)/2`\",\n \"minimum\": 0\n },\n \"radius2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The secondary (inner) radius in pixels of arc marks.\\n\\n__Default value:__ `0`\",\n \"minimum\": 0\n },\n \"shape\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SymbolShape\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Shape of the point marks. Supported values include:\\n- plotting shapes: `\\\"circle\\\"`, `\\\"square\\\"`, `\\\"cross\\\"`, `\\\"diamond\\\"`, `\\\"triangle-up\\\"`, `\\\"triangle-down\\\"`, `\\\"triangle-right\\\"`, or `\\\"triangle-left\\\"`.\\n- the line symbol `\\\"stroke\\\"`\\n- centered directional shapes `\\\"arrow\\\"`, `\\\"wedge\\\"`, or `\\\"triangle\\\"`\\n- a custom [SVG path string](https://developer.mozilla.org/en-US/docs/Web/SVG/Tutorial/Paths) (For correct sizing, custom shape paths should be defined within a square bounding box with coordinates ranging from -1 to 1 along both the x and y dimensions.)\\n\\n__Default value:__ `\\\"circle\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"size\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default size for marks.\\n- For `point`/`circle`/`square`, this represents the pixel area of the marks. Note that this value sets the area of the symbol; the side lengths will increase with the square root of this value.\\n- For `bar`, this represents the band size of the bar, in pixels.\\n- For `text`, this represents the font size, in pixels.\\n\\n__Default value:__\\n- `30` for point, circle, square marks; width/height's `step`\\n- `2` for bar marks with discrete dimensions;\\n- `5` for bar marks with continuous dimensions;\\n- `11` for text marks.\",\n \"minimum\": 0\n },\n \"smooth\": {\n \"anyOf\": [\n {\n \"description\": \"A boolean flag (default true) indicating if the image should be smoothed when resized. If false, individual pixels should be scaled directly rather than interpolated with smoothing. For SVG rendering, this option may not work in some browsers due to lack of standardization.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"startAngle\": {\n \"anyOf\": [\n {\n \"description\": \"The start angle in radians for arc marks. A value of `0` indicates up (north), increasing values proceed clockwise.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"stroke\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default stroke color. This property has higher precedence than `config.color`. Set to `null` to remove stroke.\\n\\n__Default value:__ (None)\"\n },\n \"strokeCap\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeCap\",\n \"description\": \"The stroke cap for line ending style. One of `\\\"butt\\\"`, `\\\"round\\\"`, or `\\\"square\\\"`.\\n\\n__Default value:__ `\\\"butt\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeDash\": {\n \"anyOf\": [\n {\n \"description\": \"An array of alternating stroke, space lengths for creating dashed or dotted lines.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeDashOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The offset (in pixels) into which to begin drawing with the stroke dash array.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeJoin\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeJoin\",\n \"description\": \"The stroke line join method. One of `\\\"miter\\\"`, `\\\"round\\\"` or `\\\"bevel\\\"`.\\n\\n__Default value:__ `\\\"miter\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeMiterLimit\": {\n \"anyOf\": [\n {\n \"description\": \"The miter limit at which to bevel a line join.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The offset in pixels at which to draw the group stroke and fill. If unspecified, the default behavior is to dynamically offset stroked groups such that 1 pixel stroke widths align with the pixel grid.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"The stroke opacity (value between [0,1]).\\n\\n__Default value:__ `1`\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeWidth\": {\n \"anyOf\": [\n {\n \"description\": \"The stroke width, in pixels.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"tension\": {\n \"anyOf\": [\n {\n \"description\": \"Depending on the interpolation type, sets the tension parameter (for line and area marks).\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"text\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\",\n \"description\": \"Placeholder text if the `text` channel is not specified\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"theta\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"- For arc marks, the arc length in radians if theta2 is not specified, otherwise the start arc angle. (A value of 0 indicates up or “north”, increasing values proceed clockwise.)\\n\\n- For text marks, polar coordinate angle in radians.\",\n \"maximum\": 360,\n \"minimum\": 0\n },\n \"theta2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The end angle of arc marks in radians. A value of 0 indicates up or “north”, increasing values proceed clockwise.\"\n },\n \"timeUnitBandPosition\": {\n \"description\": \"Default relative band position for a time unit. If set to `0`, the marks will be positioned at the beginning of the time unit band step. If set to `0.5`, the marks will be positioned in the middle of the time unit band step.\",\n \"type\": \"number\"\n },\n \"timeUnitBandSize\": {\n \"description\": \"Default relative band size for a time unit. If set to `1`, the bandwidth of the marks will be equal to the time unit band step. If set to `0.5`, bandwidth of the marks will be half of the time unit band step.\",\n \"type\": \"number\"\n },\n \"tooltip\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/TooltipContent\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"The tooltip text string to show upon mouse hover or an object defining which fields should the tooltip be derived from.\\n\\n- If `tooltip` is `true` or `{\\\"content\\\": \\\"encoding\\\"}`, then all fields from `encoding` will be used.\\n- If `tooltip` is `{\\\"content\\\": \\\"data\\\"}`, then all fields that appear in the highlighted data point will be used.\\n- If set to `null` or `false`, then no tooltip will be used.\\n\\nSee the [`tooltip`](https://vega.github.io/vega-lite/docs/tooltip.html) documentation for a detailed discussion about tooltip in Vega-Lite.\\n\\n__Default value:__ `null`\"\n },\n \"url\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/URI\",\n \"description\": \"The URL of the image file for image marks.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"width\": {\n \"anyOf\": [\n {\n \"description\": \"Width of the marks.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"x\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"X coordinates of the marks, or width of horizontal `\\\"bar\\\"` and `\\\"area\\\"` without specified `x2` or `width`.\\n\\nThe `value` of this channel can be a number or a string `\\\"width\\\"` for the width of the plot.\"\n },\n \"x2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"X2 coordinates for ranged `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"rect\\\"`, and `\\\"rule\\\"`.\\n\\nThe `value` of this channel can be a number or a string `\\\"width\\\"` for the width of the plot.\"\n },\n \"y\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Y coordinates of the marks, or height of vertical `\\\"bar\\\"` and `\\\"area\\\"` without specified `y2` or `height`.\\n\\nThe `value` of this channel can be a number or a string `\\\"height\\\"` for the height of the plot.\"\n },\n \"y2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Y2 coordinates for ranged `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"rect\\\"`, and `\\\"rule\\\"`.\\n\\nThe `value` of this channel can be a number or a string `\\\"height\\\"` for the height of the plot.\"\n }\n },\n \"type\": \"object\"\n },\n \"LineString\": {\n \"additionalProperties\": false,\n \"description\": \"LineString geometry object. https://tools.ietf.org/html/rfc7946#section-3.1.4\",\n \"properties\": {\n \"bbox\": {\n \"$ref\": \"#/definitions/BBox\",\n \"description\": \"Bounding box of the coordinate range of the object's Geometries, Features, or Feature Collections. https://tools.ietf.org/html/rfc7946#section-5\"\n },\n \"coordinates\": {\n \"items\": {\n \"$ref\": \"#/definitions/Position\"\n },\n \"type\": \"array\"\n },\n \"type\": {\n \"const\": \"LineString\",\n \"description\": \"Specifies the type of GeoJSON object.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"coordinates\",\n \"type\"\n ],\n \"type\": \"object\"\n },\n \"LinearGradient\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"gradient\": {\n \"const\": \"linear\",\n \"description\": \"The type of gradient. Use `\\\"linear\\\"` for a linear gradient.\",\n \"type\": \"string\"\n },\n \"id\": {\n \"type\": \"string\"\n },\n \"stops\": {\n \"description\": \"An array of gradient stops defining the gradient color sequence.\",\n \"items\": {\n \"$ref\": \"#/definitions/GradientStop\"\n },\n \"type\": \"array\"\n },\n \"x1\": {\n \"description\": \"The starting x-coordinate, in normalized [0, 1] coordinates, of the linear gradient.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n \"x2\": {\n \"description\": \"The ending x-coordinate, in normalized [0, 1] coordinates, of the linear gradient.\\n\\n__Default value:__ `1`\",\n \"type\": \"number\"\n },\n \"y1\": {\n \"description\": \"The starting y-coordinate, in normalized [0, 1] coordinates, of the linear gradient.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n \"y2\": {\n \"description\": \"The ending y-coordinate, in normalized [0, 1] coordinates, of the linear gradient.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n }\n },\n \"required\": [\n \"gradient\",\n \"stops\"\n ],\n \"type\": \"object\"\n },\n \"LocalMultiTimeUnit\": {\n \"enum\": [\n \"yearquarter\",\n \"yearquartermonth\",\n \"yearmonth\",\n \"yearmonthdate\",\n \"yearmonthdatehours\",\n \"yearmonthdatehoursminutes\",\n \"yearmonthdatehoursminutesseconds\",\n \"yearweek\",\n \"yearweekday\",\n \"yearweekdayhours\",\n \"yearweekdayhoursminutes\",\n \"yearweekdayhoursminutesseconds\",\n \"yeardayofyear\",\n \"quartermonth\",\n \"monthdate\",\n \"monthdatehours\",\n \"monthdatehoursminutes\",\n \"monthdatehoursminutesseconds\",\n \"weekday\",\n \"weeksdayhours\",\n \"weekdayhoursminutes\",\n \"weekdayhoursminutesseconds\",\n \"dayhours\",\n \"dayhoursminutes\",\n \"dayhoursminutesseconds\",\n \"hoursminutes\",\n \"hoursminutesseconds\",\n \"minutesseconds\",\n \"secondsmilliseconds\"\n ],\n \"type\": \"string\"\n },\n \"LocalSingleTimeUnit\": {\n \"enum\": [\n \"year\",\n \"quarter\",\n \"month\",\n \"week\",\n \"day\",\n \"dayofyear\",\n \"date\",\n \"hours\",\n \"minutes\",\n \"seconds\",\n \"milliseconds\"\n ],\n \"type\": \"string\"\n },\n \"Locale\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"number\": {\n \"$ref\": \"#/definitions/NumberLocale\"\n },\n \"time\": {\n \"$ref\": \"#/definitions/TimeLocale\"\n }\n },\n \"type\": \"object\"\n },\n \"LoessTransform\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"as\": {\n \"description\": \"The output field names for the smoothed points generated by the loess transform.\\n\\n__Default value:__ The field names of the input x and y values.\",\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"maxItems\": 2,\n \"minItems\": 2,\n \"type\": \"array\"\n },\n \"bandwidth\": {\n \"description\": \"A bandwidth parameter in the range `[0, 1]` that determines the amount of smoothing.\\n\\n__Default value:__ `0.3`\",\n \"type\": \"number\"\n },\n \"groupby\": {\n \"description\": \"The data fields to group by. If not specified, a single group containing all data objects will be used.\",\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"type\": \"array\"\n },\n \"loess\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"The data field of the dependent variable to smooth.\"\n },\n \"on\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"The data field of the independent variable to use a predictor.\"\n }\n },\n \"required\": [\n \"loess\",\n \"on\"\n ],\n \"type\": \"object\"\n },\n \"LogicalAnd\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"and\": {\n \"items\": {\n \"$ref\": \"#/definitions/PredicateComposition\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"and\"\n ],\n \"type\": \"object\"\n },\n \"PredicateComposition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LogicalNot\"\n },\n {\n \"$ref\": \"#/definitions/LogicalAnd\"\n },\n {\n \"$ref\": \"#/definitions/LogicalOr\"\n },\n {\n \"$ref\": \"#/definitions/Predicate\"\n }\n ]\n },\n \"LogicalNot\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"not\": {\n \"$ref\": \"#/definitions/PredicateComposition\"\n }\n },\n \"required\": [\n \"not\"\n ],\n \"type\": \"object\"\n },\n \"LogicalOr\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"or\": {\n \"items\": {\n \"$ref\": \"#/definitions/PredicateComposition\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"or\"\n ],\n \"type\": \"object\"\n },\n \"LookupData\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"data\": {\n \"$ref\": \"#/definitions/Data\",\n \"description\": \"Secondary data source to lookup in.\"\n },\n \"fields\": {\n \"description\": \"Fields in foreign data or selection to lookup. If not specified, the entire object is queried.\",\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"type\": \"array\"\n },\n \"key\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"Key in data to lookup.\"\n }\n },\n \"required\": [\n \"data\",\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"LookupSelection\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"fields\": {\n \"description\": \"Fields in foreign data or selection to lookup. If not specified, the entire object is queried.\",\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"type\": \"array\"\n },\n \"key\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"Key in data to lookup.\"\n },\n \"param\": {\n \"$ref\": \"#/definitions/ParameterName\",\n \"description\": \"Selection parameter name to look up.\"\n }\n },\n \"required\": [\n \"key\",\n \"param\"\n ],\n \"type\": \"object\"\n },\n \"LookupTransform\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"as\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FieldName\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"The output fields on which to store the looked up data values.\\n\\nFor data lookups, this property may be left blank if `from.fields` has been specified (those field names will be used); if `from.fields` has not been specified, `as` must be a string.\\n\\nFor selection lookups, this property is optional: if unspecified, looked up values will be stored under a property named for the selection; and if specified, it must correspond to `from.fields`.\"\n },\n \"default\": {\n \"description\": \"The default value to use if lookup fails.\\n\\n__Default value:__ `null`\"\n },\n \"from\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LookupData\"\n },\n {\n \"$ref\": \"#/definitions/LookupSelection\"\n }\n ],\n \"description\": \"Data source or selection for secondary data reference.\"\n },\n \"lookup\": {\n \"description\": \"Key in primary data source.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"lookup\",\n \"from\"\n ],\n \"type\": \"object\"\n },\n \"Mark\": {\n \"description\": \"All types of primitive marks.\",\n \"enum\": [\n \"arc\",\n \"area\",\n \"bar\",\n \"image\",\n \"line\",\n \"point\",\n \"rect\",\n \"rule\",\n \"text\",\n \"tick\",\n \"trail\",\n \"circle\",\n \"square\",\n \"geoshape\"\n ],\n \"type\": \"string\"\n },\n \"MarkConfig\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"align\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Align\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The horizontal alignment of the text or ranged marks (area, bar, image, rect, rule). One of `\\\"left\\\"`, `\\\"right\\\"`, `\\\"center\\\"`.\\n\\n__Note:__ Expression reference is *not* supported for range marks.\"\n },\n \"angle\": {\n \"anyOf\": [\n {\n \"description\": \"The rotation angle of the text, in degrees.\",\n \"maximum\": 360,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"aria\": {\n \"anyOf\": [\n {\n \"description\": \"A boolean flag indicating if [ARIA attributes](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) should be included (SVG output only). If `false`, the \\\"aria-hidden\\\" attribute will be set on the output SVG element, removing the mark item from the ARIA accessibility tree.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"ariaRole\": {\n \"anyOf\": [\n {\n \"description\": \"Sets the type of user interface element of the mark item for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If specified, this property determines the \\\"role\\\" attribute. Warning: this property is experimental and may be changed in the future.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"ariaRoleDescription\": {\n \"anyOf\": [\n {\n \"description\": \"A human-readable, author-localized description for the role of the mark item for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If specified, this property determines the \\\"aria-roledescription\\\" attribute. Warning: this property is experimental and may be changed in the future.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"aspect\": {\n \"anyOf\": [\n {\n \"description\": \"Whether to keep aspect ratio of image marks.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"baseline\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextBaseline\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"For text marks, the vertical text baseline. One of `\\\"alphabetic\\\"` (default), `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`, `\\\"line-top\\\"`, `\\\"line-bottom\\\"`, or an expression reference that provides one of the valid values. The `\\\"line-top\\\"` and `\\\"line-bottom\\\"` values operate similarly to `\\\"top\\\"` and `\\\"bottom\\\"`, but are calculated relative to the `lineHeight` rather than `fontSize` alone.\\n\\nFor range marks, the vertical alignment of the marks. One of `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`.\\n\\n__Note:__ Expression reference is *not* supported for range marks.\"\n },\n \"blend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Blend\",\n \"description\": \"The color blend mode for drawing an item on its current background. Any valid [CSS mix-blend-mode](https://developer.mozilla.org/en-US/docs/Web/CSS/mix-blend-mode) value can be used.\\n\\n__Default value: `\\\"source-over\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"color\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default color.\\n\\n__Default value:__ ■ `\\\"#4682b4\\\"`\\n\\n__Note:__\\n- This property cannot be used in a [style config](https://vega.github.io/vega-lite/docs/mark.html#style-config).\\n- The `fill` and `stroke` properties have higher precedence than `color` and will override `color`.\"\n },\n \"cornerRadius\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles or arcs' corners.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusBottomLeft\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' bottom left corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusBottomRight\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' bottom right corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusTopLeft\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' top right corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusTopRight\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' top left corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cursor\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Cursor\",\n \"description\": \"The mouse cursor used over the mark. Any valid [CSS cursor type](https://developer.mozilla.org/en-US/docs/Web/CSS/cursor#Values) can be used.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"description\": {\n \"anyOf\": [\n {\n \"description\": \"A text description of the mark item for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If specified, this property determines the [\\\"aria-label\\\" attribute](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/ARIA_Techniques/Using_the_aria-label_attribute).\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"dir\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextDirection\",\n \"description\": \"The direction of the text. One of `\\\"ltr\\\"` (left-to-right) or `\\\"rtl\\\"` (right-to-left). This property determines on which side is truncated in response to the limit parameter.\\n\\n__Default value:__ `\\\"ltr\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"dx\": {\n \"anyOf\": [\n {\n \"description\": \"The horizontal offset, in pixels, between the text label and its anchor point. The offset is applied after rotation by the _angle_ property.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"dy\": {\n \"anyOf\": [\n {\n \"description\": \"The vertical offset, in pixels, between the text label and its anchor point. The offset is applied after rotation by the _angle_ property.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"ellipsis\": {\n \"anyOf\": [\n {\n \"description\": \"The ellipsis string for text truncated in response to the limit parameter.\\n\\n__Default value:__ `\\\"…\\\"`\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"endAngle\": {\n \"anyOf\": [\n {\n \"description\": \"The end angle in radians for arc marks. A value of `0` indicates up (north), increasing values proceed clockwise.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fill\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default fill color. This property has higher precedence than `config.color`. Set to `null` to remove fill.\\n\\n__Default value:__ (None)\"\n },\n \"fillOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"The fill opacity (value between [0,1]).\\n\\n__Default value:__ `1`\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"filled\": {\n \"description\": \"Whether the mark's color should be used as fill color instead of stroke color.\\n\\n__Default value:__ `false` for all `point`, `line`, and `rule` marks as well as `geoshape` marks for [`graticule`](https://vega.github.io/vega-lite/docs/data.html#graticule) data sources; otherwise, `true`.\\n\\n__Note:__ This property cannot be used in a [style config](https://vega.github.io/vega-lite/docs/mark.html#style-config).\",\n \"type\": \"boolean\"\n },\n \"font\": {\n \"anyOf\": [\n {\n \"description\": \"The typeface to set the text in (e.g., `\\\"Helvetica Neue\\\"`).\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontSize\": {\n \"anyOf\": [\n {\n \"description\": \"The font size, in pixels.\\n\\n__Default value:__ `11`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontStyle\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\",\n \"description\": \"The font style (e.g., `\\\"italic\\\"`).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontWeight\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\",\n \"description\": \"The font weight. This can be either a string (e.g `\\\"bold\\\"`, `\\\"normal\\\"`) or a number (`100`, `200`, `300`, ..., `900` where `\\\"normal\\\"` = `400` and `\\\"bold\\\"` = `700`).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"height\": {\n \"anyOf\": [\n {\n \"description\": \"Height of the marks.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"href\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/URI\",\n \"description\": \"A URL to load upon mouse click. If defined, the mark acts as a hyperlink.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"innerRadius\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The inner radius in pixels of arc marks. `innerRadius` is an alias for `radius2`.\\n\\n__Default value:__ `0`\",\n \"minimum\": 0\n },\n \"interpolate\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Interpolate\",\n \"description\": \"The line interpolation method to use for line and area marks. One of the following:\\n- `\\\"linear\\\"`: piecewise linear segments, as in a polyline.\\n- `\\\"linear-closed\\\"`: close the linear segments to form a polygon.\\n- `\\\"step\\\"`: alternate between horizontal and vertical segments, as in a step function.\\n- `\\\"step-before\\\"`: alternate between vertical and horizontal segments, as in a step function.\\n- `\\\"step-after\\\"`: alternate between horizontal and vertical segments, as in a step function.\\n- `\\\"basis\\\"`: a B-spline, with control point duplication on the ends.\\n- `\\\"basis-open\\\"`: an open B-spline; may not intersect the start or end.\\n- `\\\"basis-closed\\\"`: a closed B-spline, as in a loop.\\n- `\\\"cardinal\\\"`: a Cardinal spline, with control point duplication on the ends.\\n- `\\\"cardinal-open\\\"`: an open Cardinal spline; may not intersect the start or end, but will intersect other control points.\\n- `\\\"cardinal-closed\\\"`: a closed Cardinal spline, as in a loop.\\n- `\\\"bundle\\\"`: equivalent to basis, except the tension parameter is used to straighten the spline.\\n- `\\\"monotone\\\"`: cubic interpolation that preserves monotonicity in y.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"invalid\": {\n \"description\": \"Defines how Vega-Lite should handle marks for invalid values (`null` and `NaN`).\\n- If set to `\\\"filter\\\"` (default), all data items with null values will be skipped (for line, trail, and area marks) or filtered (for other marks).\\n- If `null`, all data items are included. In this case, invalid values will be interpreted as zeroes.\",\n \"enum\": [\n \"filter\",\n null\n ],\n \"type\": [\n \"string\",\n \"null\"\n ]\n },\n \"limit\": {\n \"anyOf\": [\n {\n \"description\": \"The maximum length of the text mark in pixels. The text value will be automatically truncated if the rendered size exceeds the limit.\\n\\n__Default value:__ `0` -- indicating no limit\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"lineBreak\": {\n \"anyOf\": [\n {\n \"description\": \"A delimiter, such as a newline character, upon which to break text strings into multiple lines. This property is ignored if the text is array-valued.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"lineHeight\": {\n \"anyOf\": [\n {\n \"description\": \"The line height in pixels (the spacing between subsequent lines of text) for multi-line text marks.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"opacity\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The overall opacity (value between [0,1]).\\n\\n__Default value:__ `0.7` for non-aggregate plots with `point`, `tick`, `circle`, or `square` marks or layered `bar` charts and `1` otherwise.\",\n \"maximum\": 1,\n \"minimum\": 0\n },\n \"order\": {\n \"description\": \"For line and trail marks, this `order` property can be set to `null` or `false` to make the lines use the original order in the data sources.\",\n \"type\": [\n \"null\",\n \"boolean\"\n ]\n },\n \"orient\": {\n \"$ref\": \"#/definitions/Orientation\",\n \"description\": \"The orientation of a non-stacked bar, tick, area, and line charts. The value is either horizontal (default) or vertical.\\n- For bar, rule and tick, this determines whether the size of the bar and tick should be applied to x or y dimension.\\n- For area, this property determines the orient property of the Vega output.\\n- For line and trail marks, this property determines the sort order of the points in the line if `config.sortLineBy` is not specified. For stacked charts, this is always determined by the orientation of the stack; therefore explicitly specified value will be ignored.\"\n },\n \"outerRadius\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The outer radius in pixels of arc marks. `outerRadius` is an alias for `radius`.\\n\\n__Default value:__ `0`\",\n \"minimum\": 0\n },\n \"padAngle\": {\n \"anyOf\": [\n {\n \"description\": \"The angular padding applied to sides of the arc, in radians.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"radius\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"For arc mark, the primary (outer) radius in pixels.\\n\\nFor text marks, polar coordinate radial offset, in pixels, of the text from the origin determined by the `x` and `y` properties.\\n\\n__Default value:__ `min(plot_width, plot_height)/2`\",\n \"minimum\": 0\n },\n \"radius2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The secondary (inner) radius in pixels of arc marks.\\n\\n__Default value:__ `0`\",\n \"minimum\": 0\n },\n \"shape\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SymbolShape\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Shape of the point marks. Supported values include:\\n- plotting shapes: `\\\"circle\\\"`, `\\\"square\\\"`, `\\\"cross\\\"`, `\\\"diamond\\\"`, `\\\"triangle-up\\\"`, `\\\"triangle-down\\\"`, `\\\"triangle-right\\\"`, or `\\\"triangle-left\\\"`.\\n- the line symbol `\\\"stroke\\\"`\\n- centered directional shapes `\\\"arrow\\\"`, `\\\"wedge\\\"`, or `\\\"triangle\\\"`\\n- a custom [SVG path string](https://developer.mozilla.org/en-US/docs/Web/SVG/Tutorial/Paths) (For correct sizing, custom shape paths should be defined within a square bounding box with coordinates ranging from -1 to 1 along both the x and y dimensions.)\\n\\n__Default value:__ `\\\"circle\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"size\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default size for marks.\\n- For `point`/`circle`/`square`, this represents the pixel area of the marks. Note that this value sets the area of the symbol; the side lengths will increase with the square root of this value.\\n- For `bar`, this represents the band size of the bar, in pixels.\\n- For `text`, this represents the font size, in pixels.\\n\\n__Default value:__\\n- `30` for point, circle, square marks; width/height's `step`\\n- `2` for bar marks with discrete dimensions;\\n- `5` for bar marks with continuous dimensions;\\n- `11` for text marks.\",\n \"minimum\": 0\n },\n \"smooth\": {\n \"anyOf\": [\n {\n \"description\": \"A boolean flag (default true) indicating if the image should be smoothed when resized. If false, individual pixels should be scaled directly rather than interpolated with smoothing. For SVG rendering, this option may not work in some browsers due to lack of standardization.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"startAngle\": {\n \"anyOf\": [\n {\n \"description\": \"The start angle in radians for arc marks. A value of `0` indicates up (north), increasing values proceed clockwise.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"stroke\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default stroke color. This property has higher precedence than `config.color`. Set to `null` to remove stroke.\\n\\n__Default value:__ (None)\"\n },\n \"strokeCap\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeCap\",\n \"description\": \"The stroke cap for line ending style. One of `\\\"butt\\\"`, `\\\"round\\\"`, or `\\\"square\\\"`.\\n\\n__Default value:__ `\\\"butt\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeDash\": {\n \"anyOf\": [\n {\n \"description\": \"An array of alternating stroke, space lengths for creating dashed or dotted lines.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeDashOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The offset (in pixels) into which to begin drawing with the stroke dash array.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeJoin\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeJoin\",\n \"description\": \"The stroke line join method. One of `\\\"miter\\\"`, `\\\"round\\\"` or `\\\"bevel\\\"`.\\n\\n__Default value:__ `\\\"miter\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeMiterLimit\": {\n \"anyOf\": [\n {\n \"description\": \"The miter limit at which to bevel a line join.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The offset in pixels at which to draw the group stroke and fill. If unspecified, the default behavior is to dynamically offset stroked groups such that 1 pixel stroke widths align with the pixel grid.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"The stroke opacity (value between [0,1]).\\n\\n__Default value:__ `1`\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeWidth\": {\n \"anyOf\": [\n {\n \"description\": \"The stroke width, in pixels.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"tension\": {\n \"anyOf\": [\n {\n \"description\": \"Depending on the interpolation type, sets the tension parameter (for line and area marks).\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"text\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\",\n \"description\": \"Placeholder text if the `text` channel is not specified\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"theta\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"- For arc marks, the arc length in radians if theta2 is not specified, otherwise the start arc angle. (A value of 0 indicates up or “north”, increasing values proceed clockwise.)\\n\\n- For text marks, polar coordinate angle in radians.\",\n \"maximum\": 360,\n \"minimum\": 0\n },\n \"theta2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The end angle of arc marks in radians. A value of 0 indicates up or “north”, increasing values proceed clockwise.\"\n },\n \"timeUnitBandPosition\": {\n \"description\": \"Default relative band position for a time unit. If set to `0`, the marks will be positioned at the beginning of the time unit band step. If set to `0.5`, the marks will be positioned in the middle of the time unit band step.\",\n \"type\": \"number\"\n },\n \"timeUnitBandSize\": {\n \"description\": \"Default relative band size for a time unit. If set to `1`, the bandwidth of the marks will be equal to the time unit band step. If set to `0.5`, bandwidth of the marks will be half of the time unit band step.\",\n \"type\": \"number\"\n },\n \"tooltip\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/TooltipContent\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"The tooltip text string to show upon mouse hover or an object defining which fields should the tooltip be derived from.\\n\\n- If `tooltip` is `true` or `{\\\"content\\\": \\\"encoding\\\"}`, then all fields from `encoding` will be used.\\n- If `tooltip` is `{\\\"content\\\": \\\"data\\\"}`, then all fields that appear in the highlighted data point will be used.\\n- If set to `null` or `false`, then no tooltip will be used.\\n\\nSee the [`tooltip`](https://vega.github.io/vega-lite/docs/tooltip.html) documentation for a detailed discussion about tooltip in Vega-Lite.\\n\\n__Default value:__ `null`\"\n },\n \"url\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/URI\",\n \"description\": \"The URL of the image file for image marks.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"width\": {\n \"anyOf\": [\n {\n \"description\": \"Width of the marks.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"x\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"X coordinates of the marks, or width of horizontal `\\\"bar\\\"` and `\\\"area\\\"` without specified `x2` or `width`.\\n\\nThe `value` of this channel can be a number or a string `\\\"width\\\"` for the width of the plot.\"\n },\n \"x2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"X2 coordinates for ranged `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"rect\\\"`, and `\\\"rule\\\"`.\\n\\nThe `value` of this channel can be a number or a string `\\\"width\\\"` for the width of the plot.\"\n },\n \"y\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Y coordinates of the marks, or height of vertical `\\\"bar\\\"` and `\\\"area\\\"` without specified `y2` or `height`.\\n\\nThe `value` of this channel can be a number or a string `\\\"height\\\"` for the height of the plot.\"\n },\n \"y2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Y2 coordinates for ranged `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"rect\\\"`, and `\\\"rule\\\"`.\\n\\nThe `value` of this channel can be a number or a string `\\\"height\\\"` for the height of the plot.\"\n }\n },\n \"type\": \"object\"\n },\n \"MarkDef\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"align\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Align\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The horizontal alignment of the text or ranged marks (area, bar, image, rect, rule). One of `\\\"left\\\"`, `\\\"right\\\"`, `\\\"center\\\"`.\\n\\n__Note:__ Expression reference is *not* supported for range marks.\"\n },\n \"angle\": {\n \"anyOf\": [\n {\n \"description\": \"The rotation angle of the text, in degrees.\",\n \"maximum\": 360,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"aria\": {\n \"anyOf\": [\n {\n \"description\": \"A boolean flag indicating if [ARIA attributes](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) should be included (SVG output only). If `false`, the \\\"aria-hidden\\\" attribute will be set on the output SVG element, removing the mark item from the ARIA accessibility tree.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"ariaRole\": {\n \"anyOf\": [\n {\n \"description\": \"Sets the type of user interface element of the mark item for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If specified, this property determines the \\\"role\\\" attribute. Warning: this property is experimental and may be changed in the future.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"ariaRoleDescription\": {\n \"anyOf\": [\n {\n \"description\": \"A human-readable, author-localized description for the role of the mark item for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If specified, this property determines the \\\"aria-roledescription\\\" attribute. Warning: this property is experimental and may be changed in the future.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"aspect\": {\n \"anyOf\": [\n {\n \"description\": \"Whether to keep aspect ratio of image marks.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"bandSize\": {\n \"description\": \"The width of the ticks.\\n\\n__Default value:__ 3/4 of step (width step for horizontal ticks and height step for vertical ticks).\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"baseline\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextBaseline\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"For text marks, the vertical text baseline. One of `\\\"alphabetic\\\"` (default), `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`, `\\\"line-top\\\"`, `\\\"line-bottom\\\"`, or an expression reference that provides one of the valid values. The `\\\"line-top\\\"` and `\\\"line-bottom\\\"` values operate similarly to `\\\"top\\\"` and `\\\"bottom\\\"`, but are calculated relative to the `lineHeight` rather than `fontSize` alone.\\n\\nFor range marks, the vertical alignment of the marks. One of `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`.\\n\\n__Note:__ Expression reference is *not* supported for range marks.\"\n },\n \"binSpacing\": {\n \"description\": \"Offset between bars for binned field. The ideal value for this is either 0 (preferred by statisticians) or 1 (Vega-Lite default, D3 example style).\\n\\n__Default value:__ `1`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"blend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Blend\",\n \"description\": \"The color blend mode for drawing an item on its current background. Any valid [CSS mix-blend-mode](https://developer.mozilla.org/en-US/docs/Web/CSS/mix-blend-mode) value can be used.\\n\\n__Default value: `\\\"source-over\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"clip\": {\n \"description\": \"Whether a mark be clipped to the enclosing group’s width and height.\",\n \"type\": \"boolean\"\n },\n \"color\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default color.\\n\\n__Default value:__ ■ `\\\"#4682b4\\\"`\\n\\n__Note:__\\n- This property cannot be used in a [style config](https://vega.github.io/vega-lite/docs/mark.html#style-config).\\n- The `fill` and `stroke` properties have higher precedence than `color` and will override `color`.\"\n },\n \"continuousBandSize\": {\n \"description\": \"The default size of the bars on continuous scales.\\n\\n__Default value:__ `5`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"cornerRadius\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles or arcs' corners.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusBottomLeft\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' bottom left corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusBottomRight\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' bottom right corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusEnd\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"- For vertical bars, top-left and top-right corner radius.\\n\\n- For horizontal bars, top-right and bottom-right corner radius.\"\n },\n \"cornerRadiusTopLeft\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' top right corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusTopRight\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' top left corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cursor\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Cursor\",\n \"description\": \"The mouse cursor used over the mark. Any valid [CSS cursor type](https://developer.mozilla.org/en-US/docs/Web/CSS/cursor#Values) can be used.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"description\": {\n \"anyOf\": [\n {\n \"description\": \"A text description of the mark item for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If specified, this property determines the [\\\"aria-label\\\" attribute](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/ARIA_Techniques/Using_the_aria-label_attribute).\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"dir\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextDirection\",\n \"description\": \"The direction of the text. One of `\\\"ltr\\\"` (left-to-right) or `\\\"rtl\\\"` (right-to-left). This property determines on which side is truncated in response to the limit parameter.\\n\\n__Default value:__ `\\\"ltr\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"discreteBandSize\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/RelativeBandSize\"\n }\n ],\n \"description\": \"The default size of the bars with discrete dimensions. If unspecified, the default size is `step-2`, which provides 2 pixel offset between bars.\",\n \"minimum\": 0\n },\n \"dx\": {\n \"anyOf\": [\n {\n \"description\": \"The horizontal offset, in pixels, between the text label and its anchor point. The offset is applied after rotation by the _angle_ property.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"dy\": {\n \"anyOf\": [\n {\n \"description\": \"The vertical offset, in pixels, between the text label and its anchor point. The offset is applied after rotation by the _angle_ property.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"ellipsis\": {\n \"anyOf\": [\n {\n \"description\": \"The ellipsis string for text truncated in response to the limit parameter.\\n\\n__Default value:__ `\\\"…\\\"`\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fill\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default fill color. This property has higher precedence than `config.color`. Set to `null` to remove fill.\\n\\n__Default value:__ (None)\"\n },\n \"fillOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"The fill opacity (value between [0,1]).\\n\\n__Default value:__ `1`\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"filled\": {\n \"description\": \"Whether the mark's color should be used as fill color instead of stroke color.\\n\\n__Default value:__ `false` for all `point`, `line`, and `rule` marks as well as `geoshape` marks for [`graticule`](https://vega.github.io/vega-lite/docs/data.html#graticule) data sources; otherwise, `true`.\\n\\n__Note:__ This property cannot be used in a [style config](https://vega.github.io/vega-lite/docs/mark.html#style-config).\",\n \"type\": \"boolean\"\n },\n \"font\": {\n \"anyOf\": [\n {\n \"description\": \"The typeface to set the text in (e.g., `\\\"Helvetica Neue\\\"`).\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontSize\": {\n \"anyOf\": [\n {\n \"description\": \"The font size, in pixels.\\n\\n__Default value:__ `11`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontStyle\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\",\n \"description\": \"The font style (e.g., `\\\"italic\\\"`).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontWeight\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\",\n \"description\": \"The font weight. This can be either a string (e.g `\\\"bold\\\"`, `\\\"normal\\\"`) or a number (`100`, `200`, `300`, ..., `900` where `\\\"normal\\\"` = `400` and `\\\"bold\\\"` = `700`).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"height\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RelativeBandSize\"\n }\n ],\n \"description\": \"Height of the marks. One of:\\n\\n- A number representing a fixed pixel height.\\n\\n- A relative band size definition. For example, `{band: 0.5}` represents half of the band\"\n },\n \"href\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/URI\",\n \"description\": \"A URL to load upon mouse click. If defined, the mark acts as a hyperlink.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"innerRadius\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The inner radius in pixels of arc marks. `innerRadius` is an alias for `radius2`.\\n\\n__Default value:__ `0`\",\n \"minimum\": 0\n },\n \"interpolate\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Interpolate\",\n \"description\": \"The line interpolation method to use for line and area marks. One of the following:\\n- `\\\"linear\\\"`: piecewise linear segments, as in a polyline.\\n- `\\\"linear-closed\\\"`: close the linear segments to form a polygon.\\n- `\\\"step\\\"`: alternate between horizontal and vertical segments, as in a step function.\\n- `\\\"step-before\\\"`: alternate between vertical and horizontal segments, as in a step function.\\n- `\\\"step-after\\\"`: alternate between horizontal and vertical segments, as in a step function.\\n- `\\\"basis\\\"`: a B-spline, with control point duplication on the ends.\\n- `\\\"basis-open\\\"`: an open B-spline; may not intersect the start or end.\\n- `\\\"basis-closed\\\"`: a closed B-spline, as in a loop.\\n- `\\\"cardinal\\\"`: a Cardinal spline, with control point duplication on the ends.\\n- `\\\"cardinal-open\\\"`: an open Cardinal spline; may not intersect the start or end, but will intersect other control points.\\n- `\\\"cardinal-closed\\\"`: a closed Cardinal spline, as in a loop.\\n- `\\\"bundle\\\"`: equivalent to basis, except the tension parameter is used to straighten the spline.\\n- `\\\"monotone\\\"`: cubic interpolation that preserves monotonicity in y.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"invalid\": {\n \"description\": \"Defines how Vega-Lite should handle marks for invalid values (`null` and `NaN`).\\n- If set to `\\\"filter\\\"` (default), all data items with null values will be skipped (for line, trail, and area marks) or filtered (for other marks).\\n- If `null`, all data items are included. In this case, invalid values will be interpreted as zeroes.\",\n \"enum\": [\n \"filter\",\n null\n ],\n \"type\": [\n \"string\",\n \"null\"\n ]\n },\n \"limit\": {\n \"anyOf\": [\n {\n \"description\": \"The maximum length of the text mark in pixels. The text value will be automatically truncated if the rendered size exceeds the limit.\\n\\n__Default value:__ `0` -- indicating no limit\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"line\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/OverlayMarkDef\"\n }\n ],\n \"description\": \"A flag for overlaying line on top of area marks, or an object defining the properties of the overlayed lines.\\n\\n- If this value is an empty object (`{}`) or `true`, lines with default properties will be used.\\n\\n- If this value is `false`, no lines would be automatically added to area marks.\\n\\n__Default value:__ `false`.\"\n },\n \"lineBreak\": {\n \"anyOf\": [\n {\n \"description\": \"A delimiter, such as a newline character, upon which to break text strings into multiple lines. This property is ignored if the text is array-valued.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"lineHeight\": {\n \"anyOf\": [\n {\n \"description\": \"The line height in pixels (the spacing between subsequent lines of text) for multi-line text marks.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"minBandSize\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The minimum band size for bar and rectangle marks. __Default value:__ `0.25`\"\n },\n \"opacity\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The overall opacity (value between [0,1]).\\n\\n__Default value:__ `0.7` for non-aggregate plots with `point`, `tick`, `circle`, or `square` marks or layered `bar` charts and `1` otherwise.\",\n \"maximum\": 1,\n \"minimum\": 0\n },\n \"order\": {\n \"description\": \"For line and trail marks, this `order` property can be set to `null` or `false` to make the lines use the original order in the data sources.\",\n \"type\": [\n \"null\",\n \"boolean\"\n ]\n },\n \"orient\": {\n \"$ref\": \"#/definitions/Orientation\",\n \"description\": \"The orientation of a non-stacked bar, tick, area, and line charts. The value is either horizontal (default) or vertical.\\n- For bar, rule and tick, this determines whether the size of the bar and tick should be applied to x or y dimension.\\n- For area, this property determines the orient property of the Vega output.\\n- For line and trail marks, this property determines the sort order of the points in the line if `config.sortLineBy` is not specified. For stacked charts, this is always determined by the orientation of the stack; therefore explicitly specified value will be ignored.\"\n },\n \"outerRadius\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The outer radius in pixels of arc marks. `outerRadius` is an alias for `radius`.\\n\\n__Default value:__ `0`\",\n \"minimum\": 0\n },\n \"padAngle\": {\n \"anyOf\": [\n {\n \"description\": \"The angular padding applied to sides of the arc, in radians.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"point\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/OverlayMarkDef\"\n },\n {\n \"const\": \"transparent\",\n \"type\": \"string\"\n }\n ],\n \"description\": \"A flag for overlaying points on top of line or area marks, or an object defining the properties of the overlayed points.\\n\\n- If this property is `\\\"transparent\\\"`, transparent points will be used (for enhancing tooltips and selections).\\n\\n- If this property is an empty object (`{}`) or `true`, filled points with default properties will be used.\\n\\n- If this property is `false`, no points would be automatically added to line or area marks.\\n\\n__Default value:__ `false`.\"\n },\n \"radius\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"For arc mark, the primary (outer) radius in pixels.\\n\\nFor text marks, polar coordinate radial offset, in pixels, of the text from the origin determined by the `x` and `y` properties.\\n\\n__Default value:__ `min(plot_width, plot_height)/2`\",\n \"minimum\": 0\n },\n \"radius2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The secondary (inner) radius in pixels of arc marks.\\n\\n__Default value:__ `0`\",\n \"minimum\": 0\n },\n \"radius2Offset\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Offset for radius2.\"\n },\n \"radiusOffset\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Offset for radius.\"\n },\n \"shape\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SymbolShape\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Shape of the point marks. Supported values include:\\n- plotting shapes: `\\\"circle\\\"`, `\\\"square\\\"`, `\\\"cross\\\"`, `\\\"diamond\\\"`, `\\\"triangle-up\\\"`, `\\\"triangle-down\\\"`, `\\\"triangle-right\\\"`, or `\\\"triangle-left\\\"`.\\n- the line symbol `\\\"stroke\\\"`\\n- centered directional shapes `\\\"arrow\\\"`, `\\\"wedge\\\"`, or `\\\"triangle\\\"`\\n- a custom [SVG path string](https://developer.mozilla.org/en-US/docs/Web/SVG/Tutorial/Paths) (For correct sizing, custom shape paths should be defined within a square bounding box with coordinates ranging from -1 to 1 along both the x and y dimensions.)\\n\\n__Default value:__ `\\\"circle\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"size\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default size for marks.\\n- For `point`/`circle`/`square`, this represents the pixel area of the marks. Note that this value sets the area of the symbol; the side lengths will increase with the square root of this value.\\n- For `bar`, this represents the band size of the bar, in pixels.\\n- For `text`, this represents the font size, in pixels.\\n\\n__Default value:__\\n- `30` for point, circle, square marks; width/height's `step`\\n- `2` for bar marks with discrete dimensions;\\n- `5` for bar marks with continuous dimensions;\\n- `11` for text marks.\",\n \"minimum\": 0\n },\n \"smooth\": {\n \"anyOf\": [\n {\n \"description\": \"A boolean flag (default true) indicating if the image should be smoothed when resized. If false, individual pixels should be scaled directly rather than interpolated with smoothing. For SVG rendering, this option may not work in some browsers due to lack of standardization.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"stroke\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default stroke color. This property has higher precedence than `config.color`. Set to `null` to remove stroke.\\n\\n__Default value:__ (None)\"\n },\n \"strokeCap\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeCap\",\n \"description\": \"The stroke cap for line ending style. One of `\\\"butt\\\"`, `\\\"round\\\"`, or `\\\"square\\\"`.\\n\\n__Default value:__ `\\\"butt\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeDash\": {\n \"anyOf\": [\n {\n \"description\": \"An array of alternating stroke, space lengths for creating dashed or dotted lines.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeDashOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The offset (in pixels) into which to begin drawing with the stroke dash array.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeJoin\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeJoin\",\n \"description\": \"The stroke line join method. One of `\\\"miter\\\"`, `\\\"round\\\"` or `\\\"bevel\\\"`.\\n\\n__Default value:__ `\\\"miter\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeMiterLimit\": {\n \"anyOf\": [\n {\n \"description\": \"The miter limit at which to bevel a line join.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The offset in pixels at which to draw the group stroke and fill. If unspecified, the default behavior is to dynamically offset stroked groups such that 1 pixel stroke widths align with the pixel grid.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"The stroke opacity (value between [0,1]).\\n\\n__Default value:__ `1`\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeWidth\": {\n \"anyOf\": [\n {\n \"description\": \"The stroke width, in pixels.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"style\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A string or array of strings indicating the name of custom styles to apply to the mark. A style is a named collection of mark property defaults defined within the [style configuration](https://vega.github.io/vega-lite/docs/mark.html#style-config). If style is an array, later styles will override earlier styles. Any [mark properties](https://vega.github.io/vega-lite/docs/encoding.html#mark-prop) explicitly defined within the `encoding` will override a style default.\\n\\n__Default value:__ The mark's name. For example, a bar mark will have style `\\\"bar\\\"` by default. __Note:__ Any specified style will augment the default style. For example, a bar mark with `\\\"style\\\": \\\"foo\\\"` will receive from `config.style.bar` and `config.style.foo` (the specified style `\\\"foo\\\"` has higher precedence).\"\n },\n \"tension\": {\n \"anyOf\": [\n {\n \"description\": \"Depending on the interpolation type, sets the tension parameter (for line and area marks).\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"text\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\",\n \"description\": \"Placeholder text if the `text` channel is not specified\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"theta\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"- For arc marks, the arc length in radians if theta2 is not specified, otherwise the start arc angle. (A value of 0 indicates up or “north”, increasing values proceed clockwise.)\\n\\n- For text marks, polar coordinate angle in radians.\",\n \"maximum\": 360,\n \"minimum\": 0\n },\n \"theta2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The end angle of arc marks in radians. A value of 0 indicates up or “north”, increasing values proceed clockwise.\"\n },\n \"theta2Offset\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Offset for theta2.\"\n },\n \"thetaOffset\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Offset for theta.\"\n },\n \"thickness\": {\n \"description\": \"Thickness of the tick mark.\\n\\n__Default value:__ `1`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"timeUnitBandPosition\": {\n \"description\": \"Default relative band position for a time unit. If set to `0`, the marks will be positioned at the beginning of the time unit band step. If set to `0.5`, the marks will be positioned in the middle of the time unit band step.\",\n \"type\": \"number\"\n },\n \"timeUnitBandSize\": {\n \"description\": \"Default relative band size for a time unit. If set to `1`, the bandwidth of the marks will be equal to the time unit band step. If set to `0.5`, bandwidth of the marks will be half of the time unit band step.\",\n \"type\": \"number\"\n },\n \"tooltip\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/TooltipContent\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"The tooltip text string to show upon mouse hover or an object defining which fields should the tooltip be derived from.\\n\\n- If `tooltip` is `true` or `{\\\"content\\\": \\\"encoding\\\"}`, then all fields from `encoding` will be used.\\n- If `tooltip` is `{\\\"content\\\": \\\"data\\\"}`, then all fields that appear in the highlighted data point will be used.\\n- If set to `null` or `false`, then no tooltip will be used.\\n\\nSee the [`tooltip`](https://vega.github.io/vega-lite/docs/tooltip.html) documentation for a detailed discussion about tooltip in Vega-Lite.\\n\\n__Default value:__ `null`\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/Mark\",\n \"description\": \"The mark type. This could a primitive mark type (one of `\\\"bar\\\"`, `\\\"circle\\\"`, `\\\"square\\\"`, `\\\"tick\\\"`, `\\\"line\\\"`, `\\\"area\\\"`, `\\\"point\\\"`, `\\\"geoshape\\\"`, `\\\"rule\\\"`, and `\\\"text\\\"`) or a composite mark type (`\\\"boxplot\\\"`, `\\\"errorband\\\"`, `\\\"errorbar\\\"`).\"\n },\n \"url\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/URI\",\n \"description\": \"The URL of the image file for image marks.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"width\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RelativeBandSize\"\n }\n ],\n \"description\": \"Width of the marks. One of:\\n\\n- A number representing a fixed pixel width.\\n\\n- A relative band size definition. For example, `{band: 0.5}` represents half of the band.\"\n },\n \"x\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"X coordinates of the marks, or width of horizontal `\\\"bar\\\"` and `\\\"area\\\"` without specified `x2` or `width`.\\n\\nThe `value` of this channel can be a number or a string `\\\"width\\\"` for the width of the plot.\"\n },\n \"x2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"X2 coordinates for ranged `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"rect\\\"`, and `\\\"rule\\\"`.\\n\\nThe `value` of this channel can be a number or a string `\\\"width\\\"` for the width of the plot.\"\n },\n \"x2Offset\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Offset for x2-position.\"\n },\n \"xOffset\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Offset for x-position.\"\n },\n \"y\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Y coordinates of the marks, or height of vertical `\\\"bar\\\"` and `\\\"area\\\"` without specified `y2` or `height`.\\n\\nThe `value` of this channel can be a number or a string `\\\"height\\\"` for the height of the plot.\"\n },\n \"y2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Y2 coordinates for ranged `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"rect\\\"`, and `\\\"rule\\\"`.\\n\\nThe `value` of this channel can be a number or a string `\\\"height\\\"` for the height of the plot.\"\n },\n \"y2Offset\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Offset for y2-position.\"\n },\n \"yOffset\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Offset for y-position.\"\n }\n },\n \"required\": [\n \"type\"\n ],\n \"type\": \"object\"\n },\n \"MarkPropDef<(Gradient|string|null)>\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FieldOrDatumDefWithCondition\"\n },\n {\n \"$ref\": \"#/definitions/FieldOrDatumDefWithCondition\"\n },\n {\n \"$ref\": \"#/definitions/ValueDefWithCondition\"\n }\n ]\n },\n \"MarkPropDef<(string|null),TypeForShape>\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FieldOrDatumDefWithCondition,(string|null)>\"\n },\n {\n \"$ref\": \"#/definitions/FieldOrDatumDefWithCondition\"\n },\n {\n \"$ref\": \"#/definitions/ValueDefWithCondition,(string|null)>\"\n }\n ]\n },\n \"MarkPropDef\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FieldOrDatumDefWithCondition\"\n },\n {\n \"$ref\": \"#/definitions/FieldOrDatumDefWithCondition\"\n },\n {\n \"$ref\": \"#/definitions/ValueDefWithCondition\"\n }\n ]\n },\n \"MarkPropDef\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FieldOrDatumDefWithCondition\"\n },\n {\n \"$ref\": \"#/definitions/FieldOrDatumDefWithCondition\"\n },\n {\n \"$ref\": \"#/definitions/ValueDefWithCondition\"\n }\n ]\n },\n \"MarkType\": {\n \"enum\": [\n \"arc\",\n \"area\",\n \"image\",\n \"group\",\n \"line\",\n \"path\",\n \"rect\",\n \"rule\",\n \"shape\",\n \"symbol\",\n \"text\",\n \"trail\"\n ],\n \"type\": \"string\"\n },\n \"MergedStream\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"between\": {\n \"items\": {\n \"$ref\": \"#/definitions/Stream\"\n },\n \"type\": \"array\"\n },\n \"consume\": {\n \"type\": \"boolean\"\n },\n \"debounce\": {\n \"type\": \"number\"\n },\n \"filter\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Expr\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/Expr\"\n },\n \"type\": \"array\"\n }\n ]\n },\n \"markname\": {\n \"type\": \"string\"\n },\n \"marktype\": {\n \"$ref\": \"#/definitions/MarkType\"\n },\n \"merge\": {\n \"items\": {\n \"$ref\": \"#/definitions/Stream\"\n },\n \"type\": \"array\"\n },\n \"throttle\": {\n \"type\": \"number\"\n }\n },\n \"required\": [\n \"merge\"\n ],\n \"type\": \"object\"\n },\n \"Month\": {\n \"maximum\": 12,\n \"minimum\": 1,\n \"type\": \"number\"\n },\n \"MultiLineString\": {\n \"additionalProperties\": false,\n \"description\": \"MultiLineString geometry object. https://tools.ietf.org/html/rfc7946#section-3.1.5\",\n \"properties\": {\n \"bbox\": {\n \"$ref\": \"#/definitions/BBox\",\n \"description\": \"Bounding box of the coordinate range of the object's Geometries, Features, or Feature Collections. https://tools.ietf.org/html/rfc7946#section-5\"\n },\n \"coordinates\": {\n \"items\": {\n \"items\": {\n \"$ref\": \"#/definitions/Position\"\n },\n \"type\": \"array\"\n },\n \"type\": \"array\"\n },\n \"type\": {\n \"const\": \"MultiLineString\",\n \"description\": \"Specifies the type of GeoJSON object.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"coordinates\",\n \"type\"\n ],\n \"type\": \"object\"\n },\n \"MultiPoint\": {\n \"additionalProperties\": false,\n \"description\": \"MultiPoint geometry object. https://tools.ietf.org/html/rfc7946#section-3.1.3\",\n \"properties\": {\n \"bbox\": {\n \"$ref\": \"#/definitions/BBox\",\n \"description\": \"Bounding box of the coordinate range of the object's Geometries, Features, or Feature Collections. https://tools.ietf.org/html/rfc7946#section-5\"\n },\n \"coordinates\": {\n \"items\": {\n \"$ref\": \"#/definitions/Position\"\n },\n \"type\": \"array\"\n },\n \"type\": {\n \"const\": \"MultiPoint\",\n \"description\": \"Specifies the type of GeoJSON object.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"coordinates\",\n \"type\"\n ],\n \"type\": \"object\"\n },\n \"MultiPolygon\": {\n \"additionalProperties\": false,\n \"description\": \"MultiPolygon geometry object. https://tools.ietf.org/html/rfc7946#section-3.1.7\",\n \"properties\": {\n \"bbox\": {\n \"$ref\": \"#/definitions/BBox\",\n \"description\": \"Bounding box of the coordinate range of the object's Geometries, Features, or Feature Collections. https://tools.ietf.org/html/rfc7946#section-5\"\n },\n \"coordinates\": {\n \"items\": {\n \"items\": {\n \"items\": {\n \"$ref\": \"#/definitions/Position\"\n },\n \"type\": \"array\"\n },\n \"type\": \"array\"\n },\n \"type\": \"array\"\n },\n \"type\": {\n \"const\": \"MultiPolygon\",\n \"description\": \"Specifies the type of GeoJSON object.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"coordinates\",\n \"type\"\n ],\n \"type\": \"object\"\n },\n \"MultiTimeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LocalMultiTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/UtcMultiTimeUnit\"\n }\n ]\n },\n \"NamedData\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"format\": {\n \"$ref\": \"#/definitions/DataFormat\",\n \"description\": \"An object that specifies the format for parsing the data.\"\n },\n \"name\": {\n \"description\": \"Provide a placeholder name and bind data at runtime.\\n\\nNew data may change the layout but Vega does not always resize the chart. To update the layout when the data updates, set [autosize](https://vega.github.io/vega-lite/docs/size.html#autosize) or explicitly use [view.resize](https://vega.github.io/vega/docs/api/view/#view_resize).\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"NonArgAggregateOp\": {\n \"enum\": [\n \"average\",\n \"count\",\n \"distinct\",\n \"max\",\n \"mean\",\n \"median\",\n \"min\",\n \"missing\",\n \"product\",\n \"q1\",\n \"q3\",\n \"ci0\",\n \"ci1\",\n \"stderr\",\n \"stdev\",\n \"stdevp\",\n \"sum\",\n \"valid\",\n \"values\",\n \"variance\",\n \"variancep\"\n ],\n \"type\": \"string\"\n },\n \"NonLayerRepeatSpec\": {\n \"additionalProperties\": false,\n \"description\": \"Base interface for a repeat specification.\",\n \"properties\": {\n \"align\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LayoutAlign\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"The alignment to apply to grid rows and columns. The supported string values are `\\\"all\\\"`, `\\\"each\\\"`, and `\\\"none\\\"`.\\n\\n- For `\\\"none\\\"`, a flow layout will be used, in which adjacent subviews are simply placed one after the other.\\n- For `\\\"each\\\"`, subviews will be aligned into a clean grid structure, but each row or column may be of variable size.\\n- For `\\\"all\\\"`, subviews will be aligned and each row or column will be sized identically based on the maximum observed size. String values for this property will be applied to both grid rows and columns.\\n\\nAlternatively, an object value of the form `{\\\"row\\\": string, \\\"column\\\": string}` can be used to supply different alignments for rows and columns.\\n\\n__Default value:__ `\\\"all\\\"`.\"\n },\n \"bounds\": {\n \"description\": \"The bounds calculation method to use for determining the extent of a sub-plot. One of `full` (the default) or `flush`.\\n\\n- If set to `full`, the entire calculated bounds (including axes, title, and legend) will be used.\\n- If set to `flush`, only the specified width and height values for the sub-view will be used. The `flush` setting can be useful when attempting to place sub-plots without axes or legends into a uniform grid structure.\\n\\n__Default value:__ `\\\"full\\\"`\",\n \"enum\": [\n \"full\",\n \"flush\"\n ],\n \"type\": \"string\"\n },\n \"center\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"Boolean flag indicating if subviews should be centered relative to their respective rows or columns.\\n\\nAn object value of the form `{\\\"row\\\": boolean, \\\"column\\\": boolean}` can be used to supply different centering values for rows and columns.\\n\\n__Default value:__ `false`\"\n },\n \"columns\": {\n \"description\": \"The number of columns to include in the view composition layout.\\n\\n__Default value__: `undefined` -- An infinite number of columns (a single row) will be assumed. This is equivalent to `hconcat` (for `concat`) and to using the `column` channel (for `facet` and `repeat`).\\n\\n__Note__:\\n\\n1) This property is only for:\\n- the general (wrappable) `concat` operator (not `hconcat`/`vconcat`)\\n- the `facet` and `repeat` operator with one field/repetition definition (without row/column nesting)\\n\\n2) Setting the `columns` to `1` is equivalent to `vconcat` (for `concat`) and to using the `row` channel (for `facet` and `repeat`).\",\n \"type\": \"number\"\n },\n \"data\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Data\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object describing the data source. Set to `null` to ignore the parent's data source. If no data is set, it is derived from the parent.\"\n },\n \"description\": {\n \"description\": \"Description of this mark for commenting purpose.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the visualization for later reference.\",\n \"type\": \"string\"\n },\n \"repeat\": {\n \"anyOf\": [\n {\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/RepeatMapping\"\n }\n ],\n \"description\": \"Definition for fields to be repeated. One of: 1) An array of fields to be repeated. If `\\\"repeat\\\"` is an array, the field can be referred to as `{\\\"repeat\\\": \\\"repeat\\\"}`. The repeated views are laid out in a wrapped row. You can set the number of columns to control the wrapping. 2) An object that maps `\\\"row\\\"` and/or `\\\"column\\\"` to the listed fields to be repeated along the particular orientations. The objects `{\\\"repeat\\\": \\\"row\\\"}` and `{\\\"repeat\\\": \\\"column\\\"}` can be used to refer to the repeated field respectively.\"\n },\n \"resolve\": {\n \"$ref\": \"#/definitions/Resolve\",\n \"description\": \"Scale, axis, and legend resolutions for view composition specifications.\"\n },\n \"spacing\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"The spacing in pixels between sub-views of the composition operator. An object of the form `{\\\"row\\\": number, \\\"column\\\": number}` can be used to set different spacing values for rows and columns.\\n\\n__Default value__: Depends on `\\\"spacing\\\"` property of [the view composition configuration](https://vega.github.io/vega-lite/docs/config.html#view-config) (`20` by default)\"\n },\n \"spec\": {\n \"$ref\": \"#/definitions/NonNormalizedSpec\",\n \"description\": \"A specification of the view that gets repeated.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"$ref\": \"#/definitions/TitleParams\"\n }\n ],\n \"description\": \"Title for the plot.\"\n },\n \"transform\": {\n \"description\": \"An array of data transformations such as filter and new field calculation.\",\n \"items\": {\n \"$ref\": \"#/definitions/Transform\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"repeat\",\n \"spec\"\n ],\n \"type\": \"object\"\n },\n \"NonNormalizedSpec\": {\n \"$ref\": \"#/definitions/Spec\"\n },\n \"NumberLocale\": {\n \"additionalProperties\": false,\n \"description\": \"Locale definition for formatting numbers.\",\n \"properties\": {\n \"currency\": {\n \"$ref\": \"#/definitions/Vector2\",\n \"description\": \"The currency prefix and suffix (e.g., [\\\"$\\\", \\\"\\\"]).\"\n },\n \"decimal\": {\n \"description\": \"The decimal point (e.g., \\\".\\\").\",\n \"type\": \"string\"\n },\n \"grouping\": {\n \"description\": \"The array of group sizes (e.g., [3]), cycled as needed.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n \"minus\": {\n \"description\": \"The minus sign (defaults to hyphen-minus, \\\"-\\\").\",\n \"type\": \"string\"\n },\n \"nan\": {\n \"description\": \"The not-a-number value (defaults to \\\"NaN\\\").\",\n \"type\": \"string\"\n },\n \"numerals\": {\n \"$ref\": \"#/definitions/Vector10\",\n \"description\": \"An array of ten strings to replace the numerals 0-9.\"\n },\n \"percent\": {\n \"description\": \"The percent sign (defaults to \\\"%\\\").\",\n \"type\": \"string\"\n },\n \"thousands\": {\n \"description\": \"The group separator (e.g., \\\",\\\").\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"decimal\",\n \"thousands\",\n \"grouping\",\n \"currency\"\n ],\n \"type\": \"object\"\n },\n \"NumericArrayMarkPropDef\": {\n \"$ref\": \"#/definitions/MarkPropDef\"\n },\n \"NumericMarkPropDef\": {\n \"$ref\": \"#/definitions/MarkPropDef\"\n },\n \"OffsetDef\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ScaleFieldDef\"\n },\n {\n \"$ref\": \"#/definitions/ScaleDatumDef\"\n },\n {\n \"$ref\": \"#/definitions/ValueDef\"\n }\n ]\n },\n \"OrderFieldDef\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"const\": \"binned\",\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/SortOrder\",\n \"description\": \"The sort order. One of `\\\"ascending\\\"` (default) or `\\\"descending\\\"`.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"OrderOnlyDef\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"sort\": {\n \"$ref\": \"#/definitions/SortOrder\",\n \"description\": \"The sort order. One of `\\\"ascending\\\"` (default) or `\\\"descending\\\"`.\"\n }\n },\n \"type\": \"object\"\n },\n \"OrderValueDef\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"Orient\": {\n \"enum\": [\n \"left\",\n \"right\",\n \"top\",\n \"bottom\"\n ],\n \"type\": \"string\"\n },\n \"Orientation\": {\n \"enum\": [\n \"horizontal\",\n \"vertical\"\n ],\n \"type\": \"string\"\n },\n \"OverlayMarkDef\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"align\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Align\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The horizontal alignment of the text or ranged marks (area, bar, image, rect, rule). One of `\\\"left\\\"`, `\\\"right\\\"`, `\\\"center\\\"`.\\n\\n__Note:__ Expression reference is *not* supported for range marks.\"\n },\n \"angle\": {\n \"anyOf\": [\n {\n \"description\": \"The rotation angle of the text, in degrees.\",\n \"maximum\": 360,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"aria\": {\n \"anyOf\": [\n {\n \"description\": \"A boolean flag indicating if [ARIA attributes](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) should be included (SVG output only). If `false`, the \\\"aria-hidden\\\" attribute will be set on the output SVG element, removing the mark item from the ARIA accessibility tree.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"ariaRole\": {\n \"anyOf\": [\n {\n \"description\": \"Sets the type of user interface element of the mark item for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If specified, this property determines the \\\"role\\\" attribute. Warning: this property is experimental and may be changed in the future.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"ariaRoleDescription\": {\n \"anyOf\": [\n {\n \"description\": \"A human-readable, author-localized description for the role of the mark item for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If specified, this property determines the \\\"aria-roledescription\\\" attribute. Warning: this property is experimental and may be changed in the future.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"aspect\": {\n \"anyOf\": [\n {\n \"description\": \"Whether to keep aspect ratio of image marks.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"baseline\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextBaseline\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"For text marks, the vertical text baseline. One of `\\\"alphabetic\\\"` (default), `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`, `\\\"line-top\\\"`, `\\\"line-bottom\\\"`, or an expression reference that provides one of the valid values. The `\\\"line-top\\\"` and `\\\"line-bottom\\\"` values operate similarly to `\\\"top\\\"` and `\\\"bottom\\\"`, but are calculated relative to the `lineHeight` rather than `fontSize` alone.\\n\\nFor range marks, the vertical alignment of the marks. One of `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`.\\n\\n__Note:__ Expression reference is *not* supported for range marks.\"\n },\n \"blend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Blend\",\n \"description\": \"The color blend mode for drawing an item on its current background. Any valid [CSS mix-blend-mode](https://developer.mozilla.org/en-US/docs/Web/CSS/mix-blend-mode) value can be used.\\n\\n__Default value: `\\\"source-over\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"clip\": {\n \"description\": \"Whether a mark be clipped to the enclosing group’s width and height.\",\n \"type\": \"boolean\"\n },\n \"color\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default color.\\n\\n__Default value:__ ■ `\\\"#4682b4\\\"`\\n\\n__Note:__\\n- This property cannot be used in a [style config](https://vega.github.io/vega-lite/docs/mark.html#style-config).\\n- The `fill` and `stroke` properties have higher precedence than `color` and will override `color`.\"\n },\n \"cornerRadius\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles or arcs' corners.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusBottomLeft\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' bottom left corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusBottomRight\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' bottom right corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusTopLeft\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' top right corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusTopRight\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' top left corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cursor\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Cursor\",\n \"description\": \"The mouse cursor used over the mark. Any valid [CSS cursor type](https://developer.mozilla.org/en-US/docs/Web/CSS/cursor#Values) can be used.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"description\": {\n \"anyOf\": [\n {\n \"description\": \"A text description of the mark item for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If specified, this property determines the [\\\"aria-label\\\" attribute](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/ARIA_Techniques/Using_the_aria-label_attribute).\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"dir\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextDirection\",\n \"description\": \"The direction of the text. One of `\\\"ltr\\\"` (left-to-right) or `\\\"rtl\\\"` (right-to-left). This property determines on which side is truncated in response to the limit parameter.\\n\\n__Default value:__ `\\\"ltr\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"dx\": {\n \"anyOf\": [\n {\n \"description\": \"The horizontal offset, in pixels, between the text label and its anchor point. The offset is applied after rotation by the _angle_ property.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"dy\": {\n \"anyOf\": [\n {\n \"description\": \"The vertical offset, in pixels, between the text label and its anchor point. The offset is applied after rotation by the _angle_ property.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"ellipsis\": {\n \"anyOf\": [\n {\n \"description\": \"The ellipsis string for text truncated in response to the limit parameter.\\n\\n__Default value:__ `\\\"…\\\"`\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"endAngle\": {\n \"anyOf\": [\n {\n \"description\": \"The end angle in radians for arc marks. A value of `0` indicates up (north), increasing values proceed clockwise.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fill\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default fill color. This property has higher precedence than `config.color`. Set to `null` to remove fill.\\n\\n__Default value:__ (None)\"\n },\n \"fillOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"The fill opacity (value between [0,1]).\\n\\n__Default value:__ `1`\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"filled\": {\n \"description\": \"Whether the mark's color should be used as fill color instead of stroke color.\\n\\n__Default value:__ `false` for all `point`, `line`, and `rule` marks as well as `geoshape` marks for [`graticule`](https://vega.github.io/vega-lite/docs/data.html#graticule) data sources; otherwise, `true`.\\n\\n__Note:__ This property cannot be used in a [style config](https://vega.github.io/vega-lite/docs/mark.html#style-config).\",\n \"type\": \"boolean\"\n },\n \"font\": {\n \"anyOf\": [\n {\n \"description\": \"The typeface to set the text in (e.g., `\\\"Helvetica Neue\\\"`).\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontSize\": {\n \"anyOf\": [\n {\n \"description\": \"The font size, in pixels.\\n\\n__Default value:__ `11`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontStyle\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\",\n \"description\": \"The font style (e.g., `\\\"italic\\\"`).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontWeight\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\",\n \"description\": \"The font weight. This can be either a string (e.g `\\\"bold\\\"`, `\\\"normal\\\"`) or a number (`100`, `200`, `300`, ..., `900` where `\\\"normal\\\"` = `400` and `\\\"bold\\\"` = `700`).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"height\": {\n \"anyOf\": [\n {\n \"description\": \"Height of the marks.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"href\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/URI\",\n \"description\": \"A URL to load upon mouse click. If defined, the mark acts as a hyperlink.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"innerRadius\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The inner radius in pixels of arc marks. `innerRadius` is an alias for `radius2`.\\n\\n__Default value:__ `0`\",\n \"minimum\": 0\n },\n \"interpolate\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Interpolate\",\n \"description\": \"The line interpolation method to use for line and area marks. One of the following:\\n- `\\\"linear\\\"`: piecewise linear segments, as in a polyline.\\n- `\\\"linear-closed\\\"`: close the linear segments to form a polygon.\\n- `\\\"step\\\"`: alternate between horizontal and vertical segments, as in a step function.\\n- `\\\"step-before\\\"`: alternate between vertical and horizontal segments, as in a step function.\\n- `\\\"step-after\\\"`: alternate between horizontal and vertical segments, as in a step function.\\n- `\\\"basis\\\"`: a B-spline, with control point duplication on the ends.\\n- `\\\"basis-open\\\"`: an open B-spline; may not intersect the start or end.\\n- `\\\"basis-closed\\\"`: a closed B-spline, as in a loop.\\n- `\\\"cardinal\\\"`: a Cardinal spline, with control point duplication on the ends.\\n- `\\\"cardinal-open\\\"`: an open Cardinal spline; may not intersect the start or end, but will intersect other control points.\\n- `\\\"cardinal-closed\\\"`: a closed Cardinal spline, as in a loop.\\n- `\\\"bundle\\\"`: equivalent to basis, except the tension parameter is used to straighten the spline.\\n- `\\\"monotone\\\"`: cubic interpolation that preserves monotonicity in y.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"invalid\": {\n \"description\": \"Defines how Vega-Lite should handle marks for invalid values (`null` and `NaN`).\\n- If set to `\\\"filter\\\"` (default), all data items with null values will be skipped (for line, trail, and area marks) or filtered (for other marks).\\n- If `null`, all data items are included. In this case, invalid values will be interpreted as zeroes.\",\n \"enum\": [\n \"filter\",\n null\n ],\n \"type\": [\n \"string\",\n \"null\"\n ]\n },\n \"limit\": {\n \"anyOf\": [\n {\n \"description\": \"The maximum length of the text mark in pixels. The text value will be automatically truncated if the rendered size exceeds the limit.\\n\\n__Default value:__ `0` -- indicating no limit\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"lineBreak\": {\n \"anyOf\": [\n {\n \"description\": \"A delimiter, such as a newline character, upon which to break text strings into multiple lines. This property is ignored if the text is array-valued.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"lineHeight\": {\n \"anyOf\": [\n {\n \"description\": \"The line height in pixels (the spacing between subsequent lines of text) for multi-line text marks.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"opacity\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The overall opacity (value between [0,1]).\\n\\n__Default value:__ `0.7` for non-aggregate plots with `point`, `tick`, `circle`, or `square` marks or layered `bar` charts and `1` otherwise.\",\n \"maximum\": 1,\n \"minimum\": 0\n },\n \"order\": {\n \"description\": \"For line and trail marks, this `order` property can be set to `null` or `false` to make the lines use the original order in the data sources.\",\n \"type\": [\n \"null\",\n \"boolean\"\n ]\n },\n \"orient\": {\n \"$ref\": \"#/definitions/Orientation\",\n \"description\": \"The orientation of a non-stacked bar, tick, area, and line charts. The value is either horizontal (default) or vertical.\\n- For bar, rule and tick, this determines whether the size of the bar and tick should be applied to x or y dimension.\\n- For area, this property determines the orient property of the Vega output.\\n- For line and trail marks, this property determines the sort order of the points in the line if `config.sortLineBy` is not specified. For stacked charts, this is always determined by the orientation of the stack; therefore explicitly specified value will be ignored.\"\n },\n \"outerRadius\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The outer radius in pixels of arc marks. `outerRadius` is an alias for `radius`.\\n\\n__Default value:__ `0`\",\n \"minimum\": 0\n },\n \"padAngle\": {\n \"anyOf\": [\n {\n \"description\": \"The angular padding applied to sides of the arc, in radians.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"radius\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"For arc mark, the primary (outer) radius in pixels.\\n\\nFor text marks, polar coordinate radial offset, in pixels, of the text from the origin determined by the `x` and `y` properties.\\n\\n__Default value:__ `min(plot_width, plot_height)/2`\",\n \"minimum\": 0\n },\n \"radius2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The secondary (inner) radius in pixels of arc marks.\\n\\n__Default value:__ `0`\",\n \"minimum\": 0\n },\n \"radius2Offset\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Offset for radius2.\"\n },\n \"radiusOffset\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Offset for radius.\"\n },\n \"shape\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SymbolShape\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Shape of the point marks. Supported values include:\\n- plotting shapes: `\\\"circle\\\"`, `\\\"square\\\"`, `\\\"cross\\\"`, `\\\"diamond\\\"`, `\\\"triangle-up\\\"`, `\\\"triangle-down\\\"`, `\\\"triangle-right\\\"`, or `\\\"triangle-left\\\"`.\\n- the line symbol `\\\"stroke\\\"`\\n- centered directional shapes `\\\"arrow\\\"`, `\\\"wedge\\\"`, or `\\\"triangle\\\"`\\n- a custom [SVG path string](https://developer.mozilla.org/en-US/docs/Web/SVG/Tutorial/Paths) (For correct sizing, custom shape paths should be defined within a square bounding box with coordinates ranging from -1 to 1 along both the x and y dimensions.)\\n\\n__Default value:__ `\\\"circle\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"size\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default size for marks.\\n- For `point`/`circle`/`square`, this represents the pixel area of the marks. Note that this value sets the area of the symbol; the side lengths will increase with the square root of this value.\\n- For `bar`, this represents the band size of the bar, in pixels.\\n- For `text`, this represents the font size, in pixels.\\n\\n__Default value:__\\n- `30` for point, circle, square marks; width/height's `step`\\n- `2` for bar marks with discrete dimensions;\\n- `5` for bar marks with continuous dimensions;\\n- `11` for text marks.\",\n \"minimum\": 0\n },\n \"smooth\": {\n \"anyOf\": [\n {\n \"description\": \"A boolean flag (default true) indicating if the image should be smoothed when resized. If false, individual pixels should be scaled directly rather than interpolated with smoothing. For SVG rendering, this option may not work in some browsers due to lack of standardization.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"startAngle\": {\n \"anyOf\": [\n {\n \"description\": \"The start angle in radians for arc marks. A value of `0` indicates up (north), increasing values proceed clockwise.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"stroke\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default stroke color. This property has higher precedence than `config.color`. Set to `null` to remove stroke.\\n\\n__Default value:__ (None)\"\n },\n \"strokeCap\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeCap\",\n \"description\": \"The stroke cap for line ending style. One of `\\\"butt\\\"`, `\\\"round\\\"`, or `\\\"square\\\"`.\\n\\n__Default value:__ `\\\"butt\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeDash\": {\n \"anyOf\": [\n {\n \"description\": \"An array of alternating stroke, space lengths for creating dashed or dotted lines.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeDashOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The offset (in pixels) into which to begin drawing with the stroke dash array.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeJoin\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeJoin\",\n \"description\": \"The stroke line join method. One of `\\\"miter\\\"`, `\\\"round\\\"` or `\\\"bevel\\\"`.\\n\\n__Default value:__ `\\\"miter\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeMiterLimit\": {\n \"anyOf\": [\n {\n \"description\": \"The miter limit at which to bevel a line join.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The offset in pixels at which to draw the group stroke and fill. If unspecified, the default behavior is to dynamically offset stroked groups such that 1 pixel stroke widths align with the pixel grid.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"The stroke opacity (value between [0,1]).\\n\\n__Default value:__ `1`\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeWidth\": {\n \"anyOf\": [\n {\n \"description\": \"The stroke width, in pixels.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"style\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A string or array of strings indicating the name of custom styles to apply to the mark. A style is a named collection of mark property defaults defined within the [style configuration](https://vega.github.io/vega-lite/docs/mark.html#style-config). If style is an array, later styles will override earlier styles. Any [mark properties](https://vega.github.io/vega-lite/docs/encoding.html#mark-prop) explicitly defined within the `encoding` will override a style default.\\n\\n__Default value:__ The mark's name. For example, a bar mark will have style `\\\"bar\\\"` by default. __Note:__ Any specified style will augment the default style. For example, a bar mark with `\\\"style\\\": \\\"foo\\\"` will receive from `config.style.bar` and `config.style.foo` (the specified style `\\\"foo\\\"` has higher precedence).\"\n },\n \"tension\": {\n \"anyOf\": [\n {\n \"description\": \"Depending on the interpolation type, sets the tension parameter (for line and area marks).\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"text\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\",\n \"description\": \"Placeholder text if the `text` channel is not specified\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"theta\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"- For arc marks, the arc length in radians if theta2 is not specified, otherwise the start arc angle. (A value of 0 indicates up or “north”, increasing values proceed clockwise.)\\n\\n- For text marks, polar coordinate angle in radians.\",\n \"maximum\": 360,\n \"minimum\": 0\n },\n \"theta2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The end angle of arc marks in radians. A value of 0 indicates up or “north”, increasing values proceed clockwise.\"\n },\n \"theta2Offset\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Offset for theta2.\"\n },\n \"thetaOffset\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Offset for theta.\"\n },\n \"timeUnitBandPosition\": {\n \"description\": \"Default relative band position for a time unit. If set to `0`, the marks will be positioned at the beginning of the time unit band step. If set to `0.5`, the marks will be positioned in the middle of the time unit band step.\",\n \"type\": \"number\"\n },\n \"timeUnitBandSize\": {\n \"description\": \"Default relative band size for a time unit. If set to `1`, the bandwidth of the marks will be equal to the time unit band step. If set to `0.5`, bandwidth of the marks will be half of the time unit band step.\",\n \"type\": \"number\"\n },\n \"tooltip\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/TooltipContent\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"The tooltip text string to show upon mouse hover or an object defining which fields should the tooltip be derived from.\\n\\n- If `tooltip` is `true` or `{\\\"content\\\": \\\"encoding\\\"}`, then all fields from `encoding` will be used.\\n- If `tooltip` is `{\\\"content\\\": \\\"data\\\"}`, then all fields that appear in the highlighted data point will be used.\\n- If set to `null` or `false`, then no tooltip will be used.\\n\\nSee the [`tooltip`](https://vega.github.io/vega-lite/docs/tooltip.html) documentation for a detailed discussion about tooltip in Vega-Lite.\\n\\n__Default value:__ `null`\"\n },\n \"url\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/URI\",\n \"description\": \"The URL of the image file for image marks.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"width\": {\n \"anyOf\": [\n {\n \"description\": \"Width of the marks.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"x\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"X coordinates of the marks, or width of horizontal `\\\"bar\\\"` and `\\\"area\\\"` without specified `x2` or `width`.\\n\\nThe `value` of this channel can be a number or a string `\\\"width\\\"` for the width of the plot.\"\n },\n \"x2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"X2 coordinates for ranged `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"rect\\\"`, and `\\\"rule\\\"`.\\n\\nThe `value` of this channel can be a number or a string `\\\"width\\\"` for the width of the plot.\"\n },\n \"x2Offset\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Offset for x2-position.\"\n },\n \"xOffset\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Offset for x-position.\"\n },\n \"y\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Y coordinates of the marks, or height of vertical `\\\"bar\\\"` and `\\\"area\\\"` without specified `y2` or `height`.\\n\\nThe `value` of this channel can be a number or a string `\\\"height\\\"` for the height of the plot.\"\n },\n \"y2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Y2 coordinates for ranged `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"rect\\\"`, and `\\\"rule\\\"`.\\n\\nThe `value` of this channel can be a number or a string `\\\"height\\\"` for the height of the plot.\"\n },\n \"y2Offset\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Offset for y2-position.\"\n },\n \"yOffset\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Offset for y-position.\"\n }\n },\n \"type\": \"object\"\n },\n \"Padding\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"bottom\": {\n \"type\": \"number\"\n },\n \"left\": {\n \"type\": \"number\"\n },\n \"right\": {\n \"type\": \"number\"\n },\n \"top\": {\n \"type\": \"number\"\n }\n },\n \"type\": \"object\"\n }\n ],\n \"minimum\": 0\n },\n \"ParameterExtent\": {\n \"anyOf\": [\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"field\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"If a selection parameter is specified, the field name to extract selected values for when the selection is [projected](https://vega.github.io/vega-lite/docs/selection.html#project) over multiple fields or encodings.\"\n },\n \"param\": {\n \"$ref\": \"#/definitions/ParameterName\",\n \"description\": \"The name of a parameter.\"\n }\n },\n \"required\": [\n \"param\"\n ],\n \"type\": \"object\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"encoding\": {\n \"$ref\": \"#/definitions/SingleDefUnitChannel\",\n \"description\": \"If a selection parameter is specified, the encoding channel to extract selected values for when a selection is [projected](https://vega.github.io/vega-lite/docs/selection.html#project) over multiple fields or encodings.\"\n },\n \"param\": {\n \"$ref\": \"#/definitions/ParameterName\",\n \"description\": \"The name of a parameter.\"\n }\n },\n \"required\": [\n \"param\"\n ],\n \"type\": \"object\"\n }\n ]\n },\n \"ParameterName\": {\n \"type\": \"string\"\n },\n \"ParameterPredicate\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"empty\": {\n \"description\": \"For selection parameters, the predicate of empty selections returns true by default. Override this behavior, by setting this property `empty: false`.\",\n \"type\": \"boolean\"\n },\n \"param\": {\n \"$ref\": \"#/definitions/ParameterName\",\n \"description\": \"Filter using a parameter name.\"\n }\n },\n \"required\": [\n \"param\"\n ],\n \"type\": \"object\"\n },\n \"Parse\": {\n \"additionalProperties\": {\n \"$ref\": \"#/definitions/ParseValue\"\n },\n \"type\": \"object\"\n },\n \"ParseValue\": {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"const\": \"string\",\n \"type\": \"string\"\n },\n {\n \"const\": \"boolean\",\n \"type\": \"string\"\n },\n {\n \"const\": \"date\",\n \"type\": \"string\"\n },\n {\n \"const\": \"number\",\n \"type\": \"string\"\n }\n ]\n },\n \"PivotTransform\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"groupby\": {\n \"description\": \"The optional data fields to group by. If not specified, a single group containing all data objects will be used.\",\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"type\": \"array\"\n },\n \"limit\": {\n \"description\": \"An optional parameter indicating the maximum number of pivoted fields to generate. The default (`0`) applies no limit. The pivoted `pivot` names are sorted in ascending order prior to enforcing the limit. __Default value:__ `0`\",\n \"type\": \"number\"\n },\n \"op\": {\n \"$ref\": \"#/definitions/AggregateOp\",\n \"description\": \"The aggregation operation to apply to grouped `value` field values. __Default value:__ `sum`\"\n },\n \"pivot\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"The data field to pivot on. The unique values of this field become new field names in the output stream.\"\n },\n \"value\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"The data field to populate pivoted fields. The aggregate values of this field become the values of the new pivoted fields.\"\n }\n },\n \"required\": [\n \"pivot\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"Point\": {\n \"additionalProperties\": false,\n \"description\": \"Point geometry object. https://tools.ietf.org/html/rfc7946#section-3.1.2\",\n \"properties\": {\n \"bbox\": {\n \"$ref\": \"#/definitions/BBox\",\n \"description\": \"Bounding box of the coordinate range of the object's Geometries, Features, or Feature Collections. https://tools.ietf.org/html/rfc7946#section-5\"\n },\n \"coordinates\": {\n \"$ref\": \"#/definitions/Position\"\n },\n \"type\": {\n \"const\": \"Point\",\n \"description\": \"Specifies the type of GeoJSON object.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"coordinates\",\n \"type\"\n ],\n \"type\": \"object\"\n },\n \"PointSelectionConfig\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"clear\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Stream\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"boolean\"\n }\n ],\n \"description\": \"Clears the selection, emptying it of all values. This property can be a [Event Stream](https://vega.github.io/vega/docs/event-streams/) or `false` to disable clear.\\n\\n__Default value:__ `dblclick`.\\n\\n__See also:__ [`clear` examples ](https://vega.github.io/vega-lite/docs/selection.html#clear) in the documentation.\"\n },\n \"encodings\": {\n \"description\": \"An array of encoding channels. The corresponding data field values must match for a data tuple to fall within the selection.\\n\\n__See also:__ The [projection with `encodings` and `fields` section](https://vega.github.io/vega-lite/docs/selection.html#project) in the documentation.\",\n \"items\": {\n \"$ref\": \"#/definitions/SingleDefUnitChannel\"\n },\n \"type\": \"array\"\n },\n \"fields\": {\n \"description\": \"An array of field names whose values must match for a data tuple to fall within the selection.\\n\\n__See also:__ The [projection with `encodings` and `fields` section](https://vega.github.io/vega-lite/docs/selection.html#project) in the documentation.\",\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"type\": \"array\"\n },\n \"nearest\": {\n \"description\": \"When true, an invisible voronoi diagram is computed to accelerate discrete selection. The data value _nearest_ the mouse cursor is added to the selection.\\n\\n__Default value:__ `false`, which means that data values must be interacted with directly (e.g., clicked on) to be added to the selection.\\n\\n__See also:__ [`nearest` examples](https://vega.github.io/vega-lite/docs/selection.html#nearest) documentation.\",\n \"type\": \"boolean\"\n },\n \"on\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Stream\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"A [Vega event stream](https://vega.github.io/vega/docs/event-streams/) (object or selector) that triggers the selection. For interval selections, the event stream must specify a [start and end](https://vega.github.io/vega/docs/event-streams/#between-filters).\\n\\n__See also:__ [`on` examples](https://vega.github.io/vega-lite/docs/selection.html#on) in the documentation.\"\n },\n \"resolve\": {\n \"$ref\": \"#/definitions/SelectionResolution\",\n \"description\": \"With layered and multi-view displays, a strategy that determines how selections' data queries are resolved when applied in a filter transform, conditional encoding rule, or scale domain.\\n\\nOne of:\\n- `\\\"global\\\"` -- only one brush exists for the entire SPLOM. When the user begins to drag, any previous brushes are cleared, and a new one is constructed.\\n- `\\\"union\\\"` -- each cell contains its own brush, and points are highlighted if they lie within _any_ of these individual brushes.\\n- `\\\"intersect\\\"` -- each cell contains its own brush, and points are highlighted only if they fall within _all_ of these individual brushes.\\n\\n__Default value:__ `global`.\\n\\n__See also:__ [`resolve` examples](https://vega.github.io/vega-lite/docs/selection.html#resolve) in the documentation.\"\n },\n \"toggle\": {\n \"description\": \"Controls whether data values should be toggled (inserted or removed from a point selection) or only ever inserted into point selections.\\n\\nOne of:\\n- `true` -- the default behavior, which corresponds to `\\\"event.shiftKey\\\"`. As a result, data values are toggled when the user interacts with the shift-key pressed.\\n- `false` -- disables toggling behaviour; the selection will only ever contain a single data value corresponding to the most recent interaction.\\n- A [Vega expression](https://vega.github.io/vega/docs/expressions/) which is re-evaluated as the user interacts. If the expression evaluates to `true`, the data value is toggled into or out of the point selection. If the expression evaluates to `false`, the point selection is first cleared, and the data value is then inserted. For example, setting the value to the Vega expression `\\\"true\\\"` will toggle data values without the user pressing the shift-key.\\n\\n__Default value:__ `true`\\n\\n__See also:__ [`toggle` examples](https://vega.github.io/vega-lite/docs/selection.html#toggle) in the documentation.\",\n \"type\": [\n \"string\",\n \"boolean\"\n ]\n },\n \"type\": {\n \"const\": \"point\",\n \"description\": \"Determines the default event processing and data query for the selection. Vega-Lite currently supports two selection types:\\n\\n- `\\\"point\\\"` -- to select multiple discrete data values; the first value is selected on `click` and additional values toggled on shift-click.\\n- `\\\"interval\\\"` -- to select a continuous range of data values on `drag`.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"type\"\n ],\n \"type\": \"object\"\n },\n \"PointSelectionConfigWithoutType\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"clear\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Stream\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"boolean\"\n }\n ],\n \"description\": \"Clears the selection, emptying it of all values. This property can be a [Event Stream](https://vega.github.io/vega/docs/event-streams/) or `false` to disable clear.\\n\\n__Default value:__ `dblclick`.\\n\\n__See also:__ [`clear` examples ](https://vega.github.io/vega-lite/docs/selection.html#clear) in the documentation.\"\n },\n \"encodings\": {\n \"description\": \"An array of encoding channels. The corresponding data field values must match for a data tuple to fall within the selection.\\n\\n__See also:__ The [projection with `encodings` and `fields` section](https://vega.github.io/vega-lite/docs/selection.html#project) in the documentation.\",\n \"items\": {\n \"$ref\": \"#/definitions/SingleDefUnitChannel\"\n },\n \"type\": \"array\"\n },\n \"fields\": {\n \"description\": \"An array of field names whose values must match for a data tuple to fall within the selection.\\n\\n__See also:__ The [projection with `encodings` and `fields` section](https://vega.github.io/vega-lite/docs/selection.html#project) in the documentation.\",\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"type\": \"array\"\n },\n \"nearest\": {\n \"description\": \"When true, an invisible voronoi diagram is computed to accelerate discrete selection. The data value _nearest_ the mouse cursor is added to the selection.\\n\\n__Default value:__ `false`, which means that data values must be interacted with directly (e.g., clicked on) to be added to the selection.\\n\\n__See also:__ [`nearest` examples](https://vega.github.io/vega-lite/docs/selection.html#nearest) documentation.\",\n \"type\": \"boolean\"\n },\n \"on\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Stream\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"A [Vega event stream](https://vega.github.io/vega/docs/event-streams/) (object or selector) that triggers the selection. For interval selections, the event stream must specify a [start and end](https://vega.github.io/vega/docs/event-streams/#between-filters).\\n\\n__See also:__ [`on` examples](https://vega.github.io/vega-lite/docs/selection.html#on) in the documentation.\"\n },\n \"resolve\": {\n \"$ref\": \"#/definitions/SelectionResolution\",\n \"description\": \"With layered and multi-view displays, a strategy that determines how selections' data queries are resolved when applied in a filter transform, conditional encoding rule, or scale domain.\\n\\nOne of:\\n- `\\\"global\\\"` -- only one brush exists for the entire SPLOM. When the user begins to drag, any previous brushes are cleared, and a new one is constructed.\\n- `\\\"union\\\"` -- each cell contains its own brush, and points are highlighted if they lie within _any_ of these individual brushes.\\n- `\\\"intersect\\\"` -- each cell contains its own brush, and points are highlighted only if they fall within _all_ of these individual brushes.\\n\\n__Default value:__ `global`.\\n\\n__See also:__ [`resolve` examples](https://vega.github.io/vega-lite/docs/selection.html#resolve) in the documentation.\"\n },\n \"toggle\": {\n \"description\": \"Controls whether data values should be toggled (inserted or removed from a point selection) or only ever inserted into point selections.\\n\\nOne of:\\n- `true` -- the default behavior, which corresponds to `\\\"event.shiftKey\\\"`. As a result, data values are toggled when the user interacts with the shift-key pressed.\\n- `false` -- disables toggling behaviour; the selection will only ever contain a single data value corresponding to the most recent interaction.\\n- A [Vega expression](https://vega.github.io/vega/docs/expressions/) which is re-evaluated as the user interacts. If the expression evaluates to `true`, the data value is toggled into or out of the point selection. If the expression evaluates to `false`, the point selection is first cleared, and the data value is then inserted. For example, setting the value to the Vega expression `\\\"true\\\"` will toggle data values without the user pressing the shift-key.\\n\\n__Default value:__ `true`\\n\\n__See also:__ [`toggle` examples](https://vega.github.io/vega-lite/docs/selection.html#toggle) in the documentation.\",\n \"type\": [\n \"string\",\n \"boolean\"\n ]\n }\n },\n \"type\": \"object\"\n },\n \"PolarDef\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PositionFieldDefBase\"\n },\n {\n \"$ref\": \"#/definitions/PositionDatumDefBase\"\n },\n {\n \"$ref\": \"#/definitions/PositionValueDef\"\n }\n ]\n },\n \"Polygon\": {\n \"additionalProperties\": false,\n \"description\": \"Polygon geometry object. https://tools.ietf.org/html/rfc7946#section-3.1.6\",\n \"properties\": {\n \"bbox\": {\n \"$ref\": \"#/definitions/BBox\",\n \"description\": \"Bounding box of the coordinate range of the object's Geometries, Features, or Feature Collections. https://tools.ietf.org/html/rfc7946#section-5\"\n },\n \"coordinates\": {\n \"items\": {\n \"items\": {\n \"$ref\": \"#/definitions/Position\"\n },\n \"type\": \"array\"\n },\n \"type\": \"array\"\n },\n \"type\": {\n \"const\": \"Polygon\",\n \"description\": \"Specifies the type of GeoJSON object.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"coordinates\",\n \"type\"\n ],\n \"type\": \"object\"\n },\n \"Position\": {\n \"description\": \"A Position is an array of coordinates. https://tools.ietf.org/html/rfc7946#section-3.1.1 Array should contain between two and three elements. The previous GeoJSON specification allowed more elements (e.g., which could be used to represent M values), but the current specification only allows X, Y, and (optionally) Z to be defined.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n \"Position2Def\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SecondaryFieldDef\"\n },\n {\n \"$ref\": \"#/definitions/DatumDef\"\n },\n {\n \"$ref\": \"#/definitions/PositionValueDef\"\n }\n ]\n },\n \"PositionDatumDef\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"axis\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Axis\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of axis's gridlines, ticks and labels. If `null`, the axis for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [axis properties](https://vega.github.io/vega-lite/docs/axis.html) are applied.\\n\\n__See also:__ [`axis`](https://vega.github.io/vega-lite/docs/axis.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"impute\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ImputeParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining the properties of the Impute Operation to be applied. The field value of the other positional channel is taken as `key` of the `Impute` Operation. The field of the `color` channel if specified is used as `groupby` of the `Impute` Operation.\\n\\n__See also:__ [`impute`](https://vega.github.io/vega-lite/docs/impute.html) documentation.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"stack\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StackOffset\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"type\": \"boolean\"\n }\n ],\n \"description\": \"Type of stacking offset if the field should be stacked. `stack` is only applicable for `x`, `y`, `theta`, and `radius` channels with continuous domains. For example, `stack` of `y` can be used to customize stacking for a vertical bar chart.\\n\\n`stack` can be one of the following values:\\n- `\\\"zero\\\"` or `true`: stacking with baseline offset at zero value of the scale (for creating typical stacked [bar](https://vega.github.io/vega-lite/docs/stack.html#bar) and [area](https://vega.github.io/vega-lite/docs/stack.html#area) chart).\\n- `\\\"normalize\\\"` - stacking with normalized domain (for creating [normalized stacked bar and area charts](https://vega.github.io/vega-lite/docs/stack.html#normalized) and pie charts [with percentage tooltip](https://vega.github.io/vega-lite/docs/arc.html#tooltip)).
\\n-`\\\"center\\\"` - stacking with center baseline (for [streamgraph](https://vega.github.io/vega-lite/docs/stack.html#streamgraph)).\\n- `null` or `false` - No-stacking. This will produce layered [bar](https://vega.github.io/vega-lite/docs/stack.html#layered-bar-chart) and area chart.\\n\\n__Default value:__ `zero` for plots with all of the following conditions are true: (1) the mark is `bar`, `area`, or `arc`; (2) the stacked measure channel (x or y) has a linear scale; (3) At least one of non-position channels mapped to an unaggregated field that is different from x and y. Otherwise, `null` by default.\\n\\n__See also:__ [`stack`](https://vega.github.io/vega-lite/docs/stack.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"PositionDatumDefBase\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"stack\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StackOffset\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"type\": \"boolean\"\n }\n ],\n \"description\": \"Type of stacking offset if the field should be stacked. `stack` is only applicable for `x`, `y`, `theta`, and `radius` channels with continuous domains. For example, `stack` of `y` can be used to customize stacking for a vertical bar chart.\\n\\n`stack` can be one of the following values:\\n- `\\\"zero\\\"` or `true`: stacking with baseline offset at zero value of the scale (for creating typical stacked [bar](https://vega.github.io/vega-lite/docs/stack.html#bar) and [area](https://vega.github.io/vega-lite/docs/stack.html#area) chart).\\n- `\\\"normalize\\\"` - stacking with normalized domain (for creating [normalized stacked bar and area charts](https://vega.github.io/vega-lite/docs/stack.html#normalized) and pie charts [with percentage tooltip](https://vega.github.io/vega-lite/docs/arc.html#tooltip)).
\\n-`\\\"center\\\"` - stacking with center baseline (for [streamgraph](https://vega.github.io/vega-lite/docs/stack.html#streamgraph)).\\n- `null` or `false` - No-stacking. This will produce layered [bar](https://vega.github.io/vega-lite/docs/stack.html#layered-bar-chart) and area chart.\\n\\n__Default value:__ `zero` for plots with all of the following conditions are true: (1) the mark is `bar`, `area`, or `arc`; (2) the stacked measure channel (x or y) has a linear scale; (3) At least one of non-position channels mapped to an unaggregated field that is different from x and y. Otherwise, `null` by default.\\n\\n__See also:__ [`stack`](https://vega.github.io/vega-lite/docs/stack.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"PositionDef\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PositionFieldDef\"\n },\n {\n \"$ref\": \"#/definitions/PositionDatumDef\"\n },\n {\n \"$ref\": \"#/definitions/PositionValueDef\"\n }\n ]\n },\n \"PositionFieldDef\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"axis\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Axis\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of axis's gridlines, ticks and labels. If `null`, the axis for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [axis properties](https://vega.github.io/vega-lite/docs/axis.html) are applied.\\n\\n__See also:__ [`axis`](https://vega.github.io/vega-lite/docs/axis.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"const\": \"binned\",\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"impute\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ImputeParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining the properties of the Impute Operation to be applied. The field value of the other positional channel is taken as `key` of the `Impute` Operation. The field of the `color` channel if specified is used as `groupby` of the `Impute` Operation.\\n\\n__See also:__ [`impute`](https://vega.github.io/vega-lite/docs/impute.html) documentation.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"stack\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StackOffset\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"type\": \"boolean\"\n }\n ],\n \"description\": \"Type of stacking offset if the field should be stacked. `stack` is only applicable for `x`, `y`, `theta`, and `radius` channels with continuous domains. For example, `stack` of `y` can be used to customize stacking for a vertical bar chart.\\n\\n`stack` can be one of the following values:\\n- `\\\"zero\\\"` or `true`: stacking with baseline offset at zero value of the scale (for creating typical stacked [bar](https://vega.github.io/vega-lite/docs/stack.html#bar) and [area](https://vega.github.io/vega-lite/docs/stack.html#area) chart).\\n- `\\\"normalize\\\"` - stacking with normalized domain (for creating [normalized stacked bar and area charts](https://vega.github.io/vega-lite/docs/stack.html#normalized) and pie charts [with percentage tooltip](https://vega.github.io/vega-lite/docs/arc.html#tooltip)).
\\n-`\\\"center\\\"` - stacking with center baseline (for [streamgraph](https://vega.github.io/vega-lite/docs/stack.html#streamgraph)).\\n- `null` or `false` - No-stacking. This will produce layered [bar](https://vega.github.io/vega-lite/docs/stack.html#layered-bar-chart) and area chart.\\n\\n__Default value:__ `zero` for plots with all of the following conditions are true: (1) the mark is `bar`, `area`, or `arc`; (2) the stacked measure channel (x or y) has a linear scale; (3) At least one of non-position channels mapped to an unaggregated field that is different from x and y. Otherwise, `null` by default.\\n\\n__See also:__ [`stack`](https://vega.github.io/vega-lite/docs/stack.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"PositionFieldDefBase\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"const\": \"binned\",\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"stack\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StackOffset\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"type\": \"boolean\"\n }\n ],\n \"description\": \"Type of stacking offset if the field should be stacked. `stack` is only applicable for `x`, `y`, `theta`, and `radius` channels with continuous domains. For example, `stack` of `y` can be used to customize stacking for a vertical bar chart.\\n\\n`stack` can be one of the following values:\\n- `\\\"zero\\\"` or `true`: stacking with baseline offset at zero value of the scale (for creating typical stacked [bar](https://vega.github.io/vega-lite/docs/stack.html#bar) and [area](https://vega.github.io/vega-lite/docs/stack.html#area) chart).\\n- `\\\"normalize\\\"` - stacking with normalized domain (for creating [normalized stacked bar and area charts](https://vega.github.io/vega-lite/docs/stack.html#normalized) and pie charts [with percentage tooltip](https://vega.github.io/vega-lite/docs/arc.html#tooltip)).
\\n-`\\\"center\\\"` - stacking with center baseline (for [streamgraph](https://vega.github.io/vega-lite/docs/stack.html#streamgraph)).\\n- `null` or `false` - No-stacking. This will produce layered [bar](https://vega.github.io/vega-lite/docs/stack.html#layered-bar-chart) and area chart.\\n\\n__Default value:__ `zero` for plots with all of the following conditions are true: (1) the mark is `bar`, `area`, or `arc`; (2) the stacked measure channel (x or y) has a linear scale; (3) At least one of non-position channels mapped to an unaggregated field that is different from x and y. Otherwise, `null` by default.\\n\\n__See also:__ [`stack`](https://vega.github.io/vega-lite/docs/stack.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"PositionValueDef\": {\n \"$ref\": \"#/definitions/ValueDef<(number|\\\"width\\\"|\\\"height\\\"|ExprRef)>\"\n },\n \"Predicate\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FieldEqualPredicate\"\n },\n {\n \"$ref\": \"#/definitions/FieldRangePredicate\"\n },\n {\n \"$ref\": \"#/definitions/FieldOneOfPredicate\"\n },\n {\n \"$ref\": \"#/definitions/FieldLTPredicate\"\n },\n {\n \"$ref\": \"#/definitions/FieldGTPredicate\"\n },\n {\n \"$ref\": \"#/definitions/FieldLTEPredicate\"\n },\n {\n \"$ref\": \"#/definitions/FieldGTEPredicate\"\n },\n {\n \"$ref\": \"#/definitions/FieldValidPredicate\"\n },\n {\n \"$ref\": \"#/definitions/ParameterPredicate\"\n },\n {\n \"type\": \"string\"\n }\n ]\n },\n \"PrimitiveValue\": {\n \"type\": [\n \"number\",\n \"string\",\n \"boolean\",\n \"null\"\n ]\n },\n \"Projection\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"center\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Vector2\",\n \"description\": \"The projection's center, a two-element array of longitude and latitude in degrees.\\n\\n__Default value:__ `[0, 0]`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"clipAngle\": {\n \"anyOf\": [\n {\n \"description\": \"The projection's clipping circle radius to the specified angle in degrees. If `null`, switches to [antimeridian](http://bl.ocks.org/mbostock/3788999) cutting rather than small-circle clipping.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"clipExtent\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Vector2>\",\n \"description\": \"The projection's viewport clip extent to the specified bounds in pixels. The extent bounds are specified as an array `[[x0, y0], [x1, y1]]`, where `x0` is the left-side of the viewport, `y0` is the top, `x1` is the right and `y1` is the bottom. If `null`, no viewport clipping is performed.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"coefficient\": {\n \"anyOf\": [\n {\n \"description\": \"The coefficient parameter for the `hammer` projection.\\n\\n__Default value:__ `2`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"distance\": {\n \"anyOf\": [\n {\n \"description\": \"For the `satellite` projection, the distance from the center of the sphere to the point of view, as a proportion of the sphere’s radius. The recommended maximum clip angle for a given `distance` is acos(1 / distance) converted to degrees. If tilt is also applied, then more conservative clipping may be necessary.\\n\\n__Default value:__ `2.0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"extent\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Vector2>\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Fit\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/Fit\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fraction\": {\n \"anyOf\": [\n {\n \"description\": \"The fraction parameter for the `bottomley` projection.\\n\\n__Default value:__ `0.5`, corresponding to a sin(ψ) where ψ = π/6.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"lobes\": {\n \"anyOf\": [\n {\n \"description\": \"The number of lobes in projections that support multi-lobe views: `berghaus`, `gingery`, or `healpix`. The default value varies based on the projection type.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"parallel\": {\n \"anyOf\": [\n {\n \"description\": \"The parallel parameter for projections that support it: `armadillo`, `bonne`, `craig`, `cylindricalEqualArea`, `cylindricalStereographic`, `hammerRetroazimuthal`, `loximuthal`, or `rectangularPolyconic`. The default value varies based on the projection type.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"parallels\": {\n \"anyOf\": [\n {\n \"description\": \"For conic projections, the [two standard parallels](https://en.wikipedia.org/wiki/Map_projection#Conic) that define the map layout. The default depends on the specific conic projection used.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"pointRadius\": {\n \"anyOf\": [\n {\n \"description\": \"The default radius (in pixels) to use when drawing GeoJSON `Point` and `MultiPoint` geometries. This parameter sets a constant default value. To modify the point radius in response to data, see the corresponding parameter of the GeoPath and GeoShape transforms.\\n\\n__Default value:__ `4.5`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"precision\": {\n \"anyOf\": [\n {\n \"description\": \"The threshold for the projection's [adaptive resampling](http://bl.ocks.org/mbostock/3795544) to the specified value in pixels. This value corresponds to the [Douglas–Peucker distance](http://en.wikipedia.org/wiki/Ramer%E2%80%93Douglas%E2%80%93Peucker_algorithm). If precision is not specified, returns the projection's current resampling precision which defaults to `√0.5 ≅ 0.70710…`.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"radius\": {\n \"anyOf\": [\n {\n \"description\": \"The radius parameter for the `airy` or `gingery` projection. The default value varies based on the projection type.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"ratio\": {\n \"anyOf\": [\n {\n \"description\": \"The ratio parameter for the `hill`, `hufnagel`, or `wagner` projections. The default value varies based on the projection type.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"reflectX\": {\n \"anyOf\": [\n {\n \"description\": \"Sets whether or not the x-dimension is reflected (negated) in the output.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"reflectY\": {\n \"anyOf\": [\n {\n \"description\": \"Sets whether or not the y-dimension is reflected (negated) in the output.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"rotate\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Vector2\"\n },\n {\n \"$ref\": \"#/definitions/Vector3\"\n }\n ],\n \"description\": \"The projection's three-axis rotation to the specified angles, which must be a two- or three-element array of numbers [`lambda`, `phi`, `gamma`] specifying the rotation angles in degrees about each spherical axis. (These correspond to yaw, pitch and roll.)\\n\\n__Default value:__ `[0, 0, 0]`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The projection’s scale (zoom) factor, overriding automatic fitting. The default scale is projection-specific. The scale factor corresponds linearly to the distance between projected points; however, scale factor values are not equivalent across projections.\"\n },\n \"size\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Vector2\",\n \"description\": \"Used in conjunction with fit, provides the width and height in pixels of the area to which the projection should be automatically fit.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"spacing\": {\n \"anyOf\": [\n {\n \"description\": \"The spacing parameter for the `lagrange` projection.\\n\\n__Default value:__ `0.5`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"tilt\": {\n \"anyOf\": [\n {\n \"description\": \"The tilt angle (in degrees) for the `satellite` projection.\\n\\n__Default value:__ `0`.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"translate\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Vector2\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The projection’s translation offset as a two-element array `[tx, ty]`.\"\n },\n \"type\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ProjectionType\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The cartographic projection to use. This value is case-insensitive, for example `\\\"albers\\\"` and `\\\"Albers\\\"` indicate the same projection type. You can find all valid projection types [in the documentation](https://vega.github.io/vega-lite/docs/projection.html#projection-types).\\n\\n__Default value:__ `equalEarth`\"\n }\n },\n \"type\": \"object\"\n },\n \"ProjectionConfig\": {\n \"$ref\": \"#/definitions/Projection\",\n \"description\": \"Any property of Projection can be in config\"\n },\n \"ProjectionType\": {\n \"enum\": [\n \"albers\",\n \"albersUsa\",\n \"azimuthalEqualArea\",\n \"azimuthalEquidistant\",\n \"conicConformal\",\n \"conicEqualArea\",\n \"conicEquidistant\",\n \"equalEarth\",\n \"equirectangular\",\n \"gnomonic\",\n \"identity\",\n \"mercator\",\n \"naturalEarth1\",\n \"orthographic\",\n \"stereographic\",\n \"transverseMercator\"\n ],\n \"type\": \"string\"\n },\n \"QuantileTransform\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"as\": {\n \"description\": \"The output field names for the probability and quantile values.\\n\\n__Default value:__ `[\\\"prob\\\", \\\"value\\\"]`\",\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"maxItems\": 2,\n \"minItems\": 2,\n \"type\": \"array\"\n },\n \"groupby\": {\n \"description\": \"The data fields to group by. If not specified, a single group containing all data objects will be used.\",\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"type\": \"array\"\n },\n \"probs\": {\n \"description\": \"An array of probabilities in the range (0, 1) for which to compute quantile values. If not specified, the *step* parameter will be used.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n \"quantile\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"The data field for which to perform quantile estimation.\"\n },\n \"step\": {\n \"description\": \"A probability step size (default 0.01) for sampling quantile values. All values from one-half the step size up to 1 (exclusive) will be sampled. This parameter is only used if the *probs* parameter is not provided.\",\n \"type\": \"number\"\n }\n },\n \"required\": [\n \"quantile\"\n ],\n \"type\": \"object\"\n },\n \"RadialGradient\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"gradient\": {\n \"const\": \"radial\",\n \"description\": \"The type of gradient. Use `\\\"radial\\\"` for a radial gradient.\",\n \"type\": \"string\"\n },\n \"id\": {\n \"type\": \"string\"\n },\n \"r1\": {\n \"description\": \"The radius length, in normalized [0, 1] coordinates, of the inner circle for the gradient.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n \"r2\": {\n \"description\": \"The radius length, in normalized [0, 1] coordinates, of the outer circle for the gradient.\\n\\n__Default value:__ `0.5`\",\n \"type\": \"number\"\n },\n \"stops\": {\n \"description\": \"An array of gradient stops defining the gradient color sequence.\",\n \"items\": {\n \"$ref\": \"#/definitions/GradientStop\"\n },\n \"type\": \"array\"\n },\n \"x1\": {\n \"description\": \"The x-coordinate, in normalized [0, 1] coordinates, for the center of the inner circle for the gradient.\\n\\n__Default value:__ `0.5`\",\n \"type\": \"number\"\n },\n \"x2\": {\n \"description\": \"The x-coordinate, in normalized [0, 1] coordinates, for the center of the outer circle for the gradient.\\n\\n__Default value:__ `0.5`\",\n \"type\": \"number\"\n },\n \"y1\": {\n \"description\": \"The y-coordinate, in normalized [0, 1] coordinates, for the center of the inner circle for the gradient.\\n\\n__Default value:__ `0.5`\",\n \"type\": \"number\"\n },\n \"y2\": {\n \"description\": \"The y-coordinate, in normalized [0, 1] coordinates, for the center of the outer circle for the gradient.\\n\\n__Default value:__ `0.5`\",\n \"type\": \"number\"\n }\n },\n \"required\": [\n \"gradient\",\n \"stops\"\n ],\n \"type\": \"object\"\n },\n \"RangeConfig\": {\n \"additionalProperties\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/RangeScheme\"\n },\n {\n \"type\": \"array\"\n }\n ]\n },\n \"properties\": {\n \"category\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/RangeScheme\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/Color\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"Default [color scheme](https://vega.github.io/vega/docs/schemes/) for categorical data.\"\n },\n \"diverging\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/RangeScheme\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/Color\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"Default [color scheme](https://vega.github.io/vega/docs/schemes/) for diverging quantitative ramps.\"\n },\n \"heatmap\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/RangeScheme\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/Color\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"Default [color scheme](https://vega.github.io/vega/docs/schemes/) for quantitative heatmaps.\"\n },\n \"ordinal\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/RangeScheme\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/Color\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"Default [color scheme](https://vega.github.io/vega/docs/schemes/) for rank-ordered data.\"\n },\n \"ramp\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/RangeScheme\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/Color\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"Default [color scheme](https://vega.github.io/vega/docs/schemes/) for sequential quantitative ramps.\"\n },\n \"symbol\": {\n \"description\": \"Array of [symbol](https://vega.github.io/vega/docs/marks/symbol/) names or paths for the default shape palette.\",\n \"items\": {\n \"$ref\": \"#/definitions/SymbolShape\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"RangeEnum\": {\n \"enum\": [\n \"width\",\n \"height\",\n \"symbol\",\n \"category\",\n \"ordinal\",\n \"ramp\",\n \"diverging\",\n \"heatmap\"\n ],\n \"type\": \"string\"\n },\n \"RangeRaw\": {\n \"items\": {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"type\": \"boolean\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/RangeRawArray\"\n }\n ]\n },\n \"type\": \"array\"\n },\n \"RangeRawArray\": {\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n \"RangeScheme\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/RangeEnum\"\n },\n {\n \"$ref\": \"#/definitions/RangeRaw\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"count\": {\n \"type\": \"number\"\n },\n \"extent\": {\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n \"scheme\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ColorScheme\"\n }\n ]\n }\n },\n \"required\": [\n \"scheme\"\n ],\n \"type\": \"object\"\n }\n ]\n },\n \"RectConfig\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"align\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Align\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The horizontal alignment of the text or ranged marks (area, bar, image, rect, rule). One of `\\\"left\\\"`, `\\\"right\\\"`, `\\\"center\\\"`.\\n\\n__Note:__ Expression reference is *not* supported for range marks.\"\n },\n \"angle\": {\n \"anyOf\": [\n {\n \"description\": \"The rotation angle of the text, in degrees.\",\n \"maximum\": 360,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"aria\": {\n \"anyOf\": [\n {\n \"description\": \"A boolean flag indicating if [ARIA attributes](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) should be included (SVG output only). If `false`, the \\\"aria-hidden\\\" attribute will be set on the output SVG element, removing the mark item from the ARIA accessibility tree.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"ariaRole\": {\n \"anyOf\": [\n {\n \"description\": \"Sets the type of user interface element of the mark item for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If specified, this property determines the \\\"role\\\" attribute. Warning: this property is experimental and may be changed in the future.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"ariaRoleDescription\": {\n \"anyOf\": [\n {\n \"description\": \"A human-readable, author-localized description for the role of the mark item for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If specified, this property determines the \\\"aria-roledescription\\\" attribute. Warning: this property is experimental and may be changed in the future.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"aspect\": {\n \"anyOf\": [\n {\n \"description\": \"Whether to keep aspect ratio of image marks.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"baseline\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextBaseline\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"For text marks, the vertical text baseline. One of `\\\"alphabetic\\\"` (default), `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`, `\\\"line-top\\\"`, `\\\"line-bottom\\\"`, or an expression reference that provides one of the valid values. The `\\\"line-top\\\"` and `\\\"line-bottom\\\"` values operate similarly to `\\\"top\\\"` and `\\\"bottom\\\"`, but are calculated relative to the `lineHeight` rather than `fontSize` alone.\\n\\nFor range marks, the vertical alignment of the marks. One of `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`.\\n\\n__Note:__ Expression reference is *not* supported for range marks.\"\n },\n \"binSpacing\": {\n \"description\": \"Offset between bars for binned field. The ideal value for this is either 0 (preferred by statisticians) or 1 (Vega-Lite default, D3 example style).\\n\\n__Default value:__ `1`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"blend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Blend\",\n \"description\": \"The color blend mode for drawing an item on its current background. Any valid [CSS mix-blend-mode](https://developer.mozilla.org/en-US/docs/Web/CSS/mix-blend-mode) value can be used.\\n\\n__Default value: `\\\"source-over\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"color\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default color.\\n\\n__Default value:__ ■ `\\\"#4682b4\\\"`\\n\\n__Note:__\\n- This property cannot be used in a [style config](https://vega.github.io/vega-lite/docs/mark.html#style-config).\\n- The `fill` and `stroke` properties have higher precedence than `color` and will override `color`.\"\n },\n \"continuousBandSize\": {\n \"description\": \"The default size of the bars on continuous scales.\\n\\n__Default value:__ `5`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"cornerRadius\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles or arcs' corners.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusBottomLeft\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' bottom left corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusBottomRight\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' bottom right corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusTopLeft\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' top right corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusTopRight\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' top left corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cursor\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Cursor\",\n \"description\": \"The mouse cursor used over the mark. Any valid [CSS cursor type](https://developer.mozilla.org/en-US/docs/Web/CSS/cursor#Values) can be used.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"description\": {\n \"anyOf\": [\n {\n \"description\": \"A text description of the mark item for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If specified, this property determines the [\\\"aria-label\\\" attribute](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/ARIA_Techniques/Using_the_aria-label_attribute).\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"dir\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextDirection\",\n \"description\": \"The direction of the text. One of `\\\"ltr\\\"` (left-to-right) or `\\\"rtl\\\"` (right-to-left). This property determines on which side is truncated in response to the limit parameter.\\n\\n__Default value:__ `\\\"ltr\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"discreteBandSize\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/RelativeBandSize\"\n }\n ],\n \"description\": \"The default size of the bars with discrete dimensions. If unspecified, the default size is `step-2`, which provides 2 pixel offset between bars.\",\n \"minimum\": 0\n },\n \"dx\": {\n \"anyOf\": [\n {\n \"description\": \"The horizontal offset, in pixels, between the text label and its anchor point. The offset is applied after rotation by the _angle_ property.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"dy\": {\n \"anyOf\": [\n {\n \"description\": \"The vertical offset, in pixels, between the text label and its anchor point. The offset is applied after rotation by the _angle_ property.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"ellipsis\": {\n \"anyOf\": [\n {\n \"description\": \"The ellipsis string for text truncated in response to the limit parameter.\\n\\n__Default value:__ `\\\"…\\\"`\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"endAngle\": {\n \"anyOf\": [\n {\n \"description\": \"The end angle in radians for arc marks. A value of `0` indicates up (north), increasing values proceed clockwise.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fill\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default fill color. This property has higher precedence than `config.color`. Set to `null` to remove fill.\\n\\n__Default value:__ (None)\"\n },\n \"fillOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"The fill opacity (value between [0,1]).\\n\\n__Default value:__ `1`\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"filled\": {\n \"description\": \"Whether the mark's color should be used as fill color instead of stroke color.\\n\\n__Default value:__ `false` for all `point`, `line`, and `rule` marks as well as `geoshape` marks for [`graticule`](https://vega.github.io/vega-lite/docs/data.html#graticule) data sources; otherwise, `true`.\\n\\n__Note:__ This property cannot be used in a [style config](https://vega.github.io/vega-lite/docs/mark.html#style-config).\",\n \"type\": \"boolean\"\n },\n \"font\": {\n \"anyOf\": [\n {\n \"description\": \"The typeface to set the text in (e.g., `\\\"Helvetica Neue\\\"`).\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontSize\": {\n \"anyOf\": [\n {\n \"description\": \"The font size, in pixels.\\n\\n__Default value:__ `11`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontStyle\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\",\n \"description\": \"The font style (e.g., `\\\"italic\\\"`).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontWeight\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\",\n \"description\": \"The font weight. This can be either a string (e.g `\\\"bold\\\"`, `\\\"normal\\\"`) or a number (`100`, `200`, `300`, ..., `900` where `\\\"normal\\\"` = `400` and `\\\"bold\\\"` = `700`).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"height\": {\n \"anyOf\": [\n {\n \"description\": \"Height of the marks.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"href\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/URI\",\n \"description\": \"A URL to load upon mouse click. If defined, the mark acts as a hyperlink.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"innerRadius\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The inner radius in pixels of arc marks. `innerRadius` is an alias for `radius2`.\\n\\n__Default value:__ `0`\",\n \"minimum\": 0\n },\n \"interpolate\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Interpolate\",\n \"description\": \"The line interpolation method to use for line and area marks. One of the following:\\n- `\\\"linear\\\"`: piecewise linear segments, as in a polyline.\\n- `\\\"linear-closed\\\"`: close the linear segments to form a polygon.\\n- `\\\"step\\\"`: alternate between horizontal and vertical segments, as in a step function.\\n- `\\\"step-before\\\"`: alternate between vertical and horizontal segments, as in a step function.\\n- `\\\"step-after\\\"`: alternate between horizontal and vertical segments, as in a step function.\\n- `\\\"basis\\\"`: a B-spline, with control point duplication on the ends.\\n- `\\\"basis-open\\\"`: an open B-spline; may not intersect the start or end.\\n- `\\\"basis-closed\\\"`: a closed B-spline, as in a loop.\\n- `\\\"cardinal\\\"`: a Cardinal spline, with control point duplication on the ends.\\n- `\\\"cardinal-open\\\"`: an open Cardinal spline; may not intersect the start or end, but will intersect other control points.\\n- `\\\"cardinal-closed\\\"`: a closed Cardinal spline, as in a loop.\\n- `\\\"bundle\\\"`: equivalent to basis, except the tension parameter is used to straighten the spline.\\n- `\\\"monotone\\\"`: cubic interpolation that preserves monotonicity in y.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"invalid\": {\n \"description\": \"Defines how Vega-Lite should handle marks for invalid values (`null` and `NaN`).\\n- If set to `\\\"filter\\\"` (default), all data items with null values will be skipped (for line, trail, and area marks) or filtered (for other marks).\\n- If `null`, all data items are included. In this case, invalid values will be interpreted as zeroes.\",\n \"enum\": [\n \"filter\",\n null\n ],\n \"type\": [\n \"string\",\n \"null\"\n ]\n },\n \"limit\": {\n \"anyOf\": [\n {\n \"description\": \"The maximum length of the text mark in pixels. The text value will be automatically truncated if the rendered size exceeds the limit.\\n\\n__Default value:__ `0` -- indicating no limit\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"lineBreak\": {\n \"anyOf\": [\n {\n \"description\": \"A delimiter, such as a newline character, upon which to break text strings into multiple lines. This property is ignored if the text is array-valued.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"lineHeight\": {\n \"anyOf\": [\n {\n \"description\": \"The line height in pixels (the spacing between subsequent lines of text) for multi-line text marks.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"minBandSize\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The minimum band size for bar and rectangle marks. __Default value:__ `0.25`\"\n },\n \"opacity\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The overall opacity (value between [0,1]).\\n\\n__Default value:__ `0.7` for non-aggregate plots with `point`, `tick`, `circle`, or `square` marks or layered `bar` charts and `1` otherwise.\",\n \"maximum\": 1,\n \"minimum\": 0\n },\n \"order\": {\n \"description\": \"For line and trail marks, this `order` property can be set to `null` or `false` to make the lines use the original order in the data sources.\",\n \"type\": [\n \"null\",\n \"boolean\"\n ]\n },\n \"orient\": {\n \"$ref\": \"#/definitions/Orientation\",\n \"description\": \"The orientation of a non-stacked bar, tick, area, and line charts. The value is either horizontal (default) or vertical.\\n- For bar, rule and tick, this determines whether the size of the bar and tick should be applied to x or y dimension.\\n- For area, this property determines the orient property of the Vega output.\\n- For line and trail marks, this property determines the sort order of the points in the line if `config.sortLineBy` is not specified. For stacked charts, this is always determined by the orientation of the stack; therefore explicitly specified value will be ignored.\"\n },\n \"outerRadius\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The outer radius in pixels of arc marks. `outerRadius` is an alias for `radius`.\\n\\n__Default value:__ `0`\",\n \"minimum\": 0\n },\n \"padAngle\": {\n \"anyOf\": [\n {\n \"description\": \"The angular padding applied to sides of the arc, in radians.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"radius\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"For arc mark, the primary (outer) radius in pixels.\\n\\nFor text marks, polar coordinate radial offset, in pixels, of the text from the origin determined by the `x` and `y` properties.\\n\\n__Default value:__ `min(plot_width, plot_height)/2`\",\n \"minimum\": 0\n },\n \"radius2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The secondary (inner) radius in pixels of arc marks.\\n\\n__Default value:__ `0`\",\n \"minimum\": 0\n },\n \"shape\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SymbolShape\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Shape of the point marks. Supported values include:\\n- plotting shapes: `\\\"circle\\\"`, `\\\"square\\\"`, `\\\"cross\\\"`, `\\\"diamond\\\"`, `\\\"triangle-up\\\"`, `\\\"triangle-down\\\"`, `\\\"triangle-right\\\"`, or `\\\"triangle-left\\\"`.\\n- the line symbol `\\\"stroke\\\"`\\n- centered directional shapes `\\\"arrow\\\"`, `\\\"wedge\\\"`, or `\\\"triangle\\\"`\\n- a custom [SVG path string](https://developer.mozilla.org/en-US/docs/Web/SVG/Tutorial/Paths) (For correct sizing, custom shape paths should be defined within a square bounding box with coordinates ranging from -1 to 1 along both the x and y dimensions.)\\n\\n__Default value:__ `\\\"circle\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"size\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default size for marks.\\n- For `point`/`circle`/`square`, this represents the pixel area of the marks. Note that this value sets the area of the symbol; the side lengths will increase with the square root of this value.\\n- For `bar`, this represents the band size of the bar, in pixels.\\n- For `text`, this represents the font size, in pixels.\\n\\n__Default value:__\\n- `30` for point, circle, square marks; width/height's `step`\\n- `2` for bar marks with discrete dimensions;\\n- `5` for bar marks with continuous dimensions;\\n- `11` for text marks.\",\n \"minimum\": 0\n },\n \"smooth\": {\n \"anyOf\": [\n {\n \"description\": \"A boolean flag (default true) indicating if the image should be smoothed when resized. If false, individual pixels should be scaled directly rather than interpolated with smoothing. For SVG rendering, this option may not work in some browsers due to lack of standardization.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"startAngle\": {\n \"anyOf\": [\n {\n \"description\": \"The start angle in radians for arc marks. A value of `0` indicates up (north), increasing values proceed clockwise.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"stroke\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default stroke color. This property has higher precedence than `config.color`. Set to `null` to remove stroke.\\n\\n__Default value:__ (None)\"\n },\n \"strokeCap\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeCap\",\n \"description\": \"The stroke cap for line ending style. One of `\\\"butt\\\"`, `\\\"round\\\"`, or `\\\"square\\\"`.\\n\\n__Default value:__ `\\\"butt\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeDash\": {\n \"anyOf\": [\n {\n \"description\": \"An array of alternating stroke, space lengths for creating dashed or dotted lines.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeDashOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The offset (in pixels) into which to begin drawing with the stroke dash array.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeJoin\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeJoin\",\n \"description\": \"The stroke line join method. One of `\\\"miter\\\"`, `\\\"round\\\"` or `\\\"bevel\\\"`.\\n\\n__Default value:__ `\\\"miter\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeMiterLimit\": {\n \"anyOf\": [\n {\n \"description\": \"The miter limit at which to bevel a line join.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The offset in pixels at which to draw the group stroke and fill. If unspecified, the default behavior is to dynamically offset stroked groups such that 1 pixel stroke widths align with the pixel grid.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"The stroke opacity (value between [0,1]).\\n\\n__Default value:__ `1`\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeWidth\": {\n \"anyOf\": [\n {\n \"description\": \"The stroke width, in pixels.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"tension\": {\n \"anyOf\": [\n {\n \"description\": \"Depending on the interpolation type, sets the tension parameter (for line and area marks).\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"text\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\",\n \"description\": \"Placeholder text if the `text` channel is not specified\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"theta\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"- For arc marks, the arc length in radians if theta2 is not specified, otherwise the start arc angle. (A value of 0 indicates up or “north”, increasing values proceed clockwise.)\\n\\n- For text marks, polar coordinate angle in radians.\",\n \"maximum\": 360,\n \"minimum\": 0\n },\n \"theta2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The end angle of arc marks in radians. A value of 0 indicates up or “north”, increasing values proceed clockwise.\"\n },\n \"timeUnitBandPosition\": {\n \"description\": \"Default relative band position for a time unit. If set to `0`, the marks will be positioned at the beginning of the time unit band step. If set to `0.5`, the marks will be positioned in the middle of the time unit band step.\",\n \"type\": \"number\"\n },\n \"timeUnitBandSize\": {\n \"description\": \"Default relative band size for a time unit. If set to `1`, the bandwidth of the marks will be equal to the time unit band step. If set to `0.5`, bandwidth of the marks will be half of the time unit band step.\",\n \"type\": \"number\"\n },\n \"tooltip\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/TooltipContent\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"The tooltip text string to show upon mouse hover or an object defining which fields should the tooltip be derived from.\\n\\n- If `tooltip` is `true` or `{\\\"content\\\": \\\"encoding\\\"}`, then all fields from `encoding` will be used.\\n- If `tooltip` is `{\\\"content\\\": \\\"data\\\"}`, then all fields that appear in the highlighted data point will be used.\\n- If set to `null` or `false`, then no tooltip will be used.\\n\\nSee the [`tooltip`](https://vega.github.io/vega-lite/docs/tooltip.html) documentation for a detailed discussion about tooltip in Vega-Lite.\\n\\n__Default value:__ `null`\"\n },\n \"url\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/URI\",\n \"description\": \"The URL of the image file for image marks.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"width\": {\n \"anyOf\": [\n {\n \"description\": \"Width of the marks.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"x\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"X coordinates of the marks, or width of horizontal `\\\"bar\\\"` and `\\\"area\\\"` without specified `x2` or `width`.\\n\\nThe `value` of this channel can be a number or a string `\\\"width\\\"` for the width of the plot.\"\n },\n \"x2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"X2 coordinates for ranged `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"rect\\\"`, and `\\\"rule\\\"`.\\n\\nThe `value` of this channel can be a number or a string `\\\"width\\\"` for the width of the plot.\"\n },\n \"y\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Y coordinates of the marks, or height of vertical `\\\"bar\\\"` and `\\\"area\\\"` without specified `y2` or `height`.\\n\\nThe `value` of this channel can be a number or a string `\\\"height\\\"` for the height of the plot.\"\n },\n \"y2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Y2 coordinates for ranged `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"rect\\\"`, and `\\\"rule\\\"`.\\n\\nThe `value` of this channel can be a number or a string `\\\"height\\\"` for the height of the plot.\"\n }\n },\n \"type\": \"object\"\n },\n \"RegressionTransform\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"as\": {\n \"description\": \"The output field names for the smoothed points generated by the regression transform.\\n\\n__Default value:__ The field names of the input x and y values.\",\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"maxItems\": 2,\n \"minItems\": 2,\n \"type\": \"array\"\n },\n \"extent\": {\n \"description\": \"A [min, max] domain over the independent (x) field for the starting and ending points of the generated trend line.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"maxItems\": 2,\n \"minItems\": 2,\n \"type\": \"array\"\n },\n \"groupby\": {\n \"description\": \"The data fields to group by. If not specified, a single group containing all data objects will be used.\",\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"type\": \"array\"\n },\n \"method\": {\n \"description\": \"The functional form of the regression model. One of `\\\"linear\\\"`, `\\\"log\\\"`, `\\\"exp\\\"`, `\\\"pow\\\"`, `\\\"quad\\\"`, or `\\\"poly\\\"`.\\n\\n__Default value:__ `\\\"linear\\\"`\",\n \"enum\": [\n \"linear\",\n \"log\",\n \"exp\",\n \"pow\",\n \"quad\",\n \"poly\"\n ],\n \"type\": \"string\"\n },\n \"on\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"The data field of the independent variable to use a predictor.\"\n },\n \"order\": {\n \"description\": \"The polynomial order (number of coefficients) for the 'poly' method.\\n\\n__Default value:__ `3`\",\n \"type\": \"number\"\n },\n \"params\": {\n \"description\": \"A boolean flag indicating if the transform should return the regression model parameters (one object per group), rather than trend line points. The resulting objects include a `coef` array of fitted coefficient values (starting with the intercept term and then including terms of increasing order) and an `rSquared` value (indicating the total variance explained by the model).\\n\\n__Default value:__ `false`\",\n \"type\": \"boolean\"\n },\n \"regression\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"The data field of the dependent variable to predict.\"\n }\n },\n \"required\": [\n \"regression\",\n \"on\"\n ],\n \"type\": \"object\"\n },\n \"RelativeBandSize\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"band\": {\n \"description\": \"The relative band size. For example `0.5` means half of the band scale's band width.\",\n \"type\": \"number\"\n }\n },\n \"required\": [\n \"band\"\n ],\n \"type\": \"object\"\n },\n \"RepeatMapping\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"column\": {\n \"description\": \"An array of fields to be repeated horizontally.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"row\": {\n \"description\": \"An array of fields to be repeated vertically.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"RepeatRef\": {\n \"additionalProperties\": false,\n \"description\": \"Reference to a repeated value.\",\n \"properties\": {\n \"repeat\": {\n \"enum\": [\n \"row\",\n \"column\",\n \"repeat\",\n \"layer\"\n ],\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"repeat\"\n ],\n \"type\": \"object\"\n },\n \"RepeatSpec\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/NonLayerRepeatSpec\"\n },\n {\n \"$ref\": \"#/definitions/LayerRepeatSpec\"\n }\n ]\n },\n \"Resolve\": {\n \"additionalProperties\": false,\n \"description\": \"Defines how scales, axes, and legends from different specs should be combined. Resolve is a mapping from `scale`, `axis`, and `legend` to a mapping from channels to resolutions. Scales and guides can be resolved to be `\\\"independent\\\"` or `\\\"shared\\\"`.\",\n \"properties\": {\n \"axis\": {\n \"$ref\": \"#/definitions/AxisResolveMap\"\n },\n \"legend\": {\n \"$ref\": \"#/definitions/LegendResolveMap\"\n },\n \"scale\": {\n \"$ref\": \"#/definitions/ScaleResolveMap\"\n }\n },\n \"type\": \"object\"\n },\n \"ResolveMode\": {\n \"enum\": [\n \"independent\",\n \"shared\"\n ],\n \"type\": \"string\"\n },\n \"RowCol\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"column\": {\n \"$ref\": \"#/definitions/LayoutAlign\"\n },\n \"row\": {\n \"$ref\": \"#/definitions/LayoutAlign\"\n }\n },\n \"type\": \"object\"\n },\n \"RowCol\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"column\": {\n \"type\": \"boolean\"\n },\n \"row\": {\n \"type\": \"boolean\"\n }\n },\n \"type\": \"object\"\n },\n \"RowCol\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"column\": {\n \"type\": \"number\"\n },\n \"row\": {\n \"type\": \"number\"\n }\n },\n \"type\": \"object\"\n },\n \"RowColumnEncodingFieldDef\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"align\": {\n \"$ref\": \"#/definitions/LayoutAlign\",\n \"description\": \"The alignment to apply to row/column facet's subplot. The supported string values are `\\\"all\\\"`, `\\\"each\\\"`, and `\\\"none\\\"`.\\n\\n- For `\\\"none\\\"`, a flow layout will be used, in which adjacent subviews are simply placed one after the other.\\n- For `\\\"each\\\"`, subviews will be aligned into a clean grid structure, but each row or column may be of variable size.\\n- For `\\\"all\\\"`, subviews will be aligned and each row or column will be sized identically based on the maximum observed size. String values for this property will be applied to both grid rows and columns.\\n\\n__Default value:__ `\\\"all\\\"`.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"center\": {\n \"description\": \"Boolean flag indicating if facet's subviews should be centered relative to their respective rows or columns.\\n\\n__Default value:__ `false`\",\n \"type\": \"boolean\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"header\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Header\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of a facet's header.\"\n },\n \"sort\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SortArray\"\n },\n {\n \"$ref\": \"#/definitions/SortOrder\"\n },\n {\n \"$ref\": \"#/definitions/EncodingSortField\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` is not supported for `row` and `column`.\"\n },\n \"spacing\": {\n \"description\": \"The spacing in pixels between facet's sub-views.\\n\\n__Default value__: Depends on `\\\"spacing\\\"` property of [the view composition configuration](https://vega.github.io/vega-lite/docs/config.html#view-config) (`20` by default)\",\n \"type\": \"number\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"SampleTransform\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"sample\": {\n \"description\": \"The maximum number of data objects to include in the sample.\\n\\n__Default value:__ `1000`\",\n \"type\": \"number\"\n }\n },\n \"required\": [\n \"sample\"\n ],\n \"type\": \"object\"\n },\n \"Scale\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"align\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The alignment of the steps within the scale range.\\n\\nThis value must lie in the range `[0,1]`. A value of `0.5` indicates that the steps should be centered within the range. A value of `0` or `1` may be used to shift the bands to one side, say to position them adjacent to an axis.\\n\\n__Default value:__ `0.5`\"\n },\n \"base\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The logarithm base of the `log` scale (default `10`).\"\n },\n \"bins\": {\n \"$ref\": \"#/definitions/ScaleBins\",\n \"description\": \"Bin boundaries can be provided to scales as either an explicit array of bin boundaries or as a bin specification object. The legal values are:\\n- An [array](../types/#Array) literal of bin boundary values. For example, `[0, 5, 10, 15, 20]`. The array must include both starting and ending boundaries. The previous example uses five values to indicate a total of four bin intervals: [0-5), [5-10), [10-15), [15-20]. Array literals may include signal references as elements.\\n- A [bin specification object](https://vega.github.io/vega-lite/docs/scale.html#bins) that indicates the bin _step_ size, and optionally the _start_ and _stop_ boundaries.\\n- An array of bin boundaries over the scale domain. If provided, axes and legends will use the bin boundaries to inform the choice of tick marks and text labels.\"\n },\n \"clamp\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"If `true`, values that exceed the data domain are clamped to either the minimum or maximum range value\\n\\n__Default value:__ derived from the [scale config](https://vega.github.io/vega-lite/docs/config.html#scale-config)'s `clamp` (`true` by default).\"\n },\n \"constant\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant determining the slope of the symlog function around zero. Only used for `symlog` scales.\\n\\n__Default value:__ `1`\"\n },\n \"domain\": {\n \"anyOf\": [\n {\n \"items\": {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"number\"\n },\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"type\": \"array\"\n },\n {\n \"const\": \"unaggregated\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ParameterExtent\"\n },\n {\n \"$ref\": \"#/definitions/DomainUnionWith\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Customized domain values in the form of constant values or dynamic values driven by a parameter.\\n\\n1) Constant `domain` for _quantitative_ fields can take one of the following forms:\\n\\n- A two-element array with minimum and maximum values. To create a diverging scale, this two-element array can be combined with the `domainMid` property.\\n- An array with more than two entries, for [Piecewise quantitative scales](https://vega.github.io/vega-lite/docs/scale.html#piecewise).\\n- A string value `\\\"unaggregated\\\"`, if the input field is aggregated, to indicate that the domain should include the raw data values prior to the aggregation.\\n\\n2) Constant `domain` for _temporal_ fields can be a two-element array with minimum and maximum values, in the form of either timestamps or the [DateTime definition objects](https://vega.github.io/vega-lite/docs/types.html#datetime).\\n\\n3) Constant `domain` for _ordinal_ and _nominal_ fields can be an array that lists valid input values.\\n\\n4) To combine (union) specified constant domain with the field's values, `domain` can be an object with a `unionWith` property that specify constant domain to be combined. For example, `domain: {unionWith: [0, 100]}` for a quantitative scale means that the scale domain always includes `[0, 100]`, but will include other values in the fields beyond `[0, 100]`.\\n\\n5) Domain can also takes an object defining a field or encoding of a parameter that [interactively determines](https://vega.github.io/vega-lite/docs/selection.html#scale-domains) the scale domain.\"\n },\n \"domainMax\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Sets the maximum value in the scale domain, overriding the `domain` property. This property is only intended for use with scales having continuous domains.\"\n },\n \"domainMid\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Inserts a single mid-point value into a two-element domain. The mid-point value must lie between the domain minimum and maximum values. This property can be useful for setting a midpoint for [diverging color scales](https://vega.github.io/vega-lite/docs/scale.html#piecewise). The domainMid property is only intended for use with scales supporting continuous, piecewise domains.\"\n },\n \"domainMin\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Sets the minimum value in the scale domain, overriding the domain property. This property is only intended for use with scales having continuous domains.\"\n },\n \"domainRaw\": {\n \"$ref\": \"#/definitions/ExprRef\",\n \"description\": \"An expression for an array of raw values that, if non-null, directly overrides the _domain_ property. This is useful for supporting interactions such as panning or zooming a scale. The scale may be initially determined using a data-driven domain, then modified in response to user input by setting the rawDomain value.\"\n },\n \"exponent\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The exponent of the `pow` scale.\"\n },\n \"interpolate\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ScaleInterpolateEnum\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/ScaleInterpolateParams\"\n }\n ],\n \"description\": \"The interpolation method for range values. By default, a general interpolator for numbers, dates, strings and colors (in HCL space) is used. For color ranges, this property allows interpolation in alternative color spaces. Legal values include `rgb`, `hsl`, `hsl-long`, `lab`, `hcl`, `hcl-long`, `cubehelix` and `cubehelix-long` ('-long' variants use longer paths in polar coordinate spaces). If object-valued, this property accepts an object with a string-valued _type_ property and an optional numeric _gamma_ property applicable to rgb and cubehelix interpolators. For more, see the [d3-interpolate documentation](https://github.com/d3/d3-interpolate).\\n\\n* __Default value:__ `hcl`\"\n },\n \"nice\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/TimeInterval\"\n },\n {\n \"$ref\": \"#/definitions/TimeIntervalStep\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Extending the domain so that it starts and ends on nice round values. This method typically modifies the scale’s domain, and may only extend the bounds to the nearest round value. Nicing is useful if the domain is computed from data and may be irregular. For example, for a domain of _[0.201479…, 0.996679…]_, a nice domain might be _[0.2, 1.0]_.\\n\\nFor quantitative scales such as linear, `nice` can be either a boolean flag or a number. If `nice` is a number, it will represent a desired tick count. This allows greater control over the step size used to extend the bounds, guaranteeing that the returned ticks will exactly cover the domain.\\n\\nFor temporal fields with time and utc scales, the `nice` value can be a string indicating the desired time interval. Legal values are `\\\"millisecond\\\"`, `\\\"second\\\"`, `\\\"minute\\\"`, `\\\"hour\\\"`, `\\\"day\\\"`, `\\\"week\\\"`, `\\\"month\\\"`, and `\\\"year\\\"`. Alternatively, `time` and `utc` scales can accept an object-valued interval specifier of the form `{\\\"interval\\\": \\\"month\\\", \\\"step\\\": 3}`, which includes a desired number of interval steps. Here, the domain would snap to quarter (Jan, Apr, Jul, Oct) boundaries.\\n\\n__Default value:__ `true` for unbinned _quantitative_ fields without explicit domain bounds; `false` otherwise.\"\n },\n \"padding\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"For _[continuous](https://vega.github.io/vega-lite/docs/scale.html#continuous)_ scales, expands the scale domain to accommodate the specified number of pixels on each of the scale range. The scale range must represent pixels for this parameter to function as intended. Padding adjustment is performed prior to all other adjustments, including the effects of the `zero`, `nice`, `domainMin`, and `domainMax` properties.\\n\\nFor _[band](https://vega.github.io/vega-lite/docs/scale.html#band)_ scales, shortcut for setting `paddingInner` and `paddingOuter` to the same value.\\n\\nFor _[point](https://vega.github.io/vega-lite/docs/scale.html#point)_ scales, alias for `paddingOuter`.\\n\\n__Default value:__ For _continuous_ scales, derived from the [scale config](https://vega.github.io/vega-lite/docs/scale.html#config)'s `continuousPadding`. For _band and point_ scales, see `paddingInner` and `paddingOuter`. By default, Vega-Lite sets padding such that _width/height = number of unique values * step_.\",\n \"minimum\": 0\n },\n \"paddingInner\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The inner padding (spacing) within each band step of band scales, as a fraction of the step size. This value must lie in the range [0,1].\\n\\nFor point scale, this property is invalid as point scales do not have internal band widths (only step sizes between bands).\\n\\n__Default value:__ derived from the [scale config](https://vega.github.io/vega-lite/docs/scale.html#config)'s `bandPaddingInner`.\",\n \"maximum\": 1,\n \"minimum\": 0\n },\n \"paddingOuter\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The outer padding (spacing) at the ends of the range of band and point scales, as a fraction of the step size. This value must lie in the range [0,1].\\n\\n__Default value:__ derived from the [scale config](https://vega.github.io/vega-lite/docs/scale.html#config)'s `bandPaddingOuter` for band scales and `pointPadding` for point scales. By default, Vega-Lite sets outer padding such that _width/height = number of unique values * step_.\",\n \"maximum\": 1,\n \"minimum\": 0\n },\n \"range\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/RangeEnum\"\n },\n {\n \"items\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/FieldRange\"\n }\n ],\n \"description\": \"The range of the scale. One of:\\n\\n- A string indicating a [pre-defined named scale range](https://vega.github.io/vega-lite/docs/scale.html#range-config) (e.g., example, `\\\"symbol\\\"`, or `\\\"diverging\\\"`).\\n\\n- For [continuous scales](https://vega.github.io/vega-lite/docs/scale.html#continuous), two-element array indicating minimum and maximum values, or an array with more than two entries for specifying a [piecewise scale](https://vega.github.io/vega-lite/docs/scale.html#piecewise).\\n\\n- For [discrete](https://vega.github.io/vega-lite/docs/scale.html#discrete) and [discretizing](https://vega.github.io/vega-lite/docs/scale.html#discretizing) scales, an array of desired output values or an object with a `field` property representing the range values. For example, if a field `color` contains CSS color names, we can set `range` to `{field: \\\"color\\\"}`.\\n\\n__Notes:__\\n\\n1) For color scales you can also specify a color [`scheme`](https://vega.github.io/vega-lite/docs/scale.html#scheme) instead of `range`.\\n\\n2) Any directly specified `range` for `x` and `y` channels will be ignored. Range can be customized via the view's corresponding [size](https://vega.github.io/vega-lite/docs/size.html) (`width` and `height`).\"\n },\n \"rangeMax\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Sets the maximum value in the scale range, overriding the `range` property or the default range. This property is only intended for use with scales having continuous ranges.\"\n },\n \"rangeMin\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Sets the minimum value in the scale range, overriding the `range` property or the default range. This property is only intended for use with scales having continuous ranges.\"\n },\n \"reverse\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"If true, reverses the order of the scale range. __Default value:__ `false`.\"\n },\n \"round\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"If `true`, rounds numeric output values to integers. This can be helpful for snapping to the pixel grid.\\n\\n__Default value:__ `false`.\"\n },\n \"scheme\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ColorScheme\"\n },\n {\n \"$ref\": \"#/definitions/SchemeParams\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A string indicating a color [scheme](https://vega.github.io/vega-lite/docs/scale.html#scheme) name (e.g., `\\\"category10\\\"` or `\\\"blues\\\"`) or a [scheme parameter object](https://vega.github.io/vega-lite/docs/scale.html#scheme-params).\\n\\nDiscrete color schemes may be used with [discrete](https://vega.github.io/vega-lite/docs/scale.html#discrete) or [discretizing](https://vega.github.io/vega-lite/docs/scale.html#discretizing) scales. Continuous color schemes are intended for use with color scales.\\n\\nFor the full list of supported schemes, please refer to the [Vega Scheme](https://vega.github.io/vega/docs/schemes/#reference) reference.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/ScaleType\",\n \"description\": \"The type of scale. Vega-Lite supports the following categories of scale types:\\n\\n1) [**Continuous Scales**](https://vega.github.io/vega-lite/docs/scale.html#continuous) -- mapping continuous domains to continuous output ranges ([`\\\"linear\\\"`](https://vega.github.io/vega-lite/docs/scale.html#linear), [`\\\"pow\\\"`](https://vega.github.io/vega-lite/docs/scale.html#pow), [`\\\"sqrt\\\"`](https://vega.github.io/vega-lite/docs/scale.html#sqrt), [`\\\"symlog\\\"`](https://vega.github.io/vega-lite/docs/scale.html#symlog), [`\\\"log\\\"`](https://vega.github.io/vega-lite/docs/scale.html#log), [`\\\"time\\\"`](https://vega.github.io/vega-lite/docs/scale.html#time), [`\\\"utc\\\"`](https://vega.github.io/vega-lite/docs/scale.html#utc).\\n\\n2) [**Discrete Scales**](https://vega.github.io/vega-lite/docs/scale.html#discrete) -- mapping discrete domains to discrete ([`\\\"ordinal\\\"`](https://vega.github.io/vega-lite/docs/scale.html#ordinal)) or continuous ([`\\\"band\\\"`](https://vega.github.io/vega-lite/docs/scale.html#band) and [`\\\"point\\\"`](https://vega.github.io/vega-lite/docs/scale.html#point)) output ranges.\\n\\n3) [**Discretizing Scales**](https://vega.github.io/vega-lite/docs/scale.html#discretizing) -- mapping continuous domains to discrete output ranges [`\\\"bin-ordinal\\\"`](https://vega.github.io/vega-lite/docs/scale.html#bin-ordinal), [`\\\"quantile\\\"`](https://vega.github.io/vega-lite/docs/scale.html#quantile), [`\\\"quantize\\\"`](https://vega.github.io/vega-lite/docs/scale.html#quantize) and [`\\\"threshold\\\"`](https://vega.github.io/vega-lite/docs/scale.html#threshold).\\n\\n__Default value:__ please see the [scale type table](https://vega.github.io/vega-lite/docs/scale.html#type).\"\n },\n \"zero\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"If `true`, ensures that a zero baseline value is included in the scale domain.\\n\\n__Default value:__ `true` for x and y channels if the quantitative field is not binned and no custom `domain` is provided; `false` otherwise.\\n\\n__Note:__ Log, time, and utc scales do not support `zero`.\"\n }\n },\n \"type\": \"object\"\n },\n \"ScaleBinParams\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"start\": {\n \"description\": \"The starting (lowest-valued) bin boundary.\\n\\n__Default value:__ The lowest value of the scale domain will be used.\",\n \"type\": \"number\"\n },\n \"step\": {\n \"description\": \"The step size defining the bin interval width.\",\n \"type\": \"number\"\n },\n \"stop\": {\n \"description\": \"The stopping (highest-valued) bin boundary.\\n\\n__Default value:__ The highest value of the scale domain will be used.\",\n \"type\": \"number\"\n }\n },\n \"required\": [\n \"step\"\n ],\n \"type\": \"object\"\n },\n \"ScaleBins\": {\n \"anyOf\": [\n {\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ScaleBinParams\"\n }\n ]\n },\n \"ScaleConfig\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"bandPaddingInner\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default inner padding for `x` and `y` band scales.\\n\\n__Default value:__\\n- `nestedOffsetPaddingInner` for x/y scales with nested x/y offset scales.\\n- `barBandPaddingInner` for bar marks (`0.1` by default)\\n- `rectBandPaddingInner` for rect and other marks (`0` by default)\",\n \"maximum\": 1,\n \"minimum\": 0\n },\n \"bandPaddingOuter\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default outer padding for `x` and `y` band scales.\\n\\n__Default value:__ `paddingInner/2` (which makes _width/height = number of unique values * step_)\",\n \"maximum\": 1,\n \"minimum\": 0\n },\n \"bandWithNestedOffsetPaddingInner\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default inner padding for `x` and `y` band scales with nested `xOffset` and `yOffset` encoding.\\n\\n__Default value:__ `0.2`\",\n \"maximum\": 1,\n \"minimum\": 0\n },\n \"bandWithNestedOffsetPaddingOuter\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default outer padding for `x` and `y` band scales with nested `xOffset` and `yOffset` encoding.\\n\\n__Default value:__ `0.2`\",\n \"maximum\": 1,\n \"minimum\": 0\n },\n \"barBandPaddingInner\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default inner padding for `x` and `y` band-ordinal scales of `\\\"bar\\\"` marks.\\n\\n__Default value:__ `0.1`\",\n \"maximum\": 1,\n \"minimum\": 0\n },\n \"clamp\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"If true, values that exceed the data domain are clamped to either the minimum or maximum range value\"\n },\n \"continuousPadding\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default padding for continuous x/y scales.\\n\\n__Default:__ The bar width for continuous x-scale of a vertical bar and continuous y-scale of a horizontal bar.; `0` otherwise.\",\n \"minimum\": 0\n },\n \"maxBandSize\": {\n \"description\": \"The default max value for mapping quantitative fields to bar's size/bandSize.\\n\\nIf undefined (default), we will use the axis's size (width or height) - 1.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"maxFontSize\": {\n \"description\": \"The default max value for mapping quantitative fields to text's size/fontSize.\\n\\n__Default value:__ `40`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"maxOpacity\": {\n \"description\": \"Default max opacity for mapping a field to opacity.\\n\\n__Default value:__ `0.8`\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"maxSize\": {\n \"description\": \"Default max value for point size scale.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"maxStrokeWidth\": {\n \"description\": \"Default max strokeWidth for the scale of strokeWidth for rule and line marks and of size for trail marks.\\n\\n__Default value:__ `4`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"minBandSize\": {\n \"description\": \"The default min value for mapping quantitative fields to bar and tick's size/bandSize scale with zero=false.\\n\\n__Default value:__ `2`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"minFontSize\": {\n \"description\": \"The default min value for mapping quantitative fields to tick's size/fontSize scale with zero=false\\n\\n__Default value:__ `8`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"minOpacity\": {\n \"description\": \"Default minimum opacity for mapping a field to opacity.\\n\\n__Default value:__ `0.3`\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"minSize\": {\n \"description\": \"Default minimum value for point size scale with zero=false.\\n\\n__Default value:__ `9`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"minStrokeWidth\": {\n \"description\": \"Default minimum strokeWidth for the scale of strokeWidth for rule and line marks and of size for trail marks with zero=false.\\n\\n__Default value:__ `1`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"offsetBandPaddingInner\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default padding inner for xOffset/yOffset's band scales.\\n\\n__Default Value:__ `0`\"\n },\n \"offsetBandPaddingOuter\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default padding outer for xOffset/yOffset's band scales.\\n\\n__Default Value:__ `0`\"\n },\n \"pointPadding\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default outer padding for `x` and `y` point-ordinal scales.\\n\\n__Default value:__ `0.5` (which makes _width/height = number of unique values * step_)\",\n \"maximum\": 1,\n \"minimum\": 0\n },\n \"quantileCount\": {\n \"description\": \"Default range cardinality for [`quantile`](https://vega.github.io/vega-lite/docs/scale.html#quantile) scale.\\n\\n__Default value:__ `4`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"quantizeCount\": {\n \"description\": \"Default range cardinality for [`quantize`](https://vega.github.io/vega-lite/docs/scale.html#quantize) scale.\\n\\n__Default value:__ `4`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"rectBandPaddingInner\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default inner padding for `x` and `y` band-ordinal scales of `\\\"rect\\\"` marks.\\n\\n__Default value:__ `0`\",\n \"maximum\": 1,\n \"minimum\": 0\n },\n \"round\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"If true, rounds numeric output values to integers. This can be helpful for snapping to the pixel grid. (Only available for `x`, `y`, and `size` scales.)\"\n },\n \"useUnaggregatedDomain\": {\n \"description\": \"Use the source data range before aggregation as scale domain instead of aggregated data for aggregate axis.\\n\\nThis is equivalent to setting `domain` to `\\\"unaggregate\\\"` for aggregated _quantitative_ fields by default.\\n\\nThis property only works with aggregate functions that produce values within the raw data domain (`\\\"mean\\\"`, `\\\"average\\\"`, `\\\"median\\\"`, `\\\"q1\\\"`, `\\\"q3\\\"`, `\\\"min\\\"`, `\\\"max\\\"`). For other aggregations that produce values outside of the raw data domain (e.g. `\\\"count\\\"`, `\\\"sum\\\"`), this property is ignored.\\n\\n__Default value:__ `false`\",\n \"type\": \"boolean\"\n },\n \"xReverse\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Reverse x-scale by default (useful for right-to-left charts).\"\n },\n \"zero\": {\n \"description\": \"Default `scale.zero` for [`continuous`](https://vega.github.io/vega-lite/docs/scale.html#continuous) scales except for (1) x/y-scales of non-ranged bar or area charts and (2) size scales.\\n\\n__Default value:__ `true`\",\n \"type\": \"boolean\"\n }\n },\n \"type\": \"object\"\n },\n \"ScaleDatumDef\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"ScaleFieldDef\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"ScaleInterpolateEnum\": {\n \"enum\": [\n \"rgb\",\n \"lab\",\n \"hcl\",\n \"hsl\",\n \"hsl-long\",\n \"hcl-long\",\n \"cubehelix\",\n \"cubehelix-long\"\n ],\n \"type\": \"string\"\n },\n \"ScaleInterpolateParams\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"gamma\": {\n \"type\": \"number\"\n },\n \"type\": {\n \"enum\": [\n \"rgb\",\n \"cubehelix\",\n \"cubehelix-long\"\n ],\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"type\"\n ],\n \"type\": \"object\"\n },\n \"ScaleResolveMap\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"angle\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"color\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"fill\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"fillOpacity\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"opacity\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"radius\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"shape\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"size\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"stroke\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"strokeDash\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"strokeOpacity\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"strokeWidth\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"theta\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"x\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"xOffset\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"y\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n },\n \"yOffset\": {\n \"$ref\": \"#/definitions/ResolveMode\"\n }\n },\n \"type\": \"object\"\n },\n \"ScaleType\": {\n \"enum\": [\n \"linear\",\n \"log\",\n \"pow\",\n \"sqrt\",\n \"symlog\",\n \"identity\",\n \"sequential\",\n \"time\",\n \"utc\",\n \"quantile\",\n \"quantize\",\n \"threshold\",\n \"bin-ordinal\",\n \"ordinal\",\n \"point\",\n \"band\"\n ],\n \"type\": \"string\"\n },\n \"SchemeParams\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"count\": {\n \"description\": \"The number of colors to use in the scheme. This can be useful for scale types such as `\\\"quantize\\\"`, which use the length of the scale range to determine the number of discrete bins for the scale domain.\",\n \"type\": \"number\"\n },\n \"extent\": {\n \"description\": \"The extent of the color range to use. For example `[0.2, 1]` will rescale the color scheme such that color values in the range _[0, 0.2)_ are excluded from the scheme.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n \"name\": {\n \"$ref\": \"#/definitions/ColorScheme\",\n \"description\": \"A color scheme name for ordinal scales (e.g., `\\\"category10\\\"` or `\\\"blues\\\"`).\\n\\nFor the full list of supported schemes, please refer to the [Vega Scheme](https://vega.github.io/vega/docs/schemes/#reference) reference.\"\n }\n },\n \"required\": [\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"SecondaryFieldDef\": {\n \"additionalProperties\": false,\n \"description\": \"A field definition of a secondary channel that shares a scale with another primary channel. For example, `x2`, `xError` and `xError2` share the same scale with `x`.\",\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\",\n \"type\": \"null\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n }\n },\n \"type\": \"object\"\n },\n \"SelectionConfig\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"interval\": {\n \"$ref\": \"#/definitions/IntervalSelectionConfigWithoutType\",\n \"description\": \"The default definition for an [`interval`](https://vega.github.io/vega-lite/docs/parameter.html#select) selection. All properties and transformations for an interval selection definition (except `type`) may be specified here.\\n\\nFor instance, setting `interval` to `{\\\"translate\\\": false}` disables the ability to move interval selections by default.\"\n },\n \"point\": {\n \"$ref\": \"#/definitions/PointSelectionConfigWithoutType\",\n \"description\": \"The default definition for a [`point`](https://vega.github.io/vega-lite/docs/parameter.html#select) selection. All properties and transformations for a point selection definition (except `type`) may be specified here.\\n\\nFor instance, setting `point` to `{\\\"on\\\": \\\"dblclick\\\"}` populates point selections on double-click by default.\"\n }\n },\n \"type\": \"object\"\n },\n \"SelectionInit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n }\n ]\n },\n \"SelectionInitInterval\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Vector2\"\n },\n {\n \"$ref\": \"#/definitions/Vector2\"\n },\n {\n \"$ref\": \"#/definitions/Vector2\"\n },\n {\n \"$ref\": \"#/definitions/Vector2\"\n }\n ]\n },\n \"SelectionInitIntervalMapping\": {\n \"$ref\": \"#/definitions/Dict\"\n },\n \"SelectionInitMapping\": {\n \"$ref\": \"#/definitions/Dict\"\n },\n \"SelectionParameter\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"bind\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Binding\"\n },\n {\n \"additionalProperties\": {\n \"$ref\": \"#/definitions/Binding\"\n },\n \"type\": \"object\"\n },\n {\n \"$ref\": \"#/definitions/LegendBinding\"\n },\n {\n \"const\": \"scales\",\n \"type\": \"string\"\n }\n ],\n \"description\": \"When set, a selection is populated by input elements (also known as dynamic query widgets) or by interacting with the corresponding legend. Direct manipulation interaction is disabled by default; to re-enable it, set the selection's [`on`](https://vega.github.io/vega-lite/docs/selection.html#common-selection-properties) property.\\n\\nLegend bindings are restricted to selections that only specify a single field or encoding.\\n\\nQuery widget binding takes the form of Vega's [input element binding definition](https://vega.github.io/vega/docs/signals/#bind) or can be a mapping between projected field/encodings and binding definitions.\\n\\n__See also:__ [`bind`](https://vega.github.io/vega-lite/docs/bind.html) documentation.\"\n },\n \"name\": {\n \"$ref\": \"#/definitions/ParameterName\",\n \"description\": \"Required. A unique name for the selection parameter. Selection names should be valid JavaScript identifiers: they should contain only alphanumeric characters (or \\\"$\\\", or \\\"_\\\") and may not start with a digit. Reserved keywords that may not be used as parameter names are \\\"datum\\\", \\\"event\\\", \\\"item\\\", and \\\"parent\\\".\"\n },\n \"select\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SelectionType\"\n },\n {\n \"$ref\": \"#/definitions/PointSelectionConfig\"\n },\n {\n \"$ref\": \"#/definitions/IntervalSelectionConfig\"\n }\n ],\n \"description\": \"Determines the default event processing and data query for the selection. Vega-Lite currently supports two selection types:\\n\\n- `\\\"point\\\"` -- to select multiple discrete data values; the first value is selected on `click` and additional values toggled on shift-click.\\n- `\\\"interval\\\"` -- to select a continuous range of data values on `drag`.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SelectionInit\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/SelectionInitMapping\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/SelectionInitIntervalMapping\"\n }\n ],\n \"description\": \"Initialize the selection with a mapping between [projected channels or field names](https://vega.github.io/vega-lite/docs/selection.html#project) and initial values.\\n\\n__See also:__ [`init`](https://vega.github.io/vega-lite/docs/value.html) documentation.\"\n }\n },\n \"required\": [\n \"name\",\n \"select\"\n ],\n \"type\": \"object\"\n },\n \"SelectionResolution\": {\n \"enum\": [\n \"global\",\n \"union\",\n \"intersect\"\n ],\n \"type\": \"string\"\n },\n \"SelectionType\": {\n \"enum\": [\n \"point\",\n \"interval\"\n ],\n \"type\": \"string\"\n },\n \"SequenceGenerator\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"name\": {\n \"description\": \"Provide a placeholder name and bind data at runtime.\",\n \"type\": \"string\"\n },\n \"sequence\": {\n \"$ref\": \"#/definitions/SequenceParams\",\n \"description\": \"Generate a sequence of numbers.\"\n }\n },\n \"required\": [\n \"sequence\"\n ],\n \"type\": \"object\"\n },\n \"SequenceParams\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"as\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"The name of the generated sequence field.\\n\\n__Default value:__ `\\\"data\\\"`\"\n },\n \"start\": {\n \"description\": \"The starting value of the sequence (inclusive).\",\n \"type\": \"number\"\n },\n \"step\": {\n \"description\": \"The step value between sequence entries.\\n\\n__Default value:__ `1`\",\n \"type\": \"number\"\n },\n \"stop\": {\n \"description\": \"The ending value of the sequence (exclusive).\",\n \"type\": \"number\"\n }\n },\n \"required\": [\n \"start\",\n \"stop\"\n ],\n \"type\": \"object\"\n },\n \"SequentialMultiHue\": {\n \"enum\": [\n \"turbo\",\n \"viridis\",\n \"inferno\",\n \"magma\",\n \"plasma\",\n \"cividis\",\n \"bluegreen\",\n \"bluegreen-3\",\n \"bluegreen-4\",\n \"bluegreen-5\",\n \"bluegreen-6\",\n \"bluegreen-7\",\n \"bluegreen-8\",\n \"bluegreen-9\",\n \"bluepurple\",\n \"bluepurple-3\",\n \"bluepurple-4\",\n \"bluepurple-5\",\n \"bluepurple-6\",\n \"bluepurple-7\",\n \"bluepurple-8\",\n \"bluepurple-9\",\n \"goldgreen\",\n \"goldgreen-3\",\n \"goldgreen-4\",\n \"goldgreen-5\",\n \"goldgreen-6\",\n \"goldgreen-7\",\n \"goldgreen-8\",\n \"goldgreen-9\",\n \"goldorange\",\n \"goldorange-3\",\n \"goldorange-4\",\n \"goldorange-5\",\n \"goldorange-6\",\n \"goldorange-7\",\n \"goldorange-8\",\n \"goldorange-9\",\n \"goldred\",\n \"goldred-3\",\n \"goldred-4\",\n \"goldred-5\",\n \"goldred-6\",\n \"goldred-7\",\n \"goldred-8\",\n \"goldred-9\",\n \"greenblue\",\n \"greenblue-3\",\n \"greenblue-4\",\n \"greenblue-5\",\n \"greenblue-6\",\n \"greenblue-7\",\n \"greenblue-8\",\n \"greenblue-9\",\n \"orangered\",\n \"orangered-3\",\n \"orangered-4\",\n \"orangered-5\",\n \"orangered-6\",\n \"orangered-7\",\n \"orangered-8\",\n \"orangered-9\",\n \"purplebluegreen\",\n \"purplebluegreen-3\",\n \"purplebluegreen-4\",\n \"purplebluegreen-5\",\n \"purplebluegreen-6\",\n \"purplebluegreen-7\",\n \"purplebluegreen-8\",\n \"purplebluegreen-9\",\n \"purpleblue\",\n \"purpleblue-3\",\n \"purpleblue-4\",\n \"purpleblue-5\",\n \"purpleblue-6\",\n \"purpleblue-7\",\n \"purpleblue-8\",\n \"purpleblue-9\",\n \"purplered\",\n \"purplered-3\",\n \"purplered-4\",\n \"purplered-5\",\n \"purplered-6\",\n \"purplered-7\",\n \"purplered-8\",\n \"purplered-9\",\n \"redpurple\",\n \"redpurple-3\",\n \"redpurple-4\",\n \"redpurple-5\",\n \"redpurple-6\",\n \"redpurple-7\",\n \"redpurple-8\",\n \"redpurple-9\",\n \"yellowgreenblue\",\n \"yellowgreenblue-3\",\n \"yellowgreenblue-4\",\n \"yellowgreenblue-5\",\n \"yellowgreenblue-6\",\n \"yellowgreenblue-7\",\n \"yellowgreenblue-8\",\n \"yellowgreenblue-9\",\n \"yellowgreen\",\n \"yellowgreen-3\",\n \"yellowgreen-4\",\n \"yellowgreen-5\",\n \"yellowgreen-6\",\n \"yellowgreen-7\",\n \"yellowgreen-8\",\n \"yellowgreen-9\",\n \"yelloworangebrown\",\n \"yelloworangebrown-3\",\n \"yelloworangebrown-4\",\n \"yelloworangebrown-5\",\n \"yelloworangebrown-6\",\n \"yelloworangebrown-7\",\n \"yelloworangebrown-8\",\n \"yelloworangebrown-9\",\n \"yelloworangered\",\n \"yelloworangered-3\",\n \"yelloworangered-4\",\n \"yelloworangered-5\",\n \"yelloworangered-6\",\n \"yelloworangered-7\",\n \"yelloworangered-8\",\n \"yelloworangered-9\",\n \"darkblue\",\n \"darkblue-3\",\n \"darkblue-4\",\n \"darkblue-5\",\n \"darkblue-6\",\n \"darkblue-7\",\n \"darkblue-8\",\n \"darkblue-9\",\n \"darkgold\",\n \"darkgold-3\",\n \"darkgold-4\",\n \"darkgold-5\",\n \"darkgold-6\",\n \"darkgold-7\",\n \"darkgold-8\",\n \"darkgold-9\",\n \"darkgreen\",\n \"darkgreen-3\",\n \"darkgreen-4\",\n \"darkgreen-5\",\n \"darkgreen-6\",\n \"darkgreen-7\",\n \"darkgreen-8\",\n \"darkgreen-9\",\n \"darkmulti\",\n \"darkmulti-3\",\n \"darkmulti-4\",\n \"darkmulti-5\",\n \"darkmulti-6\",\n \"darkmulti-7\",\n \"darkmulti-8\",\n \"darkmulti-9\",\n \"darkred\",\n \"darkred-3\",\n \"darkred-4\",\n \"darkred-5\",\n \"darkred-6\",\n \"darkred-7\",\n \"darkred-8\",\n \"darkred-9\",\n \"lightgreyred\",\n \"lightgreyred-3\",\n \"lightgreyred-4\",\n \"lightgreyred-5\",\n \"lightgreyred-6\",\n \"lightgreyred-7\",\n \"lightgreyred-8\",\n \"lightgreyred-9\",\n \"lightgreyteal\",\n \"lightgreyteal-3\",\n \"lightgreyteal-4\",\n \"lightgreyteal-5\",\n \"lightgreyteal-6\",\n \"lightgreyteal-7\",\n \"lightgreyteal-8\",\n \"lightgreyteal-9\",\n \"lightmulti\",\n \"lightmulti-3\",\n \"lightmulti-4\",\n \"lightmulti-5\",\n \"lightmulti-6\",\n \"lightmulti-7\",\n \"lightmulti-8\",\n \"lightmulti-9\",\n \"lightorange\",\n \"lightorange-3\",\n \"lightorange-4\",\n \"lightorange-5\",\n \"lightorange-6\",\n \"lightorange-7\",\n \"lightorange-8\",\n \"lightorange-9\",\n \"lighttealblue\",\n \"lighttealblue-3\",\n \"lighttealblue-4\",\n \"lighttealblue-5\",\n \"lighttealblue-6\",\n \"lighttealblue-7\",\n \"lighttealblue-8\",\n \"lighttealblue-9\"\n ],\n \"type\": \"string\"\n },\n \"SequentialSingleHue\": {\n \"enum\": [\n \"blues\",\n \"tealblues\",\n \"teals\",\n \"greens\",\n \"browns\",\n \"greys\",\n \"purples\",\n \"warmgreys\",\n \"reds\",\n \"oranges\"\n ],\n \"type\": \"string\"\n },\n \"ShapeDef\": {\n \"$ref\": \"#/definitions/MarkPropDef<(string|null),TypeForShape>\"\n },\n \"SharedEncoding\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"angle\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalMarkPropFieldOrDatumDef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A field definition or one or more value definition(s) with a parameter predicate.\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"legend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Legend\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the legend. If `null`, the legend for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [legend properties](https://vega.github.io/vega-lite/docs/legend.html) are applied.\\n\\n__See also:__ [`legend`](https://vega.github.io/vega-lite/docs/legend.html) documentation.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n ],\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"color\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Gradient|string|null|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Gradient|string|null|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalMarkPropFieldOrDatumDef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Gradient|string|null|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Gradient|string|null|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A field definition or one or more value definition(s) with a parameter predicate.\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"legend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Legend\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the legend. If `null`, the legend for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [legend properties](https://vega.github.io/vega-lite/docs/legend.html) are applied.\\n\\n__See also:__ [`legend`](https://vega.github.io/vega-lite/docs/legend.html) documentation.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n ],\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"description\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"const\": \"binned\",\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalMarkPropFieldOrDatumDef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|null|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|null|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A field definition or one or more value definition(s) with a parameter predicate.\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"format\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Dict\"\n }\n ],\n \"description\": \"When used with the default `\\\"number\\\"` and `\\\"time\\\"` format type, the text formatting pattern for labels of guides (axes, legends, headers) and text marks.\\n\\n- If the format type is `\\\"number\\\"` (e.g., for quantitative fields), this is D3's [number format pattern](https://github.com/d3/d3-format#locale_format).\\n- If the format type is `\\\"time\\\"` (e.g., for temporal fields), this is D3's [time format pattern](https://github.com/d3/d3-time-format#locale_format).\\n\\nSee the [format documentation](https://vega.github.io/vega-lite/docs/format.html) for more examples.\\n\\nWhen used with a [custom `formatType`](https://vega.github.io/vega-lite/docs/config.html#custom-format-type), this value will be passed as `format` alongside `datum.value` to the registered function.\\n\\n__Default value:__ Derived from [numberFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for number format and from [timeFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for time format.\"\n },\n \"formatType\": {\n \"description\": \"The format type for labels. One of `\\\"number\\\"`, `\\\"time\\\"`, or a [registered custom format type](https://vega.github.io/vega-lite/docs/config.html#custom-format-type).\\n\\n__Default value:__\\n- `\\\"time\\\"` for temporal fields and ordinal and nominal fields with `timeUnit`.\\n- `\\\"number\\\"` for quantitative fields as well as ordinal and nominal fields without `timeUnit`.\",\n \"type\": \"string\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"detail\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FieldDefWithoutScale\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/FieldDefWithoutScale\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"Additional levels of detail for grouping data in aggregate views and in line, trail, and area marks without mapping data to a specific visual channel.\"\n },\n \"fill\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Gradient|string|null|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Gradient|string|null|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalMarkPropFieldOrDatumDef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Gradient|string|null|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Gradient|string|null|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A field definition or one or more value definition(s) with a parameter predicate.\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"legend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Legend\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the legend. If `null`, the legend for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [legend properties](https://vega.github.io/vega-lite/docs/legend.html) are applied.\\n\\n__See also:__ [`legend`](https://vega.github.io/vega-lite/docs/legend.html) documentation.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n ],\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"fillOpacity\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalMarkPropFieldOrDatumDef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A field definition or one or more value definition(s) with a parameter predicate.\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"legend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Legend\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the legend. If `null`, the legend for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [legend properties](https://vega.github.io/vega-lite/docs/legend.html) are applied.\\n\\n__See also:__ [`legend`](https://vega.github.io/vega-lite/docs/legend.html) documentation.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n ],\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"href\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"const\": \"binned\",\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalMarkPropFieldOrDatumDef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|null|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|null|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A field definition or one or more value definition(s) with a parameter predicate.\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"format\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Dict\"\n }\n ],\n \"description\": \"When used with the default `\\\"number\\\"` and `\\\"time\\\"` format type, the text formatting pattern for labels of guides (axes, legends, headers) and text marks.\\n\\n- If the format type is `\\\"number\\\"` (e.g., for quantitative fields), this is D3's [number format pattern](https://github.com/d3/d3-format#locale_format).\\n- If the format type is `\\\"time\\\"` (e.g., for temporal fields), this is D3's [time format pattern](https://github.com/d3/d3-time-format#locale_format).\\n\\nSee the [format documentation](https://vega.github.io/vega-lite/docs/format.html) for more examples.\\n\\nWhen used with a [custom `formatType`](https://vega.github.io/vega-lite/docs/config.html#custom-format-type), this value will be passed as `format` alongside `datum.value` to the registered function.\\n\\n__Default value:__ Derived from [numberFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for number format and from [timeFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for time format.\"\n },\n \"formatType\": {\n \"description\": \"The format type for labels. One of `\\\"number\\\"`, `\\\"time\\\"`, or a [registered custom format type](https://vega.github.io/vega-lite/docs/config.html#custom-format-type).\\n\\n__Default value:__\\n- `\\\"time\\\"` for temporal fields and ordinal and nominal fields with `timeUnit`.\\n- `\\\"number\\\"` for quantitative fields as well as ordinal and nominal fields without `timeUnit`.\",\n \"type\": \"string\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"key\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"const\": \"binned\",\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"latitude\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\",\n \"type\": \"null\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"anyOf\": [\n {\n \"const\": \"quantitative\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n ],\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"latitude2\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\",\n \"type\": \"null\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"longitude\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\",\n \"type\": \"null\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"anyOf\": [\n {\n \"const\": \"quantitative\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n ],\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"longitude2\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\",\n \"type\": \"null\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"opacity\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalMarkPropFieldOrDatumDef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A field definition or one or more value definition(s) with a parameter predicate.\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"legend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Legend\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the legend. If `null`, the legend for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [legend properties](https://vega.github.io/vega-lite/docs/legend.html) are applied.\\n\\n__See also:__ [`legend`](https://vega.github.io/vega-lite/docs/legend.html) documentation.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n ],\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"order\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/OrderFieldDef\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/OrderFieldDef\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/OrderValueDef\"\n },\n {\n \"$ref\": \"#/definitions/OrderOnlyDef\"\n }\n ],\n \"description\": \"Order of the marks.\\n- For stacked marks, this `order` channel encodes [stack order](https://vega.github.io/vega-lite/docs/stack.html#order).\\n- For line and trail marks, this `order` channel encodes order of data points in the lines. This can be useful for creating [a connected scatterplot](https://vega.github.io/vega-lite/examples/connected_scatterplot.html). Setting `order` to `{\\\"value\\\": null}` makes the line marks use the original order in the data sources.\\n- Otherwise, this `order` channel encodes layer order of the marks.\\n\\n__Note__: In aggregate plots, `order` field should be `aggregate`d to avoid creating additional aggregation grouping.\"\n },\n \"radius\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"const\": \"binned\",\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"stack\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StackOffset\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"type\": \"boolean\"\n }\n ],\n \"description\": \"Type of stacking offset if the field should be stacked. `stack` is only applicable for `x`, `y`, `theta`, and `radius` channels with continuous domains. For example, `stack` of `y` can be used to customize stacking for a vertical bar chart.\\n\\n`stack` can be one of the following values:\\n- `\\\"zero\\\"` or `true`: stacking with baseline offset at zero value of the scale (for creating typical stacked [bar](https://vega.github.io/vega-lite/docs/stack.html#bar) and [area](https://vega.github.io/vega-lite/docs/stack.html#area) chart).\\n- `\\\"normalize\\\"` - stacking with normalized domain (for creating [normalized stacked bar and area charts](https://vega.github.io/vega-lite/docs/stack.html#normalized) and pie charts [with percentage tooltip](https://vega.github.io/vega-lite/docs/arc.html#tooltip)).
\\n-`\\\"center\\\"` - stacking with center baseline (for [streamgraph](https://vega.github.io/vega-lite/docs/stack.html#streamgraph)).\\n- `null` or `false` - No-stacking. This will produce layered [bar](https://vega.github.io/vega-lite/docs/stack.html#layered-bar-chart) and area chart.\\n\\n__Default value:__ `zero` for plots with all of the following conditions are true: (1) the mark is `bar`, `area`, or `arc`; (2) the stacked measure channel (x or y) has a linear scale; (3) At least one of non-position channels mapped to an unaggregated field that is different from x and y. Otherwise, `null` by default.\\n\\n__See also:__ [`stack`](https://vega.github.io/vega-lite/docs/stack.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n ],\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"radius2\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\",\n \"type\": \"null\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"shape\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|null|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|null|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalMarkPropFieldOrDatumDef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|null|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|null|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A field definition or one or more value definition(s) with a parameter predicate.\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"legend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Legend\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the legend. If `null`, the legend for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [legend properties](https://vega.github.io/vega-lite/docs/legend.html) are applied.\\n\\n__See also:__ [`legend`](https://vega.github.io/vega-lite/docs/legend.html) documentation.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TypeForShape\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n ],\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"size\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalMarkPropFieldOrDatumDef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A field definition or one or more value definition(s) with a parameter predicate.\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"legend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Legend\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the legend. If `null`, the legend for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [legend properties](https://vega.github.io/vega-lite/docs/legend.html) are applied.\\n\\n__See also:__ [`legend`](https://vega.github.io/vega-lite/docs/legend.html) documentation.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n ],\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"stroke\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Gradient|string|null|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Gradient|string|null|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalMarkPropFieldOrDatumDef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Gradient|string|null|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Gradient|string|null|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A field definition or one or more value definition(s) with a parameter predicate.\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"legend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Legend\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the legend. If `null`, the legend for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [legend properties](https://vega.github.io/vega-lite/docs/legend.html) are applied.\\n\\n__See also:__ [`legend`](https://vega.github.io/vega-lite/docs/legend.html) documentation.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n ],\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"strokeDash\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number[]|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number[]|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalMarkPropFieldOrDatumDef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number[]|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number[]|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A field definition or one or more value definition(s) with a parameter predicate.\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"legend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Legend\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the legend. If `null`, the legend for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [legend properties](https://vega.github.io/vega-lite/docs/legend.html) are applied.\\n\\n__See also:__ [`legend`](https://vega.github.io/vega-lite/docs/legend.html) documentation.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n ],\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"strokeOpacity\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalMarkPropFieldOrDatumDef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A field definition or one or more value definition(s) with a parameter predicate.\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"legend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Legend\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the legend. If `null`, the legend for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [legend properties](https://vega.github.io/vega-lite/docs/legend.html) are applied.\\n\\n__See also:__ [`legend`](https://vega.github.io/vega-lite/docs/legend.html) documentation.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n ],\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"strokeWidth\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalMarkPropFieldOrDatumDef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A field definition or one or more value definition(s) with a parameter predicate.\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"legend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Legend\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the legend. If `null`, the legend for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [legend properties](https://vega.github.io/vega-lite/docs/legend.html) are applied.\\n\\n__See also:__ [`legend`](https://vega.github.io/vega-lite/docs/legend.html) documentation.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n ],\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"text\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"const\": \"binned\",\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Text|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Text|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalStringFieldDef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Text|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Text|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A field definition or one or more value definition(s) with a parameter predicate.\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"format\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Dict\"\n }\n ],\n \"description\": \"When used with the default `\\\"number\\\"` and `\\\"time\\\"` format type, the text formatting pattern for labels of guides (axes, legends, headers) and text marks.\\n\\n- If the format type is `\\\"number\\\"` (e.g., for quantitative fields), this is D3's [number format pattern](https://github.com/d3/d3-format#locale_format).\\n- If the format type is `\\\"time\\\"` (e.g., for temporal fields), this is D3's [time format pattern](https://github.com/d3/d3-time-format#locale_format).\\n\\nSee the [format documentation](https://vega.github.io/vega-lite/docs/format.html) for more examples.\\n\\nWhen used with a [custom `formatType`](https://vega.github.io/vega-lite/docs/config.html#custom-format-type), this value will be passed as `format` alongside `datum.value` to the registered function.\\n\\n__Default value:__ Derived from [numberFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for number format and from [timeFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for time format.\"\n },\n \"formatType\": {\n \"description\": \"The format type for labels. One of `\\\"number\\\"`, `\\\"time\\\"`, or a [registered custom format type](https://vega.github.io/vega-lite/docs/config.html#custom-format-type).\\n\\n__Default value:__\\n- `\\\"time\\\"` for temporal fields and ordinal and nominal fields with `timeUnit`.\\n- `\\\"number\\\"` for quantitative fields as well as ordinal and nominal fields without `timeUnit`.\",\n \"type\": \"string\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n ],\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"theta\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"const\": \"binned\",\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"stack\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StackOffset\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"type\": \"boolean\"\n }\n ],\n \"description\": \"Type of stacking offset if the field should be stacked. `stack` is only applicable for `x`, `y`, `theta`, and `radius` channels with continuous domains. For example, `stack` of `y` can be used to customize stacking for a vertical bar chart.\\n\\n`stack` can be one of the following values:\\n- `\\\"zero\\\"` or `true`: stacking with baseline offset at zero value of the scale (for creating typical stacked [bar](https://vega.github.io/vega-lite/docs/stack.html#bar) and [area](https://vega.github.io/vega-lite/docs/stack.html#area) chart).\\n- `\\\"normalize\\\"` - stacking with normalized domain (for creating [normalized stacked bar and area charts](https://vega.github.io/vega-lite/docs/stack.html#normalized) and pie charts [with percentage tooltip](https://vega.github.io/vega-lite/docs/arc.html#tooltip)).
\\n-`\\\"center\\\"` - stacking with center baseline (for [streamgraph](https://vega.github.io/vega-lite/docs/stack.html#streamgraph)).\\n- `null` or `false` - No-stacking. This will produce layered [bar](https://vega.github.io/vega-lite/docs/stack.html#layered-bar-chart) and area chart.\\n\\n__Default value:__ `zero` for plots with all of the following conditions are true: (1) the mark is `bar`, `area`, or `arc`; (2) the stacked measure channel (x or y) has a linear scale; (3) At least one of non-position channels mapped to an unaggregated field that is different from x and y. Otherwise, `null` by default.\\n\\n__See also:__ [`stack`](https://vega.github.io/vega-lite/docs/stack.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n ],\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"theta2\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\",\n \"type\": \"null\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"tooltip\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StringFieldDefWithCondition\"\n },\n {\n \"$ref\": \"#/definitions/StringValueDefWithCondition\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/StringFieldDef\"\n },\n \"type\": \"array\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"The tooltip text to show upon mouse hover. Specifying `tooltip` encoding overrides [the `tooltip` property in the mark definition](https://vega.github.io/vega-lite/docs/mark.html#mark-def).\\n\\nSee the [`tooltip`](https://vega.github.io/vega-lite/docs/tooltip.html) documentation for a detailed discussion about tooltip in Vega-Lite.\"\n },\n \"url\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"const\": \"binned\",\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"condition\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalMarkPropFieldOrDatumDef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|null|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|null|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A field definition or one or more value definition(s) with a parameter predicate.\"\n }\n ],\n \"description\": \"One or more value definition(s) with [a parameter or a test predicate](https://vega.github.io/vega-lite/docs/condition.html).\\n\\n__Note:__ A field definition's `condition` property can only contain [conditional value definitions](https://vega.github.io/vega-lite/docs/condition.html#value) since Vega-Lite only allows at most one encoded field per encoding channel.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"format\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Dict\"\n }\n ],\n \"description\": \"When used with the default `\\\"number\\\"` and `\\\"time\\\"` format type, the text formatting pattern for labels of guides (axes, legends, headers) and text marks.\\n\\n- If the format type is `\\\"number\\\"` (e.g., for quantitative fields), this is D3's [number format pattern](https://github.com/d3/d3-format#locale_format).\\n- If the format type is `\\\"time\\\"` (e.g., for temporal fields), this is D3's [time format pattern](https://github.com/d3/d3-time-format#locale_format).\\n\\nSee the [format documentation](https://vega.github.io/vega-lite/docs/format.html) for more examples.\\n\\nWhen used with a [custom `formatType`](https://vega.github.io/vega-lite/docs/config.html#custom-format-type), this value will be passed as `format` alongside `datum.value` to the registered function.\\n\\n__Default value:__ Derived from [numberFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for number format and from [timeFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for time format.\"\n },\n \"formatType\": {\n \"description\": \"The format type for labels. One of `\\\"number\\\"`, `\\\"time\\\"`, or a [registered custom format type](https://vega.github.io/vega-lite/docs/config.html#custom-format-type).\\n\\n__Default value:__\\n- `\\\"time\\\"` for temporal fields and ordinal and nominal fields with `timeUnit`.\\n- `\\\"number\\\"` for quantitative fields as well as ordinal and nominal fields without `timeUnit`.\",\n \"type\": \"string\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"x\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"axis\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Axis\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of axis's gridlines, ticks and labels. If `null`, the axis for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [axis properties](https://vega.github.io/vega-lite/docs/axis.html) are applied.\\n\\n__See also:__ [`axis`](https://vega.github.io/vega-lite/docs/axis.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"const\": \"binned\",\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"impute\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ImputeParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining the properties of the Impute Operation to be applied. The field value of the other positional channel is taken as `key` of the `Impute` Operation. The field of the `color` channel if specified is used as `groupby` of the `Impute` Operation.\\n\\n__See also:__ [`impute`](https://vega.github.io/vega-lite/docs/impute.html) documentation.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"stack\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StackOffset\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"type\": \"boolean\"\n }\n ],\n \"description\": \"Type of stacking offset if the field should be stacked. `stack` is only applicable for `x`, `y`, `theta`, and `radius` channels with continuous domains. For example, `stack` of `y` can be used to customize stacking for a vertical bar chart.\\n\\n`stack` can be one of the following values:\\n- `\\\"zero\\\"` or `true`: stacking with baseline offset at zero value of the scale (for creating typical stacked [bar](https://vega.github.io/vega-lite/docs/stack.html#bar) and [area](https://vega.github.io/vega-lite/docs/stack.html#area) chart).\\n- `\\\"normalize\\\"` - stacking with normalized domain (for creating [normalized stacked bar and area charts](https://vega.github.io/vega-lite/docs/stack.html#normalized) and pie charts [with percentage tooltip](https://vega.github.io/vega-lite/docs/arc.html#tooltip)).
\\n-`\\\"center\\\"` - stacking with center baseline (for [streamgraph](https://vega.github.io/vega-lite/docs/stack.html#streamgraph)).\\n- `null` or `false` - No-stacking. This will produce layered [bar](https://vega.github.io/vega-lite/docs/stack.html#layered-bar-chart) and area chart.\\n\\n__Default value:__ `zero` for plots with all of the following conditions are true: (1) the mark is `bar`, `area`, or `arc`; (2) the stacked measure channel (x or y) has a linear scale; (3) At least one of non-position channels mapped to an unaggregated field that is different from x and y. Otherwise, `null` by default.\\n\\n__See also:__ [`stack`](https://vega.github.io/vega-lite/docs/stack.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n ],\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"x2\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\",\n \"type\": \"null\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"xError\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\",\n \"type\": \"null\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"value\": {\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\",\n \"type\": \"number\"\n }\n },\n \"type\": \"object\"\n },\n \"xError2\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\",\n \"type\": \"null\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"value\": {\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\",\n \"type\": \"number\"\n }\n },\n \"type\": \"object\"\n },\n \"xOffset\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n ],\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\",\n \"type\": \"number\"\n }\n },\n \"type\": \"object\"\n },\n \"y\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"axis\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Axis\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of axis's gridlines, ticks and labels. If `null`, the axis for the encoding channel will be removed.\\n\\n__Default value:__ If undefined, default [axis properties](https://vega.github.io/vega-lite/docs/axis.html) are applied.\\n\\n__See also:__ [`axis`](https://vega.github.io/vega-lite/docs/axis.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"const\": \"binned\",\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"impute\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ImputeParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining the properties of the Impute Operation to be applied. The field value of the other positional channel is taken as `key` of the `Impute` Operation. The field of the `color` channel if specified is used as `groupby` of the `Impute` Operation.\\n\\n__See also:__ [`impute`](https://vega.github.io/vega-lite/docs/impute.html) documentation.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"stack\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StackOffset\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"type\": \"boolean\"\n }\n ],\n \"description\": \"Type of stacking offset if the field should be stacked. `stack` is only applicable for `x`, `y`, `theta`, and `radius` channels with continuous domains. For example, `stack` of `y` can be used to customize stacking for a vertical bar chart.\\n\\n`stack` can be one of the following values:\\n- `\\\"zero\\\"` or `true`: stacking with baseline offset at zero value of the scale (for creating typical stacked [bar](https://vega.github.io/vega-lite/docs/stack.html#bar) and [area](https://vega.github.io/vega-lite/docs/stack.html#area) chart).\\n- `\\\"normalize\\\"` - stacking with normalized domain (for creating [normalized stacked bar and area charts](https://vega.github.io/vega-lite/docs/stack.html#normalized) and pie charts [with percentage tooltip](https://vega.github.io/vega-lite/docs/arc.html#tooltip)).
\\n-`\\\"center\\\"` - stacking with center baseline (for [streamgraph](https://vega.github.io/vega-lite/docs/stack.html#streamgraph)).\\n- `null` or `false` - No-stacking. This will produce layered [bar](https://vega.github.io/vega-lite/docs/stack.html#layered-bar-chart) and area chart.\\n\\n__Default value:__ `zero` for plots with all of the following conditions are true: (1) the mark is `bar`, `area`, or `arc`; (2) the stacked measure channel (x or y) has a linear scale; (3) At least one of non-position channels mapped to an unaggregated field that is different from x and y. Otherwise, `null` by default.\\n\\n__See also:__ [`stack`](https://vega.github.io/vega-lite/docs/stack.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n ],\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"y2\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\",\n \"type\": \"null\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"yError\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\",\n \"type\": \"null\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"value\": {\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\",\n \"type\": \"number\"\n }\n },\n \"type\": \"object\"\n },\n \"yError2\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\",\n \"type\": \"null\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"value\": {\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\",\n \"type\": \"number\"\n }\n },\n \"type\": \"object\"\n },\n \"yOffset\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"datum\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/PrimitiveValue\"\n },\n {\n \"$ref\": \"#/definitions/DateTime\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"$ref\": \"#/definitions/RepeatRef\"\n }\n ],\n \"description\": \"A constant value in data domain.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"scale\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Scale\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object defining properties of the channel's scale, which is the function that transforms values in the data domain (numbers, dates, strings, etc) to visual values (pixels, colors, sizes) of the encoding channels.\\n\\nIf `null`, the scale will be [disabled and the data value will be directly encoded](https://vega.github.io/vega-lite/docs/scale.html#disable).\\n\\n__Default value:__ If undefined, default [scale properties](https://vega.github.io/vega-lite/docs/scale.html) are applied.\\n\\n__See also:__ [`scale`](https://vega.github.io/vega-lite/docs/scale.html) documentation.\"\n },\n \"sort\": {\n \"$ref\": \"#/definitions/Sort\",\n \"description\": \"Sort order for the encoded field.\\n\\nFor continuous fields (quantitative or temporal), `sort` can be either `\\\"ascending\\\"` or `\\\"descending\\\"`.\\n\\nFor discrete fields, `sort` can be one of the following:\\n- `\\\"ascending\\\"` or `\\\"descending\\\"` -- for sorting by the values' natural order in JavaScript.\\n- [A string indicating an encoding channel name to sort by](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding) (e.g., `\\\"x\\\"` or `\\\"y\\\"`) with an optional minus prefix for descending sort (e.g., `\\\"-x\\\"` to sort by x-field, descending). This channel string is short-form of [a sort-by-encoding definition](https://vega.github.io/vega-lite/docs/sort.html#sort-by-encoding). For example, `\\\"sort\\\": \\\"-x\\\"` is equivalent to `\\\"sort\\\": {\\\"encoding\\\": \\\"x\\\", \\\"order\\\": \\\"descending\\\"}`.\\n- [A sort field definition](https://vega.github.io/vega-lite/docs/sort.html#sort-field) for sorting by another field.\\n- [An array specifying the field values in preferred order](https://vega.github.io/vega-lite/docs/sort.html#sort-array). In this case, the sort order will obey the values in the array, followed by any unspecified values in their original order. For discrete time field, values in the sort array can be [date-time definition objects](types#datetime). In addition, for time units `\\\"month\\\"` and `\\\"day\\\"`, the values can be the month or day names (case insensitive) or their 3-letter initials (e.g., `\\\"Mon\\\"`, `\\\"Tue\\\"`).\\n- `null` indicating no sort.\\n\\n__Default value:__ `\\\"ascending\\\"`\\n\\n__Note:__ `null` and sorting by another channel is not supported for `row` and `column`.\\n\\n__See also:__ [`sort`](https://vega.github.io/vega-lite/docs/sort.html) documentation.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n {\n \"$ref\": \"#/definitions/Type\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n ],\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n },\n \"value\": {\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\",\n \"type\": \"number\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"SingleDefUnitChannel\": {\n \"enum\": [\n \"x\",\n \"y\",\n \"xOffset\",\n \"yOffset\",\n \"x2\",\n \"y2\",\n \"longitude\",\n \"latitude\",\n \"longitude2\",\n \"latitude2\",\n \"theta\",\n \"theta2\",\n \"radius\",\n \"radius2\",\n \"color\",\n \"fill\",\n \"stroke\",\n \"opacity\",\n \"fillOpacity\",\n \"strokeOpacity\",\n \"strokeWidth\",\n \"strokeDash\",\n \"size\",\n \"angle\",\n \"shape\",\n \"key\",\n \"text\",\n \"href\",\n \"url\",\n \"description\"\n ],\n \"type\": \"string\"\n },\n \"SingleTimeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LocalSingleTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/UtcSingleTimeUnit\"\n }\n ]\n },\n \"Sort\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SortArray\"\n },\n {\n \"$ref\": \"#/definitions/AllSortString\"\n },\n {\n \"$ref\": \"#/definitions/EncodingSortField\"\n },\n {\n \"$ref\": \"#/definitions/SortByEncoding\"\n },\n {\n \"type\": \"null\"\n }\n ]\n },\n \"SortArray\": {\n \"anyOf\": [\n {\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n {\n \"items\": {\n \"type\": \"boolean\"\n },\n \"type\": \"array\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/DateTime\"\n },\n \"type\": \"array\"\n }\n ]\n },\n \"SortByChannel\": {\n \"enum\": [\n \"x\",\n \"y\",\n \"color\",\n \"fill\",\n \"stroke\",\n \"strokeWidth\",\n \"size\",\n \"shape\",\n \"fillOpacity\",\n \"strokeOpacity\",\n \"opacity\",\n \"text\"\n ],\n \"type\": \"string\"\n },\n \"SortByChannelDesc\": {\n \"enum\": [\n \"-x\",\n \"-y\",\n \"-color\",\n \"-fill\",\n \"-stroke\",\n \"-strokeWidth\",\n \"-size\",\n \"-shape\",\n \"-fillOpacity\",\n \"-strokeOpacity\",\n \"-opacity\",\n \"-text\"\n ],\n \"type\": \"string\"\n },\n \"SortByEncoding\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"encoding\": {\n \"$ref\": \"#/definitions/SortByChannel\",\n \"description\": \"The [encoding channel](https://vega.github.io/vega-lite/docs/encoding.html#channels) to sort by (e.g., `\\\"x\\\"`, `\\\"y\\\"`)\"\n },\n \"order\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SortOrder\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"The sort order. One of `\\\"ascending\\\"` (default), `\\\"descending\\\"`, or `null` (no not sort).\"\n }\n },\n \"required\": [\n \"encoding\"\n ],\n \"type\": \"object\"\n },\n \"SortField\": {\n \"additionalProperties\": false,\n \"description\": \"A sort definition for transform\",\n \"properties\": {\n \"field\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"The name of the field to sort.\"\n },\n \"order\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SortOrder\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"Whether to sort the field in ascending or descending order. One of `\\\"ascending\\\"` (default), `\\\"descending\\\"`, or `null` (no not sort).\"\n }\n },\n \"required\": [\n \"field\"\n ],\n \"type\": \"object\"\n },\n \"SortOrder\": {\n \"enum\": [\n \"ascending\",\n \"descending\"\n ],\n \"type\": \"string\"\n },\n \"SphereGenerator\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"name\": {\n \"description\": \"Provide a placeholder name and bind data at runtime.\",\n \"type\": \"string\"\n },\n \"sphere\": {\n \"anyOf\": [\n {\n \"const\": true,\n \"type\": \"boolean\"\n },\n {\n \"additionalProperties\": false,\n \"type\": \"object\"\n }\n ],\n \"description\": \"Generate sphere GeoJSON data for the full globe.\"\n }\n },\n \"required\": [\n \"sphere\"\n ],\n \"type\": \"object\"\n },\n \"StackOffset\": {\n \"enum\": [\n \"zero\",\n \"center\",\n \"normalize\"\n ],\n \"type\": \"string\"\n },\n \"StackTransform\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"as\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FieldName\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"maxItems\": 2,\n \"minItems\": 2,\n \"type\": \"array\"\n }\n ],\n \"description\": \"Output field names. This can be either a string or an array of strings with two elements denoting the name for the fields for stack start and stack end respectively. If a single string(e.g., `\\\"val\\\"`) is provided, the end field will be `\\\"val_end\\\"`.\"\n },\n \"groupby\": {\n \"description\": \"The data fields to group by.\",\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"type\": \"array\"\n },\n \"offset\": {\n \"description\": \"Mode for stacking marks. One of `\\\"zero\\\"` (default), `\\\"center\\\"`, or `\\\"normalize\\\"`. The `\\\"zero\\\"` offset will stack starting at `0`. The `\\\"center\\\"` offset will center the stacks. The `\\\"normalize\\\"` offset will compute percentage values for each stack point, with output values in the range `[0,1]`.\\n\\n__Default value:__ `\\\"zero\\\"`\",\n \"enum\": [\n \"zero\",\n \"center\",\n \"normalize\"\n ],\n \"type\": \"string\"\n },\n \"sort\": {\n \"description\": \"Field that determines the order of leaves in the stacked charts.\",\n \"items\": {\n \"$ref\": \"#/definitions/SortField\"\n },\n \"type\": \"array\"\n },\n \"stack\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"The field which is stacked.\"\n }\n },\n \"required\": [\n \"stack\",\n \"groupby\",\n \"as\"\n ],\n \"type\": \"object\"\n },\n \"StandardType\": {\n \"enum\": [\n \"quantitative\",\n \"ordinal\",\n \"temporal\",\n \"nominal\"\n ],\n \"type\": \"string\"\n },\n \"Step\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"for\": {\n \"$ref\": \"#/definitions/StepFor\",\n \"description\": \"Whether to apply the step to position scale or offset scale when there are both `x` and `xOffset` or both `y` and `yOffset` encodings.\"\n },\n \"step\": {\n \"description\": \"The size (width/height) per discrete step.\",\n \"type\": \"number\"\n }\n },\n \"required\": [\n \"step\"\n ],\n \"type\": \"object\"\n },\n \"StepFor\": {\n \"enum\": [\n \"position\",\n \"offset\"\n ],\n \"type\": \"string\"\n },\n \"Stream\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/EventStream\"\n },\n {\n \"$ref\": \"#/definitions/DerivedStream\"\n },\n {\n \"$ref\": \"#/definitions/MergedStream\"\n }\n ]\n },\n \"StringFieldDef\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"const\": \"binned\",\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"format\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Dict\"\n }\n ],\n \"description\": \"When used with the default `\\\"number\\\"` and `\\\"time\\\"` format type, the text formatting pattern for labels of guides (axes, legends, headers) and text marks.\\n\\n- If the format type is `\\\"number\\\"` (e.g., for quantitative fields), this is D3's [number format pattern](https://github.com/d3/d3-format#locale_format).\\n- If the format type is `\\\"time\\\"` (e.g., for temporal fields), this is D3's [time format pattern](https://github.com/d3/d3-time-format#locale_format).\\n\\nSee the [format documentation](https://vega.github.io/vega-lite/docs/format.html) for more examples.\\n\\nWhen used with a [custom `formatType`](https://vega.github.io/vega-lite/docs/config.html#custom-format-type), this value will be passed as `format` alongside `datum.value` to the registered function.\\n\\n__Default value:__ Derived from [numberFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for number format and from [timeFormat](https://vega.github.io/vega-lite/docs/config.html#format) config for time format.\"\n },\n \"formatType\": {\n \"description\": \"The format type for labels. One of `\\\"number\\\"`, `\\\"time\\\"`, or a [registered custom format type](https://vega.github.io/vega-lite/docs/config.html#custom-format-type).\\n\\n__Default value:__\\n- `\\\"time\\\"` for temporal fields and ordinal and nominal fields with `timeUnit`.\\n- `\\\"number\\\"` for quantitative fields as well as ordinal and nominal fields without `timeUnit`.\",\n \"type\": \"string\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"StringFieldDefWithCondition\": {\n \"$ref\": \"#/definitions/FieldOrDatumDefWithCondition\"\n },\n \"StringValueDefWithCondition\": {\n \"$ref\": \"#/definitions/ValueDefWithCondition\"\n },\n \"StrokeCap\": {\n \"enum\": [\n \"butt\",\n \"round\",\n \"square\"\n ],\n \"type\": \"string\"\n },\n \"StrokeJoin\": {\n \"enum\": [\n \"miter\",\n \"round\",\n \"bevel\"\n ],\n \"type\": \"string\"\n },\n \"StyleConfigIndex\": {\n \"additionalProperties\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/AnyMarkConfig\"\n },\n {\n \"$ref\": \"#/definitions/Axis\"\n }\n ]\n },\n \"properties\": {\n \"arc\": {\n \"$ref\": \"#/definitions/RectConfig\",\n \"description\": \"Arc-specific Config\"\n },\n \"area\": {\n \"$ref\": \"#/definitions/AreaConfig\",\n \"description\": \"Area-Specific Config\"\n },\n \"bar\": {\n \"$ref\": \"#/definitions/BarConfig\",\n \"description\": \"Bar-Specific Config\"\n },\n \"circle\": {\n \"$ref\": \"#/definitions/MarkConfig\",\n \"description\": \"Circle-Specific Config\"\n },\n \"geoshape\": {\n \"$ref\": \"#/definitions/MarkConfig\",\n \"description\": \"Geoshape-Specific Config\"\n },\n \"group-subtitle\": {\n \"$ref\": \"#/definitions/MarkConfig\",\n \"description\": \"Default style for chart subtitles\"\n },\n \"group-title\": {\n \"$ref\": \"#/definitions/MarkConfig\",\n \"description\": \"Default style for chart titles\"\n },\n \"guide-label\": {\n \"$ref\": \"#/definitions/MarkConfig\",\n \"description\": \"Default style for axis, legend, and header labels.\"\n },\n \"guide-title\": {\n \"$ref\": \"#/definitions/MarkConfig\",\n \"description\": \"Default style for axis, legend, and header titles.\"\n },\n \"image\": {\n \"$ref\": \"#/definitions/RectConfig\",\n \"description\": \"Image-specific Config\"\n },\n \"line\": {\n \"$ref\": \"#/definitions/LineConfig\",\n \"description\": \"Line-Specific Config\"\n },\n \"mark\": {\n \"$ref\": \"#/definitions/MarkConfig\",\n \"description\": \"Mark Config\"\n },\n \"point\": {\n \"$ref\": \"#/definitions/MarkConfig\",\n \"description\": \"Point-Specific Config\"\n },\n \"rect\": {\n \"$ref\": \"#/definitions/RectConfig\",\n \"description\": \"Rect-Specific Config\"\n },\n \"rule\": {\n \"$ref\": \"#/definitions/MarkConfig\",\n \"description\": \"Rule-Specific Config\"\n },\n \"square\": {\n \"$ref\": \"#/definitions/MarkConfig\",\n \"description\": \"Square-Specific Config\"\n },\n \"text\": {\n \"$ref\": \"#/definitions/MarkConfig\",\n \"description\": \"Text-Specific Config\"\n },\n \"tick\": {\n \"$ref\": \"#/definitions/TickConfig\",\n \"description\": \"Tick-Specific Config\"\n },\n \"trail\": {\n \"$ref\": \"#/definitions/LineConfig\",\n \"description\": \"Trail-Specific Config\"\n }\n },\n \"type\": \"object\"\n },\n \"SymbolShape\": {\n \"type\": \"string\"\n },\n \"Text\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n ]\n },\n \"TextBaseline\": {\n \"anyOf\": [\n {\n \"const\": \"alphabetic\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Baseline\"\n },\n {\n \"const\": \"line-top\",\n \"type\": \"string\"\n },\n {\n \"const\": \"line-bottom\",\n \"type\": \"string\"\n }\n ]\n },\n \"TextDef\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FieldOrDatumDefWithCondition\"\n },\n {\n \"$ref\": \"#/definitions/FieldOrDatumDefWithCondition\"\n },\n {\n \"$ref\": \"#/definitions/ValueDefWithCondition\"\n }\n ]\n },\n \"TextDirection\": {\n \"enum\": [\n \"ltr\",\n \"rtl\"\n ],\n \"type\": \"string\"\n },\n \"TickConfig\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"align\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Align\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The horizontal alignment of the text or ranged marks (area, bar, image, rect, rule). One of `\\\"left\\\"`, `\\\"right\\\"`, `\\\"center\\\"`.\\n\\n__Note:__ Expression reference is *not* supported for range marks.\"\n },\n \"angle\": {\n \"anyOf\": [\n {\n \"description\": \"The rotation angle of the text, in degrees.\",\n \"maximum\": 360,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"aria\": {\n \"anyOf\": [\n {\n \"description\": \"A boolean flag indicating if [ARIA attributes](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) should be included (SVG output only). If `false`, the \\\"aria-hidden\\\" attribute will be set on the output SVG element, removing the mark item from the ARIA accessibility tree.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"ariaRole\": {\n \"anyOf\": [\n {\n \"description\": \"Sets the type of user interface element of the mark item for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If specified, this property determines the \\\"role\\\" attribute. Warning: this property is experimental and may be changed in the future.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"ariaRoleDescription\": {\n \"anyOf\": [\n {\n \"description\": \"A human-readable, author-localized description for the role of the mark item for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If specified, this property determines the \\\"aria-roledescription\\\" attribute. Warning: this property is experimental and may be changed in the future.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"aspect\": {\n \"anyOf\": [\n {\n \"description\": \"Whether to keep aspect ratio of image marks.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"bandSize\": {\n \"description\": \"The width of the ticks.\\n\\n__Default value:__ 3/4 of step (width step for horizontal ticks and height step for vertical ticks).\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"baseline\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextBaseline\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"For text marks, the vertical text baseline. One of `\\\"alphabetic\\\"` (default), `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`, `\\\"line-top\\\"`, `\\\"line-bottom\\\"`, or an expression reference that provides one of the valid values. The `\\\"line-top\\\"` and `\\\"line-bottom\\\"` values operate similarly to `\\\"top\\\"` and `\\\"bottom\\\"`, but are calculated relative to the `lineHeight` rather than `fontSize` alone.\\n\\nFor range marks, the vertical alignment of the marks. One of `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`.\\n\\n__Note:__ Expression reference is *not* supported for range marks.\"\n },\n \"blend\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Blend\",\n \"description\": \"The color blend mode for drawing an item on its current background. Any valid [CSS mix-blend-mode](https://developer.mozilla.org/en-US/docs/Web/CSS/mix-blend-mode) value can be used.\\n\\n__Default value: `\\\"source-over\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"color\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default color.\\n\\n__Default value:__ ■ `\\\"#4682b4\\\"`\\n\\n__Note:__\\n- This property cannot be used in a [style config](https://vega.github.io/vega-lite/docs/mark.html#style-config).\\n- The `fill` and `stroke` properties have higher precedence than `color` and will override `color`.\"\n },\n \"cornerRadius\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles or arcs' corners.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusBottomLeft\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' bottom left corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusBottomRight\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' bottom right corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusTopLeft\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' top right corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cornerRadiusTopRight\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles' top left corner.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cursor\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Cursor\",\n \"description\": \"The mouse cursor used over the mark. Any valid [CSS cursor type](https://developer.mozilla.org/en-US/docs/Web/CSS/cursor#Values) can be used.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"description\": {\n \"anyOf\": [\n {\n \"description\": \"A text description of the mark item for [ARIA accessibility](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) (SVG output only). If specified, this property determines the [\\\"aria-label\\\" attribute](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/ARIA_Techniques/Using_the_aria-label_attribute).\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"dir\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TextDirection\",\n \"description\": \"The direction of the text. One of `\\\"ltr\\\"` (left-to-right) or `\\\"rtl\\\"` (right-to-left). This property determines on which side is truncated in response to the limit parameter.\\n\\n__Default value:__ `\\\"ltr\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"dx\": {\n \"anyOf\": [\n {\n \"description\": \"The horizontal offset, in pixels, between the text label and its anchor point. The offset is applied after rotation by the _angle_ property.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"dy\": {\n \"anyOf\": [\n {\n \"description\": \"The vertical offset, in pixels, between the text label and its anchor point. The offset is applied after rotation by the _angle_ property.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"ellipsis\": {\n \"anyOf\": [\n {\n \"description\": \"The ellipsis string for text truncated in response to the limit parameter.\\n\\n__Default value:__ `\\\"…\\\"`\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"endAngle\": {\n \"anyOf\": [\n {\n \"description\": \"The end angle in radians for arc marks. A value of `0` indicates up (north), increasing values proceed clockwise.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fill\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default fill color. This property has higher precedence than `config.color`. Set to `null` to remove fill.\\n\\n__Default value:__ (None)\"\n },\n \"fillOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"The fill opacity (value between [0,1]).\\n\\n__Default value:__ `1`\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"filled\": {\n \"description\": \"Whether the mark's color should be used as fill color instead of stroke color.\\n\\n__Default value:__ `false` for all `point`, `line`, and `rule` marks as well as `geoshape` marks for [`graticule`](https://vega.github.io/vega-lite/docs/data.html#graticule) data sources; otherwise, `true`.\\n\\n__Note:__ This property cannot be used in a [style config](https://vega.github.io/vega-lite/docs/mark.html#style-config).\",\n \"type\": \"boolean\"\n },\n \"font\": {\n \"anyOf\": [\n {\n \"description\": \"The typeface to set the text in (e.g., `\\\"Helvetica Neue\\\"`).\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontSize\": {\n \"anyOf\": [\n {\n \"description\": \"The font size, in pixels.\\n\\n__Default value:__ `11`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontStyle\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\",\n \"description\": \"The font style (e.g., `\\\"italic\\\"`).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontWeight\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\",\n \"description\": \"The font weight. This can be either a string (e.g `\\\"bold\\\"`, `\\\"normal\\\"`) or a number (`100`, `200`, `300`, ..., `900` where `\\\"normal\\\"` = `400` and `\\\"bold\\\"` = `700`).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"height\": {\n \"anyOf\": [\n {\n \"description\": \"Height of the marks.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"href\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/URI\",\n \"description\": \"A URL to load upon mouse click. If defined, the mark acts as a hyperlink.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"innerRadius\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The inner radius in pixels of arc marks. `innerRadius` is an alias for `radius2`.\\n\\n__Default value:__ `0`\",\n \"minimum\": 0\n },\n \"interpolate\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Interpolate\",\n \"description\": \"The line interpolation method to use for line and area marks. One of the following:\\n- `\\\"linear\\\"`: piecewise linear segments, as in a polyline.\\n- `\\\"linear-closed\\\"`: close the linear segments to form a polygon.\\n- `\\\"step\\\"`: alternate between horizontal and vertical segments, as in a step function.\\n- `\\\"step-before\\\"`: alternate between vertical and horizontal segments, as in a step function.\\n- `\\\"step-after\\\"`: alternate between horizontal and vertical segments, as in a step function.\\n- `\\\"basis\\\"`: a B-spline, with control point duplication on the ends.\\n- `\\\"basis-open\\\"`: an open B-spline; may not intersect the start or end.\\n- `\\\"basis-closed\\\"`: a closed B-spline, as in a loop.\\n- `\\\"cardinal\\\"`: a Cardinal spline, with control point duplication on the ends.\\n- `\\\"cardinal-open\\\"`: an open Cardinal spline; may not intersect the start or end, but will intersect other control points.\\n- `\\\"cardinal-closed\\\"`: a closed Cardinal spline, as in a loop.\\n- `\\\"bundle\\\"`: equivalent to basis, except the tension parameter is used to straighten the spline.\\n- `\\\"monotone\\\"`: cubic interpolation that preserves monotonicity in y.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"invalid\": {\n \"description\": \"Defines how Vega-Lite should handle marks for invalid values (`null` and `NaN`).\\n- If set to `\\\"filter\\\"` (default), all data items with null values will be skipped (for line, trail, and area marks) or filtered (for other marks).\\n- If `null`, all data items are included. In this case, invalid values will be interpreted as zeroes.\",\n \"enum\": [\n \"filter\",\n null\n ],\n \"type\": [\n \"string\",\n \"null\"\n ]\n },\n \"limit\": {\n \"anyOf\": [\n {\n \"description\": \"The maximum length of the text mark in pixels. The text value will be automatically truncated if the rendered size exceeds the limit.\\n\\n__Default value:__ `0` -- indicating no limit\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"lineBreak\": {\n \"anyOf\": [\n {\n \"description\": \"A delimiter, such as a newline character, upon which to break text strings into multiple lines. This property is ignored if the text is array-valued.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"lineHeight\": {\n \"anyOf\": [\n {\n \"description\": \"The line height in pixels (the spacing between subsequent lines of text) for multi-line text marks.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"opacity\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The overall opacity (value between [0,1]).\\n\\n__Default value:__ `0.7` for non-aggregate plots with `point`, `tick`, `circle`, or `square` marks or layered `bar` charts and `1` otherwise.\",\n \"maximum\": 1,\n \"minimum\": 0\n },\n \"order\": {\n \"description\": \"For line and trail marks, this `order` property can be set to `null` or `false` to make the lines use the original order in the data sources.\",\n \"type\": [\n \"null\",\n \"boolean\"\n ]\n },\n \"orient\": {\n \"$ref\": \"#/definitions/Orientation\",\n \"description\": \"The orientation of a non-stacked bar, tick, area, and line charts. The value is either horizontal (default) or vertical.\\n- For bar, rule and tick, this determines whether the size of the bar and tick should be applied to x or y dimension.\\n- For area, this property determines the orient property of the Vega output.\\n- For line and trail marks, this property determines the sort order of the points in the line if `config.sortLineBy` is not specified. For stacked charts, this is always determined by the orientation of the stack; therefore explicitly specified value will be ignored.\"\n },\n \"outerRadius\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The outer radius in pixels of arc marks. `outerRadius` is an alias for `radius`.\\n\\n__Default value:__ `0`\",\n \"minimum\": 0\n },\n \"padAngle\": {\n \"anyOf\": [\n {\n \"description\": \"The angular padding applied to sides of the arc, in radians.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"radius\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"For arc mark, the primary (outer) radius in pixels.\\n\\nFor text marks, polar coordinate radial offset, in pixels, of the text from the origin determined by the `x` and `y` properties.\\n\\n__Default value:__ `min(plot_width, plot_height)/2`\",\n \"minimum\": 0\n },\n \"radius2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The secondary (inner) radius in pixels of arc marks.\\n\\n__Default value:__ `0`\",\n \"minimum\": 0\n },\n \"shape\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SymbolShape\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Shape of the point marks. Supported values include:\\n- plotting shapes: `\\\"circle\\\"`, `\\\"square\\\"`, `\\\"cross\\\"`, `\\\"diamond\\\"`, `\\\"triangle-up\\\"`, `\\\"triangle-down\\\"`, `\\\"triangle-right\\\"`, or `\\\"triangle-left\\\"`.\\n- the line symbol `\\\"stroke\\\"`\\n- centered directional shapes `\\\"arrow\\\"`, `\\\"wedge\\\"`, or `\\\"triangle\\\"`\\n- a custom [SVG path string](https://developer.mozilla.org/en-US/docs/Web/SVG/Tutorial/Paths) (For correct sizing, custom shape paths should be defined within a square bounding box with coordinates ranging from -1 to 1 along both the x and y dimensions.)\\n\\n__Default value:__ `\\\"circle\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"size\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default size for marks.\\n- For `point`/`circle`/`square`, this represents the pixel area of the marks. Note that this value sets the area of the symbol; the side lengths will increase with the square root of this value.\\n- For `bar`, this represents the band size of the bar, in pixels.\\n- For `text`, this represents the font size, in pixels.\\n\\n__Default value:__\\n- `30` for point, circle, square marks; width/height's `step`\\n- `2` for bar marks with discrete dimensions;\\n- `5` for bar marks with continuous dimensions;\\n- `11` for text marks.\",\n \"minimum\": 0\n },\n \"smooth\": {\n \"anyOf\": [\n {\n \"description\": \"A boolean flag (default true) indicating if the image should be smoothed when resized. If false, individual pixels should be scaled directly rather than interpolated with smoothing. For SVG rendering, this option may not work in some browsers due to lack of standardization.\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"startAngle\": {\n \"anyOf\": [\n {\n \"description\": \"The start angle in radians for arc marks. A value of `0` indicates up (north), increasing values proceed clockwise.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"stroke\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Default stroke color. This property has higher precedence than `config.color`. Set to `null` to remove stroke.\\n\\n__Default value:__ (None)\"\n },\n \"strokeCap\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeCap\",\n \"description\": \"The stroke cap for line ending style. One of `\\\"butt\\\"`, `\\\"round\\\"`, or `\\\"square\\\"`.\\n\\n__Default value:__ `\\\"butt\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeDash\": {\n \"anyOf\": [\n {\n \"description\": \"An array of alternating stroke, space lengths for creating dashed or dotted lines.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeDashOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The offset (in pixels) into which to begin drawing with the stroke dash array.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeJoin\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeJoin\",\n \"description\": \"The stroke line join method. One of `\\\"miter\\\"`, `\\\"round\\\"` or `\\\"bevel\\\"`.\\n\\n__Default value:__ `\\\"miter\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeMiterLimit\": {\n \"anyOf\": [\n {\n \"description\": \"The miter limit at which to bevel a line join.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The offset in pixels at which to draw the group stroke and fill. If unspecified, the default behavior is to dynamically offset stroked groups such that 1 pixel stroke widths align with the pixel grid.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"The stroke opacity (value between [0,1]).\\n\\n__Default value:__ `1`\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeWidth\": {\n \"anyOf\": [\n {\n \"description\": \"The stroke width, in pixels.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"tension\": {\n \"anyOf\": [\n {\n \"description\": \"Depending on the interpolation type, sets the tension parameter (for line and area marks).\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"text\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\",\n \"description\": \"Placeholder text if the `text` channel is not specified\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"theta\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"- For arc marks, the arc length in radians if theta2 is not specified, otherwise the start arc angle. (A value of 0 indicates up or “north”, increasing values proceed clockwise.)\\n\\n- For text marks, polar coordinate angle in radians.\",\n \"maximum\": 360,\n \"minimum\": 0\n },\n \"theta2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The end angle of arc marks in radians. A value of 0 indicates up or “north”, increasing values proceed clockwise.\"\n },\n \"thickness\": {\n \"description\": \"Thickness of the tick mark.\\n\\n__Default value:__ `1`\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"timeUnitBandPosition\": {\n \"description\": \"Default relative band position for a time unit. If set to `0`, the marks will be positioned at the beginning of the time unit band step. If set to `0.5`, the marks will be positioned in the middle of the time unit band step.\",\n \"type\": \"number\"\n },\n \"timeUnitBandSize\": {\n \"description\": \"Default relative band size for a time unit. If set to `1`, the bandwidth of the marks will be equal to the time unit band step. If set to `0.5`, bandwidth of the marks will be half of the time unit band step.\",\n \"type\": \"number\"\n },\n \"tooltip\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/TooltipContent\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"The tooltip text string to show upon mouse hover or an object defining which fields should the tooltip be derived from.\\n\\n- If `tooltip` is `true` or `{\\\"content\\\": \\\"encoding\\\"}`, then all fields from `encoding` will be used.\\n- If `tooltip` is `{\\\"content\\\": \\\"data\\\"}`, then all fields that appear in the highlighted data point will be used.\\n- If set to `null` or `false`, then no tooltip will be used.\\n\\nSee the [`tooltip`](https://vega.github.io/vega-lite/docs/tooltip.html) documentation for a detailed discussion about tooltip in Vega-Lite.\\n\\n__Default value:__ `null`\"\n },\n \"url\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/URI\",\n \"description\": \"The URL of the image file for image marks.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"width\": {\n \"anyOf\": [\n {\n \"description\": \"Width of the marks.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"x\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"X coordinates of the marks, or width of horizontal `\\\"bar\\\"` and `\\\"area\\\"` without specified `x2` or `width`.\\n\\nThe `value` of this channel can be a number or a string `\\\"width\\\"` for the width of the plot.\"\n },\n \"x2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"X2 coordinates for ranged `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"rect\\\"`, and `\\\"rule\\\"`.\\n\\nThe `value` of this channel can be a number or a string `\\\"width\\\"` for the width of the plot.\"\n },\n \"y\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Y coordinates of the marks, or height of vertical `\\\"bar\\\"` and `\\\"area\\\"` without specified `y2` or `height`.\\n\\nThe `value` of this channel can be a number or a string `\\\"height\\\"` for the height of the plot.\"\n },\n \"y2\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"Y2 coordinates for ranged `\\\"area\\\"`, `\\\"bar\\\"`, `\\\"rect\\\"`, and `\\\"rule\\\"`.\\n\\nThe `value` of this channel can be a number or a string `\\\"height\\\"` for the height of the plot.\"\n }\n },\n \"type\": \"object\"\n },\n \"TickCount\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/TimeInterval\"\n },\n {\n \"$ref\": \"#/definitions/TimeIntervalStep\"\n }\n ]\n },\n \"TimeInterval\": {\n \"enum\": [\n \"millisecond\",\n \"second\",\n \"minute\",\n \"hour\",\n \"day\",\n \"week\",\n \"month\",\n \"year\"\n ],\n \"type\": \"string\"\n },\n \"TimeIntervalStep\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"interval\": {\n \"$ref\": \"#/definitions/TimeInterval\"\n },\n \"step\": {\n \"type\": \"number\"\n }\n },\n \"required\": [\n \"interval\",\n \"step\"\n ],\n \"type\": \"object\"\n },\n \"TimeLocale\": {\n \"additionalProperties\": false,\n \"description\": \"Locale definition for formatting dates and times.\",\n \"properties\": {\n \"date\": {\n \"description\": \"The date (%x) format specifier (e.g., \\\"%m/%d/%Y\\\").\",\n \"type\": \"string\"\n },\n \"dateTime\": {\n \"description\": \"The date and time (%c) format specifier (e.g., \\\"%a %b %e %X %Y\\\").\",\n \"type\": \"string\"\n },\n \"days\": {\n \"$ref\": \"#/definitions/Vector7\",\n \"description\": \"The full names of the weekdays, starting with Sunday.\"\n },\n \"months\": {\n \"$ref\": \"#/definitions/Vector12\",\n \"description\": \"The full names of the months (starting with January).\"\n },\n \"periods\": {\n \"$ref\": \"#/definitions/Vector2\",\n \"description\": \"The A.M. and P.M. equivalents (e.g., [\\\"AM\\\", \\\"PM\\\"]).\"\n },\n \"shortDays\": {\n \"$ref\": \"#/definitions/Vector7\",\n \"description\": \"The abbreviated names of the weekdays, starting with Sunday.\"\n },\n \"shortMonths\": {\n \"$ref\": \"#/definitions/Vector12\",\n \"description\": \"The abbreviated names of the months (starting with January).\"\n },\n \"time\": {\n \"description\": \"The time (%X) format specifier (e.g., \\\"%H:%M:%S\\\").\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"dateTime\",\n \"date\",\n \"time\",\n \"periods\",\n \"days\",\n \"shortDays\",\n \"months\",\n \"shortMonths\"\n ],\n \"type\": \"object\"\n },\n \"TimeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SingleTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/MultiTimeUnit\"\n }\n ]\n },\n \"TimeUnitParams\": {\n \"additionalProperties\": false,\n \"description\": \"Time Unit Params for encoding predicate, which can specified if the data is already \\\"binned\\\".\",\n \"properties\": {\n \"binned\": {\n \"description\": \"Whether the data has already been binned to this time unit. If true, Vega-Lite will only format the data, marks, and guides, without applying the timeUnit transform to re-bin the data again.\",\n \"type\": \"boolean\"\n },\n \"maxbins\": {\n \"description\": \"If no `unit` is specified, maxbins is used to infer time units.\",\n \"type\": \"number\"\n },\n \"step\": {\n \"description\": \"The number of steps between bins, in terms of the least significant unit provided.\",\n \"type\": \"number\"\n },\n \"unit\": {\n \"$ref\": \"#/definitions/TimeUnit\",\n \"description\": \"Defines how date-time values should be binned.\"\n },\n \"utc\": {\n \"description\": \"True to use UTC timezone. Equivalent to using a `utc` prefixed `TimeUnit`.\",\n \"type\": \"boolean\"\n }\n },\n \"type\": \"object\"\n },\n \"TimeUnitTransform\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"as\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"The output field to write the timeUnit value.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"The data field to apply time unit.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitTransformParams\"\n }\n ],\n \"description\": \"The timeUnit.\"\n }\n },\n \"required\": [\n \"timeUnit\",\n \"field\",\n \"as\"\n ],\n \"type\": \"object\"\n },\n \"TimeUnitTransformParams\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"maxbins\": {\n \"description\": \"If no `unit` is specified, maxbins is used to infer time units.\",\n \"type\": \"number\"\n },\n \"step\": {\n \"description\": \"The number of steps between bins, in terms of the least significant unit provided.\",\n \"type\": \"number\"\n },\n \"unit\": {\n \"$ref\": \"#/definitions/TimeUnit\",\n \"description\": \"Defines how date-time values should be binned.\"\n },\n \"utc\": {\n \"description\": \"True to use UTC timezone. Equivalent to using a `utc` prefixed `TimeUnit`.\",\n \"type\": \"boolean\"\n }\n },\n \"type\": \"object\"\n },\n \"TitleAnchor\": {\n \"enum\": [\n null,\n \"start\",\n \"middle\",\n \"end\"\n ],\n \"type\": [\n \"null\",\n \"string\"\n ]\n },\n \"TitleConfig\": {\n \"$ref\": \"#/definitions/BaseTitleNoValueRefs\"\n },\n \"TitleFrame\": {\n \"enum\": [\n \"bounds\",\n \"group\"\n ],\n \"type\": \"string\"\n },\n \"TitleOrient\": {\n \"enum\": [\n \"none\",\n \"left\",\n \"right\",\n \"top\",\n \"bottom\"\n ],\n \"type\": \"string\"\n },\n \"TitleParams\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"align\": {\n \"$ref\": \"#/definitions/Align\",\n \"description\": \"Horizontal text alignment for title text. One of `\\\"left\\\"`, `\\\"center\\\"`, or `\\\"right\\\"`.\"\n },\n \"anchor\": {\n \"$ref\": \"#/definitions/TitleAnchor\",\n \"description\": \"The anchor position for placing the title. One of `\\\"start\\\"`, `\\\"middle\\\"`, or `\\\"end\\\"`. For example, with an orientation of top these anchor positions map to a left-, center-, or right-aligned title.\\n\\n__Default value:__ `\\\"middle\\\"` for [single](https://vega.github.io/vega-lite/docs/spec.html) and [layered](https://vega.github.io/vega-lite/docs/layer.html) views. `\\\"start\\\"` for other composite views.\\n\\n__Note:__ [For now](https://github.com/vega/vega-lite/issues/2875), `anchor` is only customizable only for [single](https://vega.github.io/vega-lite/docs/spec.html) and [layered](https://vega.github.io/vega-lite/docs/layer.html) views. For other composite views, `anchor` is always `\\\"start\\\"`.\"\n },\n \"angle\": {\n \"anyOf\": [\n {\n \"description\": \"Angle in degrees of title and subtitle text.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"aria\": {\n \"anyOf\": [\n {\n \"description\": \"A boolean flag indicating if [ARIA attributes](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) should be included (SVG output only). If `false`, the \\\"aria-hidden\\\" attribute will be set on the output SVG group, removing the title from the ARIA accessibility tree.\\n\\n__Default value:__ `true`\",\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"baseline\": {\n \"$ref\": \"#/definitions/TextBaseline\",\n \"description\": \"Vertical text baseline for title and subtitle text. One of `\\\"alphabetic\\\"` (default), `\\\"top\\\"`, `\\\"middle\\\"`, `\\\"bottom\\\"`, `\\\"line-top\\\"`, or `\\\"line-bottom\\\"`. The `\\\"line-top\\\"` and `\\\"line-bottom\\\"` values operate similarly to `\\\"top\\\"` and `\\\"bottom\\\"`, but are calculated relative to the *lineHeight* rather than *fontSize* alone.\"\n },\n \"color\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"Text color for title text.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"dx\": {\n \"anyOf\": [\n {\n \"description\": \"Delta offset for title and subtitle text x-coordinate.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"dy\": {\n \"anyOf\": [\n {\n \"description\": \"Delta offset for title and subtitle text y-coordinate.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"font\": {\n \"anyOf\": [\n {\n \"description\": \"Font name for title text.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontSize\": {\n \"anyOf\": [\n {\n \"description\": \"Font size in pixels for title text.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontStyle\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\",\n \"description\": \"Font style for title text.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"fontWeight\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\",\n \"description\": \"Font weight for title text. This can be either a string (e.g `\\\"bold\\\"`, `\\\"normal\\\"`) or a number (`100`, `200`, `300`, ..., `900` where `\\\"normal\\\"` = `400` and `\\\"bold\\\"` = `700`).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"frame\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TitleFrame\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"The reference frame for the anchor position, one of `\\\"bounds\\\"` (to anchor relative to the full bounding box) or `\\\"group\\\"` (to anchor relative to the group width or height).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"limit\": {\n \"anyOf\": [\n {\n \"description\": \"The maximum allowed length in pixels of title and subtitle text.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"lineHeight\": {\n \"anyOf\": [\n {\n \"description\": \"Line height in pixels for multi-line title text or title text with `\\\"line-top\\\"` or `\\\"line-bottom\\\"` baseline.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"offset\": {\n \"anyOf\": [\n {\n \"description\": \"The orthogonal offset in pixels by which to displace the title group from its position along the edge of the chart.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"orient\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TitleOrient\",\n \"description\": \"Default title orientation (`\\\"top\\\"`, `\\\"bottom\\\"`, `\\\"left\\\"`, or `\\\"right\\\"`)\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"style\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A [mark style property](https://vega.github.io/vega-lite/docs/config.html#style) to apply to the title text mark.\\n\\n__Default value:__ `\\\"group-title\\\"`.\"\n },\n \"subtitle\": {\n \"$ref\": \"#/definitions/Text\",\n \"description\": \"The subtitle Text.\"\n },\n \"subtitleColor\": {\n \"anyOf\": [\n {\n \"anyOf\": [\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/Color\"\n }\n ],\n \"description\": \"Text color for subtitle text.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"subtitleFont\": {\n \"anyOf\": [\n {\n \"description\": \"Font name for subtitle text.\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"subtitleFontSize\": {\n \"anyOf\": [\n {\n \"description\": \"Font size in pixels for subtitle text.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"subtitleFontStyle\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontStyle\",\n \"description\": \"Font style for subtitle text.\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"subtitleFontWeight\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FontWeight\",\n \"description\": \"Font weight for subtitle text. This can be either a string (e.g `\\\"bold\\\"`, `\\\"normal\\\"`) or a number (`100`, `200`, `300`, ..., `900` where `\\\"normal\\\"` = `400` and `\\\"bold\\\"` = `700`).\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"subtitleLineHeight\": {\n \"anyOf\": [\n {\n \"description\": \"Line height in pixels for multi-line subtitle text.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"subtitlePadding\": {\n \"anyOf\": [\n {\n \"description\": \"The padding in pixels between title and subtitle text.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"text\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The title text.\"\n },\n \"zindex\": {\n \"description\": \"The integer z-index indicating the layering of the title group relative to other axis, mark and legend groups.\\n\\n__Default value:__ `0`.\",\n \"minimum\": 0,\n \"type\": \"number\"\n }\n },\n \"required\": [\n \"text\"\n ],\n \"type\": \"object\"\n },\n \"TooltipContent\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"content\": {\n \"enum\": [\n \"encoding\",\n \"data\"\n ],\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"content\"\n ],\n \"type\": \"object\"\n },\n \"TopLevelConcatSpec\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"$schema\": {\n \"description\": \"URL to [JSON schema](http://json-schema.org/) for a Vega-Lite specification. Unless you have a reason to change this, use `https://vega.github.io/schema/vega-lite/v5.json`. Setting the `$schema` property allows automatic validation and autocomplete in editors that support JSON schema.\",\n \"format\": \"uri\",\n \"type\": \"string\"\n },\n \"align\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LayoutAlign\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"The alignment to apply to grid rows and columns. The supported string values are `\\\"all\\\"`, `\\\"each\\\"`, and `\\\"none\\\"`.\\n\\n- For `\\\"none\\\"`, a flow layout will be used, in which adjacent subviews are simply placed one after the other.\\n- For `\\\"each\\\"`, subviews will be aligned into a clean grid structure, but each row or column may be of variable size.\\n- For `\\\"all\\\"`, subviews will be aligned and each row or column will be sized identically based on the maximum observed size. String values for this property will be applied to both grid rows and columns.\\n\\nAlternatively, an object value of the form `{\\\"row\\\": string, \\\"column\\\": string}` can be used to supply different alignments for rows and columns.\\n\\n__Default value:__ `\\\"all\\\"`.\"\n },\n \"autosize\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/AutosizeType\"\n },\n {\n \"$ref\": \"#/definitions/AutoSizeParams\"\n }\n ],\n \"description\": \"How the visualization size should be determined. If a string, should be one of `\\\"pad\\\"`, `\\\"fit\\\"` or `\\\"none\\\"`. Object values can additionally specify parameters for content sizing and automatic resizing.\\n\\n__Default value__: `pad`\"\n },\n \"background\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"CSS color property to use as the background of the entire view.\\n\\n__Default value:__ `\\\"white\\\"`\"\n },\n \"bounds\": {\n \"description\": \"The bounds calculation method to use for determining the extent of a sub-plot. One of `full` (the default) or `flush`.\\n\\n- If set to `full`, the entire calculated bounds (including axes, title, and legend) will be used.\\n- If set to `flush`, only the specified width and height values for the sub-view will be used. The `flush` setting can be useful when attempting to place sub-plots without axes or legends into a uniform grid structure.\\n\\n__Default value:__ `\\\"full\\\"`\",\n \"enum\": [\n \"full\",\n \"flush\"\n ],\n \"type\": \"string\"\n },\n \"center\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"Boolean flag indicating if subviews should be centered relative to their respective rows or columns.\\n\\nAn object value of the form `{\\\"row\\\": boolean, \\\"column\\\": boolean}` can be used to supply different centering values for rows and columns.\\n\\n__Default value:__ `false`\"\n },\n \"columns\": {\n \"description\": \"The number of columns to include in the view composition layout.\\n\\n__Default value__: `undefined` -- An infinite number of columns (a single row) will be assumed. This is equivalent to `hconcat` (for `concat`) and to using the `column` channel (for `facet` and `repeat`).\\n\\n__Note__:\\n\\n1) This property is only for:\\n- the general (wrappable) `concat` operator (not `hconcat`/`vconcat`)\\n- the `facet` and `repeat` operator with one field/repetition definition (without row/column nesting)\\n\\n2) Setting the `columns` to `1` is equivalent to `vconcat` (for `concat`) and to using the `row` channel (for `facet` and `repeat`).\",\n \"type\": \"number\"\n },\n \"concat\": {\n \"description\": \"A list of views to be concatenated.\",\n \"items\": {\n \"$ref\": \"#/definitions/NonNormalizedSpec\"\n },\n \"type\": \"array\"\n },\n \"config\": {\n \"$ref\": \"#/definitions/Config\",\n \"description\": \"Vega-Lite configuration object. This property can only be defined at the top-level of a specification.\"\n },\n \"data\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Data\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object describing the data source. Set to `null` to ignore the parent's data source. If no data is set, it is derived from the parent.\"\n },\n \"datasets\": {\n \"$ref\": \"#/definitions/Datasets\",\n \"description\": \"A global data store for named datasets. This is a mapping from names to inline datasets. This can be an array of objects or primitive values or a string. Arrays of primitive values are ingested as objects with a `data` property.\"\n },\n \"description\": {\n \"description\": \"Description of this mark for commenting purpose.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the visualization for later reference.\",\n \"type\": \"string\"\n },\n \"padding\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Padding\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The default visualization padding, in pixels, from the edge of the visualization canvas to the data rectangle. If a number, specifies padding for all sides. If an object, the value should have the format `{\\\"left\\\": 5, \\\"top\\\": 5, \\\"right\\\": 5, \\\"bottom\\\": 5}` to specify padding for each side of the visualization.\\n\\n__Default value__: `5`\"\n },\n \"params\": {\n \"description\": \"Dynamic variables or selections that parameterize a visualization.\",\n \"items\": {\n \"$ref\": \"#/definitions/TopLevelParameter\"\n },\n \"type\": \"array\"\n },\n \"resolve\": {\n \"$ref\": \"#/definitions/Resolve\",\n \"description\": \"Scale, axis, and legend resolutions for view composition specifications.\"\n },\n \"spacing\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"The spacing in pixels between sub-views of the composition operator. An object of the form `{\\\"row\\\": number, \\\"column\\\": number}` can be used to set different spacing values for rows and columns.\\n\\n__Default value__: Depends on `\\\"spacing\\\"` property of [the view composition configuration](https://vega.github.io/vega-lite/docs/config.html#view-config) (`20` by default)\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"$ref\": \"#/definitions/TitleParams\"\n }\n ],\n \"description\": \"Title for the plot.\"\n },\n \"transform\": {\n \"description\": \"An array of data transformations such as filter and new field calculation.\",\n \"items\": {\n \"$ref\": \"#/definitions/Transform\"\n },\n \"type\": \"array\"\n },\n \"usermeta\": {\n \"$ref\": \"#/definitions/Dict\",\n \"description\": \"Optional metadata that will be passed to Vega. This object is completely ignored by Vega and Vega-Lite and can be used for custom metadata.\"\n }\n },\n \"required\": [\n \"concat\"\n ],\n \"type\": \"object\"\n },\n \"TopLevelHConcatSpec\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"$schema\": {\n \"description\": \"URL to [JSON schema](http://json-schema.org/) for a Vega-Lite specification. Unless you have a reason to change this, use `https://vega.github.io/schema/vega-lite/v5.json`. Setting the `$schema` property allows automatic validation and autocomplete in editors that support JSON schema.\",\n \"format\": \"uri\",\n \"type\": \"string\"\n },\n \"autosize\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/AutosizeType\"\n },\n {\n \"$ref\": \"#/definitions/AutoSizeParams\"\n }\n ],\n \"description\": \"How the visualization size should be determined. If a string, should be one of `\\\"pad\\\"`, `\\\"fit\\\"` or `\\\"none\\\"`. Object values can additionally specify parameters for content sizing and automatic resizing.\\n\\n__Default value__: `pad`\"\n },\n \"background\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"CSS color property to use as the background of the entire view.\\n\\n__Default value:__ `\\\"white\\\"`\"\n },\n \"bounds\": {\n \"description\": \"The bounds calculation method to use for determining the extent of a sub-plot. One of `full` (the default) or `flush`.\\n\\n- If set to `full`, the entire calculated bounds (including axes, title, and legend) will be used.\\n- If set to `flush`, only the specified width and height values for the sub-view will be used. The `flush` setting can be useful when attempting to place sub-plots without axes or legends into a uniform grid structure.\\n\\n__Default value:__ `\\\"full\\\"`\",\n \"enum\": [\n \"full\",\n \"flush\"\n ],\n \"type\": \"string\"\n },\n \"center\": {\n \"description\": \"Boolean flag indicating if subviews should be centered relative to their respective rows or columns.\\n\\n__Default value:__ `false`\",\n \"type\": \"boolean\"\n },\n \"config\": {\n \"$ref\": \"#/definitions/Config\",\n \"description\": \"Vega-Lite configuration object. This property can only be defined at the top-level of a specification.\"\n },\n \"data\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Data\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object describing the data source. Set to `null` to ignore the parent's data source. If no data is set, it is derived from the parent.\"\n },\n \"datasets\": {\n \"$ref\": \"#/definitions/Datasets\",\n \"description\": \"A global data store for named datasets. This is a mapping from names to inline datasets. This can be an array of objects or primitive values or a string. Arrays of primitive values are ingested as objects with a `data` property.\"\n },\n \"description\": {\n \"description\": \"Description of this mark for commenting purpose.\",\n \"type\": \"string\"\n },\n \"hconcat\": {\n \"description\": \"A list of views to be concatenated and put into a row.\",\n \"items\": {\n \"$ref\": \"#/definitions/NonNormalizedSpec\"\n },\n \"type\": \"array\"\n },\n \"name\": {\n \"description\": \"Name of the visualization for later reference.\",\n \"type\": \"string\"\n },\n \"padding\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Padding\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The default visualization padding, in pixels, from the edge of the visualization canvas to the data rectangle. If a number, specifies padding for all sides. If an object, the value should have the format `{\\\"left\\\": 5, \\\"top\\\": 5, \\\"right\\\": 5, \\\"bottom\\\": 5}` to specify padding for each side of the visualization.\\n\\n__Default value__: `5`\"\n },\n \"params\": {\n \"description\": \"Dynamic variables or selections that parameterize a visualization.\",\n \"items\": {\n \"$ref\": \"#/definitions/TopLevelParameter\"\n },\n \"type\": \"array\"\n },\n \"resolve\": {\n \"$ref\": \"#/definitions/Resolve\",\n \"description\": \"Scale, axis, and legend resolutions for view composition specifications.\"\n },\n \"spacing\": {\n \"description\": \"The spacing in pixels between sub-views of the concat operator.\\n\\n__Default value__: `10`\",\n \"type\": \"number\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"$ref\": \"#/definitions/TitleParams\"\n }\n ],\n \"description\": \"Title for the plot.\"\n },\n \"transform\": {\n \"description\": \"An array of data transformations such as filter and new field calculation.\",\n \"items\": {\n \"$ref\": \"#/definitions/Transform\"\n },\n \"type\": \"array\"\n },\n \"usermeta\": {\n \"$ref\": \"#/definitions/Dict\",\n \"description\": \"Optional metadata that will be passed to Vega. This object is completely ignored by Vega and Vega-Lite and can be used for custom metadata.\"\n }\n },\n \"required\": [\n \"hconcat\"\n ],\n \"type\": \"object\"\n },\n \"TopLevelVConcatSpec\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"$schema\": {\n \"description\": \"URL to [JSON schema](http://json-schema.org/) for a Vega-Lite specification. Unless you have a reason to change this, use `https://vega.github.io/schema/vega-lite/v5.json`. Setting the `$schema` property allows automatic validation and autocomplete in editors that support JSON schema.\",\n \"format\": \"uri\",\n \"type\": \"string\"\n },\n \"autosize\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/AutosizeType\"\n },\n {\n \"$ref\": \"#/definitions/AutoSizeParams\"\n }\n ],\n \"description\": \"How the visualization size should be determined. If a string, should be one of `\\\"pad\\\"`, `\\\"fit\\\"` or `\\\"none\\\"`. Object values can additionally specify parameters for content sizing and automatic resizing.\\n\\n__Default value__: `pad`\"\n },\n \"background\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"CSS color property to use as the background of the entire view.\\n\\n__Default value:__ `\\\"white\\\"`\"\n },\n \"bounds\": {\n \"description\": \"The bounds calculation method to use for determining the extent of a sub-plot. One of `full` (the default) or `flush`.\\n\\n- If set to `full`, the entire calculated bounds (including axes, title, and legend) will be used.\\n- If set to `flush`, only the specified width and height values for the sub-view will be used. The `flush` setting can be useful when attempting to place sub-plots without axes or legends into a uniform grid structure.\\n\\n__Default value:__ `\\\"full\\\"`\",\n \"enum\": [\n \"full\",\n \"flush\"\n ],\n \"type\": \"string\"\n },\n \"center\": {\n \"description\": \"Boolean flag indicating if subviews should be centered relative to their respective rows or columns.\\n\\n__Default value:__ `false`\",\n \"type\": \"boolean\"\n },\n \"config\": {\n \"$ref\": \"#/definitions/Config\",\n \"description\": \"Vega-Lite configuration object. This property can only be defined at the top-level of a specification.\"\n },\n \"data\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Data\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object describing the data source. Set to `null` to ignore the parent's data source. If no data is set, it is derived from the parent.\"\n },\n \"datasets\": {\n \"$ref\": \"#/definitions/Datasets\",\n \"description\": \"A global data store for named datasets. This is a mapping from names to inline datasets. This can be an array of objects or primitive values or a string. Arrays of primitive values are ingested as objects with a `data` property.\"\n },\n \"description\": {\n \"description\": \"Description of this mark for commenting purpose.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the visualization for later reference.\",\n \"type\": \"string\"\n },\n \"padding\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Padding\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The default visualization padding, in pixels, from the edge of the visualization canvas to the data rectangle. If a number, specifies padding for all sides. If an object, the value should have the format `{\\\"left\\\": 5, \\\"top\\\": 5, \\\"right\\\": 5, \\\"bottom\\\": 5}` to specify padding for each side of the visualization.\\n\\n__Default value__: `5`\"\n },\n \"params\": {\n \"description\": \"Dynamic variables or selections that parameterize a visualization.\",\n \"items\": {\n \"$ref\": \"#/definitions/TopLevelParameter\"\n },\n \"type\": \"array\"\n },\n \"resolve\": {\n \"$ref\": \"#/definitions/Resolve\",\n \"description\": \"Scale, axis, and legend resolutions for view composition specifications.\"\n },\n \"spacing\": {\n \"description\": \"The spacing in pixels between sub-views of the concat operator.\\n\\n__Default value__: `10`\",\n \"type\": \"number\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"$ref\": \"#/definitions/TitleParams\"\n }\n ],\n \"description\": \"Title for the plot.\"\n },\n \"transform\": {\n \"description\": \"An array of data transformations such as filter and new field calculation.\",\n \"items\": {\n \"$ref\": \"#/definitions/Transform\"\n },\n \"type\": \"array\"\n },\n \"usermeta\": {\n \"$ref\": \"#/definitions/Dict\",\n \"description\": \"Optional metadata that will be passed to Vega. This object is completely ignored by Vega and Vega-Lite and can be used for custom metadata.\"\n },\n \"vconcat\": {\n \"description\": \"A list of views to be concatenated and put into a column.\",\n \"items\": {\n \"$ref\": \"#/definitions/NonNormalizedSpec\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"vconcat\"\n ],\n \"type\": \"object\"\n },\n \"TopLevelLayerSpec\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"$schema\": {\n \"description\": \"URL to [JSON schema](http://json-schema.org/) for a Vega-Lite specification. Unless you have a reason to change this, use `https://vega.github.io/schema/vega-lite/v5.json`. Setting the `$schema` property allows automatic validation and autocomplete in editors that support JSON schema.\",\n \"format\": \"uri\",\n \"type\": \"string\"\n },\n \"autosize\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/AutosizeType\"\n },\n {\n \"$ref\": \"#/definitions/AutoSizeParams\"\n }\n ],\n \"description\": \"How the visualization size should be determined. If a string, should be one of `\\\"pad\\\"`, `\\\"fit\\\"` or `\\\"none\\\"`. Object values can additionally specify parameters for content sizing and automatic resizing.\\n\\n__Default value__: `pad`\"\n },\n \"background\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"CSS color property to use as the background of the entire view.\\n\\n__Default value:__ `\\\"white\\\"`\"\n },\n \"config\": {\n \"$ref\": \"#/definitions/Config\",\n \"description\": \"Vega-Lite configuration object. This property can only be defined at the top-level of a specification.\"\n },\n \"data\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Data\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object describing the data source. Set to `null` to ignore the parent's data source. If no data is set, it is derived from the parent.\"\n },\n \"datasets\": {\n \"$ref\": \"#/definitions/Datasets\",\n \"description\": \"A global data store for named datasets. This is a mapping from names to inline datasets. This can be an array of objects or primitive values or a string. Arrays of primitive values are ingested as objects with a `data` property.\"\n },\n \"description\": {\n \"description\": \"Description of this mark for commenting purpose.\",\n \"type\": \"string\"\n },\n \"encoding\": {\n \"$ref\": \"#/definitions/SharedEncoding\",\n \"description\": \"A shared key-value mapping between encoding channels and definition of fields in the underlying layers.\"\n },\n \"height\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"container\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Step\"\n }\n ],\n \"description\": \"The height of a visualization.\\n\\n- For a plot with a continuous y-field, height should be a number.\\n- For a plot with either a discrete y-field or no y-field, height can be either a number indicating a fixed height or an object in the form of `{step: number}` defining the height per discrete step. (No y-field is equivalent to having one discrete step.)\\n- To enable responsive sizing on height, it should be set to `\\\"container\\\"`.\\n\\n__Default value:__ Based on `config.view.continuousHeight` for a plot with a continuous y-field and `config.view.discreteHeight` otherwise.\\n\\n__Note:__ For plots with [`row` and `column` channels](https://vega.github.io/vega-lite/docs/encoding.html#facet), this represents the height of a single view and the `\\\"container\\\"` option cannot be used.\\n\\n__See also:__ [`height`](https://vega.github.io/vega-lite/docs/size.html) documentation.\"\n },\n \"layer\": {\n \"description\": \"Layer or single view specifications to be layered.\\n\\n__Note__: Specifications inside `layer` cannot use `row` and `column` channels as layering facet specifications is not allowed. Instead, use the [facet operator](https://vega.github.io/vega-lite/docs/facet.html) and place a layer inside a facet.\",\n \"items\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LayerSpec\"\n },\n {\n \"$ref\": \"#/definitions/UnitSpec\"\n }\n ]\n },\n \"type\": \"array\"\n },\n \"name\": {\n \"description\": \"Name of the visualization for later reference.\",\n \"type\": \"string\"\n },\n \"padding\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Padding\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The default visualization padding, in pixels, from the edge of the visualization canvas to the data rectangle. If a number, specifies padding for all sides. If an object, the value should have the format `{\\\"left\\\": 5, \\\"top\\\": 5, \\\"right\\\": 5, \\\"bottom\\\": 5}` to specify padding for each side of the visualization.\\n\\n__Default value__: `5`\"\n },\n \"params\": {\n \"description\": \"Dynamic variables or selections that parameterize a visualization.\",\n \"items\": {\n \"$ref\": \"#/definitions/TopLevelParameter\"\n },\n \"type\": \"array\"\n },\n \"projection\": {\n \"$ref\": \"#/definitions/Projection\",\n \"description\": \"An object defining properties of the geographic projection shared by underlying layers.\"\n },\n \"resolve\": {\n \"$ref\": \"#/definitions/Resolve\",\n \"description\": \"Scale, axis, and legend resolutions for view composition specifications.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"$ref\": \"#/definitions/TitleParams\"\n }\n ],\n \"description\": \"Title for the plot.\"\n },\n \"transform\": {\n \"description\": \"An array of data transformations such as filter and new field calculation.\",\n \"items\": {\n \"$ref\": \"#/definitions/Transform\"\n },\n \"type\": \"array\"\n },\n \"usermeta\": {\n \"$ref\": \"#/definitions/Dict\",\n \"description\": \"Optional metadata that will be passed to Vega. This object is completely ignored by Vega and Vega-Lite and can be used for custom metadata.\"\n },\n \"view\": {\n \"$ref\": \"#/definitions/ViewBackground\",\n \"description\": \"An object defining the view background's fill and stroke.\\n\\n__Default value:__ none (transparent)\"\n },\n \"width\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"container\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Step\"\n }\n ],\n \"description\": \"The width of a visualization.\\n\\n- For a plot with a continuous x-field, width should be a number.\\n- For a plot with either a discrete x-field or no x-field, width can be either a number indicating a fixed width or an object in the form of `{step: number}` defining the width per discrete step. (No x-field is equivalent to having one discrete step.)\\n- To enable responsive sizing on width, it should be set to `\\\"container\\\"`.\\n\\n__Default value:__ Based on `config.view.continuousWidth` for a plot with a continuous x-field and `config.view.discreteWidth` otherwise.\\n\\n__Note:__ For plots with [`row` and `column` channels](https://vega.github.io/vega-lite/docs/encoding.html#facet), this represents the width of a single view and the `\\\"container\\\"` option cannot be used.\\n\\n__See also:__ [`width`](https://vega.github.io/vega-lite/docs/size.html) documentation.\"\n }\n },\n \"required\": [\n \"layer\"\n ],\n \"type\": \"object\"\n },\n \"TopLevelRepeatSpec\": {\n \"anyOf\": [\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"$schema\": {\n \"description\": \"URL to [JSON schema](http://json-schema.org/) for a Vega-Lite specification. Unless you have a reason to change this, use `https://vega.github.io/schema/vega-lite/v5.json`. Setting the `$schema` property allows automatic validation and autocomplete in editors that support JSON schema.\",\n \"format\": \"uri\",\n \"type\": \"string\"\n },\n \"align\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LayoutAlign\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"The alignment to apply to grid rows and columns. The supported string values are `\\\"all\\\"`, `\\\"each\\\"`, and `\\\"none\\\"`.\\n\\n- For `\\\"none\\\"`, a flow layout will be used, in which adjacent subviews are simply placed one after the other.\\n- For `\\\"each\\\"`, subviews will be aligned into a clean grid structure, but each row or column may be of variable size.\\n- For `\\\"all\\\"`, subviews will be aligned and each row or column will be sized identically based on the maximum observed size. String values for this property will be applied to both grid rows and columns.\\n\\nAlternatively, an object value of the form `{\\\"row\\\": string, \\\"column\\\": string}` can be used to supply different alignments for rows and columns.\\n\\n__Default value:__ `\\\"all\\\"`.\"\n },\n \"autosize\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/AutosizeType\"\n },\n {\n \"$ref\": \"#/definitions/AutoSizeParams\"\n }\n ],\n \"description\": \"How the visualization size should be determined. If a string, should be one of `\\\"pad\\\"`, `\\\"fit\\\"` or `\\\"none\\\"`. Object values can additionally specify parameters for content sizing and automatic resizing.\\n\\n__Default value__: `pad`\"\n },\n \"background\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"CSS color property to use as the background of the entire view.\\n\\n__Default value:__ `\\\"white\\\"`\"\n },\n \"bounds\": {\n \"description\": \"The bounds calculation method to use for determining the extent of a sub-plot. One of `full` (the default) or `flush`.\\n\\n- If set to `full`, the entire calculated bounds (including axes, title, and legend) will be used.\\n- If set to `flush`, only the specified width and height values for the sub-view will be used. The `flush` setting can be useful when attempting to place sub-plots without axes or legends into a uniform grid structure.\\n\\n__Default value:__ `\\\"full\\\"`\",\n \"enum\": [\n \"full\",\n \"flush\"\n ],\n \"type\": \"string\"\n },\n \"center\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"Boolean flag indicating if subviews should be centered relative to their respective rows or columns.\\n\\nAn object value of the form `{\\\"row\\\": boolean, \\\"column\\\": boolean}` can be used to supply different centering values for rows and columns.\\n\\n__Default value:__ `false`\"\n },\n \"columns\": {\n \"description\": \"The number of columns to include in the view composition layout.\\n\\n__Default value__: `undefined` -- An infinite number of columns (a single row) will be assumed. This is equivalent to `hconcat` (for `concat`) and to using the `column` channel (for `facet` and `repeat`).\\n\\n__Note__:\\n\\n1) This property is only for:\\n- the general (wrappable) `concat` operator (not `hconcat`/`vconcat`)\\n- the `facet` and `repeat` operator with one field/repetition definition (without row/column nesting)\\n\\n2) Setting the `columns` to `1` is equivalent to `vconcat` (for `concat`) and to using the `row` channel (for `facet` and `repeat`).\",\n \"type\": \"number\"\n },\n \"config\": {\n \"$ref\": \"#/definitions/Config\",\n \"description\": \"Vega-Lite configuration object. This property can only be defined at the top-level of a specification.\"\n },\n \"data\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Data\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object describing the data source. Set to `null` to ignore the parent's data source. If no data is set, it is derived from the parent.\"\n },\n \"datasets\": {\n \"$ref\": \"#/definitions/Datasets\",\n \"description\": \"A global data store for named datasets. This is a mapping from names to inline datasets. This can be an array of objects or primitive values or a string. Arrays of primitive values are ingested as objects with a `data` property.\"\n },\n \"description\": {\n \"description\": \"Description of this mark for commenting purpose.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the visualization for later reference.\",\n \"type\": \"string\"\n },\n \"padding\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Padding\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The default visualization padding, in pixels, from the edge of the visualization canvas to the data rectangle. If a number, specifies padding for all sides. If an object, the value should have the format `{\\\"left\\\": 5, \\\"top\\\": 5, \\\"right\\\": 5, \\\"bottom\\\": 5}` to specify padding for each side of the visualization.\\n\\n__Default value__: `5`\"\n },\n \"params\": {\n \"description\": \"Dynamic variables or selections that parameterize a visualization.\",\n \"items\": {\n \"$ref\": \"#/definitions/TopLevelParameter\"\n },\n \"type\": \"array\"\n },\n \"repeat\": {\n \"anyOf\": [\n {\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/RepeatMapping\"\n }\n ],\n \"description\": \"Definition for fields to be repeated. One of: 1) An array of fields to be repeated. If `\\\"repeat\\\"` is an array, the field can be referred to as `{\\\"repeat\\\": \\\"repeat\\\"}`. The repeated views are laid out in a wrapped row. You can set the number of columns to control the wrapping. 2) An object that maps `\\\"row\\\"` and/or `\\\"column\\\"` to the listed fields to be repeated along the particular orientations. The objects `{\\\"repeat\\\": \\\"row\\\"}` and `{\\\"repeat\\\": \\\"column\\\"}` can be used to refer to the repeated field respectively.\"\n },\n \"resolve\": {\n \"$ref\": \"#/definitions/Resolve\",\n \"description\": \"Scale, axis, and legend resolutions for view composition specifications.\"\n },\n \"spacing\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"The spacing in pixels between sub-views of the composition operator. An object of the form `{\\\"row\\\": number, \\\"column\\\": number}` can be used to set different spacing values for rows and columns.\\n\\n__Default value__: Depends on `\\\"spacing\\\"` property of [the view composition configuration](https://vega.github.io/vega-lite/docs/config.html#view-config) (`20` by default)\"\n },\n \"spec\": {\n \"$ref\": \"#/definitions/NonNormalizedSpec\",\n \"description\": \"A specification of the view that gets repeated.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"$ref\": \"#/definitions/TitleParams\"\n }\n ],\n \"description\": \"Title for the plot.\"\n },\n \"transform\": {\n \"description\": \"An array of data transformations such as filter and new field calculation.\",\n \"items\": {\n \"$ref\": \"#/definitions/Transform\"\n },\n \"type\": \"array\"\n },\n \"usermeta\": {\n \"$ref\": \"#/definitions/Dict\",\n \"description\": \"Optional metadata that will be passed to Vega. This object is completely ignored by Vega and Vega-Lite and can be used for custom metadata.\"\n }\n },\n \"required\": [\n \"repeat\",\n \"spec\"\n ],\n \"type\": \"object\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"$schema\": {\n \"description\": \"URL to [JSON schema](http://json-schema.org/) for a Vega-Lite specification. Unless you have a reason to change this, use `https://vega.github.io/schema/vega-lite/v5.json`. Setting the `$schema` property allows automatic validation and autocomplete in editors that support JSON schema.\",\n \"format\": \"uri\",\n \"type\": \"string\"\n },\n \"align\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LayoutAlign\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"The alignment to apply to grid rows and columns. The supported string values are `\\\"all\\\"`, `\\\"each\\\"`, and `\\\"none\\\"`.\\n\\n- For `\\\"none\\\"`, a flow layout will be used, in which adjacent subviews are simply placed one after the other.\\n- For `\\\"each\\\"`, subviews will be aligned into a clean grid structure, but each row or column may be of variable size.\\n- For `\\\"all\\\"`, subviews will be aligned and each row or column will be sized identically based on the maximum observed size. String values for this property will be applied to both grid rows and columns.\\n\\nAlternatively, an object value of the form `{\\\"row\\\": string, \\\"column\\\": string}` can be used to supply different alignments for rows and columns.\\n\\n__Default value:__ `\\\"all\\\"`.\"\n },\n \"autosize\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/AutosizeType\"\n },\n {\n \"$ref\": \"#/definitions/AutoSizeParams\"\n }\n ],\n \"description\": \"How the visualization size should be determined. If a string, should be one of `\\\"pad\\\"`, `\\\"fit\\\"` or `\\\"none\\\"`. Object values can additionally specify parameters for content sizing and automatic resizing.\\n\\n__Default value__: `pad`\"\n },\n \"background\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"CSS color property to use as the background of the entire view.\\n\\n__Default value:__ `\\\"white\\\"`\"\n },\n \"bounds\": {\n \"description\": \"The bounds calculation method to use for determining the extent of a sub-plot. One of `full` (the default) or `flush`.\\n\\n- If set to `full`, the entire calculated bounds (including axes, title, and legend) will be used.\\n- If set to `flush`, only the specified width and height values for the sub-view will be used. The `flush` setting can be useful when attempting to place sub-plots without axes or legends into a uniform grid structure.\\n\\n__Default value:__ `\\\"full\\\"`\",\n \"enum\": [\n \"full\",\n \"flush\"\n ],\n \"type\": \"string\"\n },\n \"center\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"Boolean flag indicating if subviews should be centered relative to their respective rows or columns.\\n\\nAn object value of the form `{\\\"row\\\": boolean, \\\"column\\\": boolean}` can be used to supply different centering values for rows and columns.\\n\\n__Default value:__ `false`\"\n },\n \"columns\": {\n \"description\": \"The number of columns to include in the view composition layout.\\n\\n__Default value__: `undefined` -- An infinite number of columns (a single row) will be assumed. This is equivalent to `hconcat` (for `concat`) and to using the `column` channel (for `facet` and `repeat`).\\n\\n__Note__:\\n\\n1) This property is only for:\\n- the general (wrappable) `concat` operator (not `hconcat`/`vconcat`)\\n- the `facet` and `repeat` operator with one field/repetition definition (without row/column nesting)\\n\\n2) Setting the `columns` to `1` is equivalent to `vconcat` (for `concat`) and to using the `row` channel (for `facet` and `repeat`).\",\n \"type\": \"number\"\n },\n \"config\": {\n \"$ref\": \"#/definitions/Config\",\n \"description\": \"Vega-Lite configuration object. This property can only be defined at the top-level of a specification.\"\n },\n \"data\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Data\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object describing the data source. Set to `null` to ignore the parent's data source. If no data is set, it is derived from the parent.\"\n },\n \"datasets\": {\n \"$ref\": \"#/definitions/Datasets\",\n \"description\": \"A global data store for named datasets. This is a mapping from names to inline datasets. This can be an array of objects or primitive values or a string. Arrays of primitive values are ingested as objects with a `data` property.\"\n },\n \"description\": {\n \"description\": \"Description of this mark for commenting purpose.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the visualization for later reference.\",\n \"type\": \"string\"\n },\n \"padding\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Padding\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The default visualization padding, in pixels, from the edge of the visualization canvas to the data rectangle. If a number, specifies padding for all sides. If an object, the value should have the format `{\\\"left\\\": 5, \\\"top\\\": 5, \\\"right\\\": 5, \\\"bottom\\\": 5}` to specify padding for each side of the visualization.\\n\\n__Default value__: `5`\"\n },\n \"params\": {\n \"description\": \"Dynamic variables or selections that parameterize a visualization.\",\n \"items\": {\n \"$ref\": \"#/definitions/TopLevelParameter\"\n },\n \"type\": \"array\"\n },\n \"repeat\": {\n \"$ref\": \"#/definitions/LayerRepeatMapping\",\n \"description\": \"Definition for fields to be repeated. One of: 1) An array of fields to be repeated. If `\\\"repeat\\\"` is an array, the field can be referred to as `{\\\"repeat\\\": \\\"repeat\\\"}`. The repeated views are laid out in a wrapped row. You can set the number of columns to control the wrapping. 2) An object that maps `\\\"row\\\"` and/or `\\\"column\\\"` to the listed fields to be repeated along the particular orientations. The objects `{\\\"repeat\\\": \\\"row\\\"}` and `{\\\"repeat\\\": \\\"column\\\"}` can be used to refer to the repeated field respectively.\"\n },\n \"resolve\": {\n \"$ref\": \"#/definitions/Resolve\",\n \"description\": \"Scale, axis, and legend resolutions for view composition specifications.\"\n },\n \"spacing\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"The spacing in pixels between sub-views of the composition operator. An object of the form `{\\\"row\\\": number, \\\"column\\\": number}` can be used to set different spacing values for rows and columns.\\n\\n__Default value__: Depends on `\\\"spacing\\\"` property of [the view composition configuration](https://vega.github.io/vega-lite/docs/config.html#view-config) (`20` by default)\"\n },\n \"spec\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LayerSpec\"\n },\n {\n \"$ref\": \"#/definitions/UnitSpecWithFrame\"\n }\n ],\n \"description\": \"A specification of the view that gets repeated.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"$ref\": \"#/definitions/TitleParams\"\n }\n ],\n \"description\": \"Title for the plot.\"\n },\n \"transform\": {\n \"description\": \"An array of data transformations such as filter and new field calculation.\",\n \"items\": {\n \"$ref\": \"#/definitions/Transform\"\n },\n \"type\": \"array\"\n },\n \"usermeta\": {\n \"$ref\": \"#/definitions/Dict\",\n \"description\": \"Optional metadata that will be passed to Vega. This object is completely ignored by Vega and Vega-Lite and can be used for custom metadata.\"\n }\n },\n \"required\": [\n \"repeat\",\n \"spec\"\n ],\n \"type\": \"object\"\n }\n ]\n },\n \"TopLevelFacetSpec\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"$schema\": {\n \"description\": \"URL to [JSON schema](http://json-schema.org/) for a Vega-Lite specification. Unless you have a reason to change this, use `https://vega.github.io/schema/vega-lite/v5.json`. Setting the `$schema` property allows automatic validation and autocomplete in editors that support JSON schema.\",\n \"format\": \"uri\",\n \"type\": \"string\"\n },\n \"align\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LayoutAlign\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"The alignment to apply to grid rows and columns. The supported string values are `\\\"all\\\"`, `\\\"each\\\"`, and `\\\"none\\\"`.\\n\\n- For `\\\"none\\\"`, a flow layout will be used, in which adjacent subviews are simply placed one after the other.\\n- For `\\\"each\\\"`, subviews will be aligned into a clean grid structure, but each row or column may be of variable size.\\n- For `\\\"all\\\"`, subviews will be aligned and each row or column will be sized identically based on the maximum observed size. String values for this property will be applied to both grid rows and columns.\\n\\nAlternatively, an object value of the form `{\\\"row\\\": string, \\\"column\\\": string}` can be used to supply different alignments for rows and columns.\\n\\n__Default value:__ `\\\"all\\\"`.\"\n },\n \"autosize\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/AutosizeType\"\n },\n {\n \"$ref\": \"#/definitions/AutoSizeParams\"\n }\n ],\n \"description\": \"How the visualization size should be determined. If a string, should be one of `\\\"pad\\\"`, `\\\"fit\\\"` or `\\\"none\\\"`. Object values can additionally specify parameters for content sizing and automatic resizing.\\n\\n__Default value__: `pad`\"\n },\n \"background\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"CSS color property to use as the background of the entire view.\\n\\n__Default value:__ `\\\"white\\\"`\"\n },\n \"bounds\": {\n \"description\": \"The bounds calculation method to use for determining the extent of a sub-plot. One of `full` (the default) or `flush`.\\n\\n- If set to `full`, the entire calculated bounds (including axes, title, and legend) will be used.\\n- If set to `flush`, only the specified width and height values for the sub-view will be used. The `flush` setting can be useful when attempting to place sub-plots without axes or legends into a uniform grid structure.\\n\\n__Default value:__ `\\\"full\\\"`\",\n \"enum\": [\n \"full\",\n \"flush\"\n ],\n \"type\": \"string\"\n },\n \"center\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"Boolean flag indicating if subviews should be centered relative to their respective rows or columns.\\n\\nAn object value of the form `{\\\"row\\\": boolean, \\\"column\\\": boolean}` can be used to supply different centering values for rows and columns.\\n\\n__Default value:__ `false`\"\n },\n \"columns\": {\n \"description\": \"The number of columns to include in the view composition layout.\\n\\n__Default value__: `undefined` -- An infinite number of columns (a single row) will be assumed. This is equivalent to `hconcat` (for `concat`) and to using the `column` channel (for `facet` and `repeat`).\\n\\n__Note__:\\n\\n1) This property is only for:\\n- the general (wrappable) `concat` operator (not `hconcat`/`vconcat`)\\n- the `facet` and `repeat` operator with one field/repetition definition (without row/column nesting)\\n\\n2) Setting the `columns` to `1` is equivalent to `vconcat` (for `concat`) and to using the `row` channel (for `facet` and `repeat`).\",\n \"type\": \"number\"\n },\n \"config\": {\n \"$ref\": \"#/definitions/Config\",\n \"description\": \"Vega-Lite configuration object. This property can only be defined at the top-level of a specification.\"\n },\n \"data\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Data\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object describing the data source. Set to `null` to ignore the parent's data source. If no data is set, it is derived from the parent.\"\n },\n \"datasets\": {\n \"$ref\": \"#/definitions/Datasets\",\n \"description\": \"A global data store for named datasets. This is a mapping from names to inline datasets. This can be an array of objects or primitive values or a string. Arrays of primitive values are ingested as objects with a `data` property.\"\n },\n \"description\": {\n \"description\": \"Description of this mark for commenting purpose.\",\n \"type\": \"string\"\n },\n \"facet\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/FacetFieldDef\"\n },\n {\n \"$ref\": \"#/definitions/FacetMapping\"\n }\n ],\n \"description\": \"Definition for how to facet the data. One of: 1) [a field definition for faceting the plot by one field](https://vega.github.io/vega-lite/docs/facet.html#field-def) 2) [An object that maps `row` and `column` channels to their field definitions](https://vega.github.io/vega-lite/docs/facet.html#mapping)\"\n },\n \"name\": {\n \"description\": \"Name of the visualization for later reference.\",\n \"type\": \"string\"\n },\n \"padding\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Padding\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The default visualization padding, in pixels, from the edge of the visualization canvas to the data rectangle. If a number, specifies padding for all sides. If an object, the value should have the format `{\\\"left\\\": 5, \\\"top\\\": 5, \\\"right\\\": 5, \\\"bottom\\\": 5}` to specify padding for each side of the visualization.\\n\\n__Default value__: `5`\"\n },\n \"params\": {\n \"description\": \"Dynamic variables or selections that parameterize a visualization.\",\n \"items\": {\n \"$ref\": \"#/definitions/TopLevelParameter\"\n },\n \"type\": \"array\"\n },\n \"resolve\": {\n \"$ref\": \"#/definitions/Resolve\",\n \"description\": \"Scale, axis, and legend resolutions for view composition specifications.\"\n },\n \"spacing\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"The spacing in pixels between sub-views of the composition operator. An object of the form `{\\\"row\\\": number, \\\"column\\\": number}` can be used to set different spacing values for rows and columns.\\n\\n__Default value__: Depends on `\\\"spacing\\\"` property of [the view composition configuration](https://vega.github.io/vega-lite/docs/config.html#view-config) (`20` by default)\"\n },\n \"spec\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LayerSpec\"\n },\n {\n \"$ref\": \"#/definitions/UnitSpecWithFrame\"\n }\n ],\n \"description\": \"A specification of the view that gets faceted.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"$ref\": \"#/definitions/TitleParams\"\n }\n ],\n \"description\": \"Title for the plot.\"\n },\n \"transform\": {\n \"description\": \"An array of data transformations such as filter and new field calculation.\",\n \"items\": {\n \"$ref\": \"#/definitions/Transform\"\n },\n \"type\": \"array\"\n },\n \"usermeta\": {\n \"$ref\": \"#/definitions/Dict\",\n \"description\": \"Optional metadata that will be passed to Vega. This object is completely ignored by Vega and Vega-Lite and can be used for custom metadata.\"\n }\n },\n \"required\": [\n \"data\",\n \"facet\",\n \"spec\"\n ],\n \"type\": \"object\"\n },\n \"TopLevelParameter\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/VariableParameter\"\n },\n {\n \"$ref\": \"#/definitions/TopLevelSelectionParameter\"\n }\n ]\n },\n \"TopLevelSelectionParameter\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"bind\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Binding\"\n },\n {\n \"additionalProperties\": {\n \"$ref\": \"#/definitions/Binding\"\n },\n \"type\": \"object\"\n },\n {\n \"$ref\": \"#/definitions/LegendBinding\"\n },\n {\n \"const\": \"scales\",\n \"type\": \"string\"\n }\n ],\n \"description\": \"When set, a selection is populated by input elements (also known as dynamic query widgets) or by interacting with the corresponding legend. Direct manipulation interaction is disabled by default; to re-enable it, set the selection's [`on`](https://vega.github.io/vega-lite/docs/selection.html#common-selection-properties) property.\\n\\nLegend bindings are restricted to selections that only specify a single field or encoding.\\n\\nQuery widget binding takes the form of Vega's [input element binding definition](https://vega.github.io/vega/docs/signals/#bind) or can be a mapping between projected field/encodings and binding definitions.\\n\\n__See also:__ [`bind`](https://vega.github.io/vega-lite/docs/bind.html) documentation.\"\n },\n \"name\": {\n \"$ref\": \"#/definitions/ParameterName\",\n \"description\": \"Required. A unique name for the selection parameter. Selection names should be valid JavaScript identifiers: they should contain only alphanumeric characters (or \\\"$\\\", or \\\"_\\\") and may not start with a digit. Reserved keywords that may not be used as parameter names are \\\"datum\\\", \\\"event\\\", \\\"item\\\", and \\\"parent\\\".\"\n },\n \"select\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SelectionType\"\n },\n {\n \"$ref\": \"#/definitions/PointSelectionConfig\"\n },\n {\n \"$ref\": \"#/definitions/IntervalSelectionConfig\"\n }\n ],\n \"description\": \"Determines the default event processing and data query for the selection. Vega-Lite currently supports two selection types:\\n\\n- `\\\"point\\\"` -- to select multiple discrete data values; the first value is selected on `click` and additional values toggled on shift-click.\\n- `\\\"interval\\\"` -- to select a continuous range of data values on `drag`.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/SelectionInit\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/SelectionInitMapping\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/SelectionInitIntervalMapping\"\n }\n ],\n \"description\": \"Initialize the selection with a mapping between [projected channels or field names](https://vega.github.io/vega-lite/docs/selection.html#project) and initial values.\\n\\n__See also:__ [`init`](https://vega.github.io/vega-lite/docs/value.html) documentation.\"\n },\n \"views\": {\n \"description\": \"By default, top-level selections are applied to every view in the visualization. If this property is specified, selections will only be applied to views with the given names.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"name\",\n \"select\"\n ],\n \"type\": \"object\"\n },\n \"TopLevelSpec\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TopLevelUnitSpec\"\n },\n {\n \"$ref\": \"#/definitions/TopLevelFacetSpec\"\n },\n {\n \"$ref\": \"#/definitions/TopLevelLayerSpec\"\n },\n {\n \"$ref\": \"#/definitions/TopLevelRepeatSpec\"\n },\n {\n \"$ref\": \"#/definitions/TopLevelConcatSpec\"\n },\n {\n \"$ref\": \"#/definitions/TopLevelVConcatSpec\"\n },\n {\n \"$ref\": \"#/definitions/TopLevelHConcatSpec\"\n }\n ],\n \"description\": \"A Vega-Lite top-level specification. This is the root class for all Vega-Lite specifications. (The json schema is generated from this type.)\"\n },\n \"TopLevelUnitSpec\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"$schema\": {\n \"description\": \"URL to [JSON schema](http://json-schema.org/) for a Vega-Lite specification. Unless you have a reason to change this, use `https://vega.github.io/schema/vega-lite/v5.json`. Setting the `$schema` property allows automatic validation and autocomplete in editors that support JSON schema.\",\n \"format\": \"uri\",\n \"type\": \"string\"\n },\n \"align\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/LayoutAlign\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"The alignment to apply to grid rows and columns. The supported string values are `\\\"all\\\"`, `\\\"each\\\"`, and `\\\"none\\\"`.\\n\\n- For `\\\"none\\\"`, a flow layout will be used, in which adjacent subviews are simply placed one after the other.\\n- For `\\\"each\\\"`, subviews will be aligned into a clean grid structure, but each row or column may be of variable size.\\n- For `\\\"all\\\"`, subviews will be aligned and each row or column will be sized identically based on the maximum observed size. String values for this property will be applied to both grid rows and columns.\\n\\nAlternatively, an object value of the form `{\\\"row\\\": string, \\\"column\\\": string}` can be used to supply different alignments for rows and columns.\\n\\n__Default value:__ `\\\"all\\\"`.\"\n },\n \"autosize\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/AutosizeType\"\n },\n {\n \"$ref\": \"#/definitions/AutoSizeParams\"\n }\n ],\n \"description\": \"How the visualization size should be determined. If a string, should be one of `\\\"pad\\\"`, `\\\"fit\\\"` or `\\\"none\\\"`. Object values can additionally specify parameters for content sizing and automatic resizing.\\n\\n__Default value__: `pad`\"\n },\n \"background\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"CSS color property to use as the background of the entire view.\\n\\n__Default value:__ `\\\"white\\\"`\"\n },\n \"bounds\": {\n \"description\": \"The bounds calculation method to use for determining the extent of a sub-plot. One of `full` (the default) or `flush`.\\n\\n- If set to `full`, the entire calculated bounds (including axes, title, and legend) will be used.\\n- If set to `flush`, only the specified width and height values for the sub-view will be used. The `flush` setting can be useful when attempting to place sub-plots without axes or legends into a uniform grid structure.\\n\\n__Default value:__ `\\\"full\\\"`\",\n \"enum\": [\n \"full\",\n \"flush\"\n ],\n \"type\": \"string\"\n },\n \"center\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"Boolean flag indicating if subviews should be centered relative to their respective rows or columns.\\n\\nAn object value of the form `{\\\"row\\\": boolean, \\\"column\\\": boolean}` can be used to supply different centering values for rows and columns.\\n\\n__Default value:__ `false`\"\n },\n \"config\": {\n \"$ref\": \"#/definitions/Config\",\n \"description\": \"Vega-Lite configuration object. This property can only be defined at the top-level of a specification.\"\n },\n \"data\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Data\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object describing the data source. Set to `null` to ignore the parent's data source. If no data is set, it is derived from the parent.\"\n },\n \"datasets\": {\n \"$ref\": \"#/definitions/Datasets\",\n \"description\": \"A global data store for named datasets. This is a mapping from names to inline datasets. This can be an array of objects or primitive values or a string. Arrays of primitive values are ingested as objects with a `data` property.\"\n },\n \"description\": {\n \"description\": \"Description of this mark for commenting purpose.\",\n \"type\": \"string\"\n },\n \"encoding\": {\n \"$ref\": \"#/definitions/FacetedEncoding\",\n \"description\": \"A key-value mapping between encoding channels and definition of fields.\"\n },\n \"height\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"container\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Step\"\n }\n ],\n \"description\": \"The height of a visualization.\\n\\n- For a plot with a continuous y-field, height should be a number.\\n- For a plot with either a discrete y-field or no y-field, height can be either a number indicating a fixed height or an object in the form of `{step: number}` defining the height per discrete step. (No y-field is equivalent to having one discrete step.)\\n- To enable responsive sizing on height, it should be set to `\\\"container\\\"`.\\n\\n__Default value:__ Based on `config.view.continuousHeight` for a plot with a continuous y-field and `config.view.discreteHeight` otherwise.\\n\\n__Note:__ For plots with [`row` and `column` channels](https://vega.github.io/vega-lite/docs/encoding.html#facet), this represents the height of a single view and the `\\\"container\\\"` option cannot be used.\\n\\n__See also:__ [`height`](https://vega.github.io/vega-lite/docs/size.html) documentation.\"\n },\n \"mark\": {\n \"$ref\": \"#/definitions/AnyMark\",\n \"description\": \"A string describing the mark type (one of `\\\"bar\\\"`, `\\\"circle\\\"`, `\\\"square\\\"`, `\\\"tick\\\"`, `\\\"line\\\"`, `\\\"area\\\"`, `\\\"point\\\"`, `\\\"rule\\\"`, `\\\"geoshape\\\"`, and `\\\"text\\\"`) or a [mark definition object](https://vega.github.io/vega-lite/docs/mark.html#mark-def).\"\n },\n \"name\": {\n \"description\": \"Name of the visualization for later reference.\",\n \"type\": \"string\"\n },\n \"padding\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Padding\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The default visualization padding, in pixels, from the edge of the visualization canvas to the data rectangle. If a number, specifies padding for all sides. If an object, the value should have the format `{\\\"left\\\": 5, \\\"top\\\": 5, \\\"right\\\": 5, \\\"bottom\\\": 5}` to specify padding for each side of the visualization.\\n\\n__Default value__: `5`\"\n },\n \"params\": {\n \"description\": \"An array of parameters that may either be simple variables, or more complex selections that map user input to data queries.\",\n \"items\": {\n \"$ref\": \"#/definitions/TopLevelParameter\"\n },\n \"type\": \"array\"\n },\n \"projection\": {\n \"$ref\": \"#/definitions/Projection\",\n \"description\": \"An object defining properties of geographic projection, which will be applied to `shape` path for `\\\"geoshape\\\"` marks and to `latitude` and `\\\"longitude\\\"` channels for other marks.\"\n },\n \"resolve\": {\n \"$ref\": \"#/definitions/Resolve\",\n \"description\": \"Scale, axis, and legend resolutions for view composition specifications.\"\n },\n \"spacing\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/RowCol\"\n }\n ],\n \"description\": \"The spacing in pixels between sub-views of the composition operator. An object of the form `{\\\"row\\\": number, \\\"column\\\": number}` can be used to set different spacing values for rows and columns.\\n\\n__Default value__: Depends on `\\\"spacing\\\"` property of [the view composition configuration](https://vega.github.io/vega-lite/docs/config.html#view-config) (`20` by default)\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"$ref\": \"#/definitions/TitleParams\"\n }\n ],\n \"description\": \"Title for the plot.\"\n },\n \"transform\": {\n \"description\": \"An array of data transformations such as filter and new field calculation.\",\n \"items\": {\n \"$ref\": \"#/definitions/Transform\"\n },\n \"type\": \"array\"\n },\n \"usermeta\": {\n \"$ref\": \"#/definitions/Dict\",\n \"description\": \"Optional metadata that will be passed to Vega. This object is completely ignored by Vega and Vega-Lite and can be used for custom metadata.\"\n },\n \"view\": {\n \"$ref\": \"#/definitions/ViewBackground\",\n \"description\": \"An object defining the view background's fill and stroke.\\n\\n__Default value:__ none (transparent)\"\n },\n \"width\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"container\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Step\"\n }\n ],\n \"description\": \"The width of a visualization.\\n\\n- For a plot with a continuous x-field, width should be a number.\\n- For a plot with either a discrete x-field or no x-field, width can be either a number indicating a fixed width or an object in the form of `{step: number}` defining the width per discrete step. (No x-field is equivalent to having one discrete step.)\\n- To enable responsive sizing on width, it should be set to `\\\"container\\\"`.\\n\\n__Default value:__ Based on `config.view.continuousWidth` for a plot with a continuous x-field and `config.view.discreteWidth` otherwise.\\n\\n__Note:__ For plots with [`row` and `column` channels](https://vega.github.io/vega-lite/docs/encoding.html#facet), this represents the width of a single view and the `\\\"container\\\"` option cannot be used.\\n\\n__See also:__ [`width`](https://vega.github.io/vega-lite/docs/size.html) documentation.\"\n }\n },\n \"required\": [\n \"data\",\n \"mark\"\n ],\n \"type\": \"object\"\n },\n \"TopoDataFormat\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"feature\": {\n \"description\": \"The name of the TopoJSON object set to convert to a GeoJSON feature collection. For example, in a map of the world, there may be an object set named `\\\"countries\\\"`. Using the feature property, we can extract this set and generate a GeoJSON feature object for each country.\",\n \"type\": \"string\"\n },\n \"mesh\": {\n \"description\": \"The name of the TopoJSON object set to convert to mesh. Similar to the `feature` option, `mesh` extracts a named TopoJSON object set. Unlike the `feature` option, the corresponding geo data is returned as a single, unified mesh instance, not as individual GeoJSON features. Extracting a mesh is useful for more efficiently drawing borders or other geographic elements that you do not need to associate with specific regions such as individual countries, states or counties.\",\n \"type\": \"string\"\n },\n \"parse\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Parse\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"If set to `null`, disable type inference based on the spec and only use type inference based on the data. Alternatively, a parsing directive object can be provided for explicit data types. Each property of the object corresponds to a field name, and the value to the desired data type (one of `\\\"number\\\"`, `\\\"boolean\\\"`, `\\\"date\\\"`, or null (do not parse the field)). For example, `\\\"parse\\\": {\\\"modified_on\\\": \\\"date\\\"}` parses the `modified_on` field in each input record a Date value.\\n\\nFor `\\\"date\\\"`, we parse data based using JavaScript's [`Date.parse()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/parse). For Specific date formats can be provided (e.g., `{foo: \\\"date:'%m%d%Y'\\\"}`), using the [d3-time-format syntax](https://github.com/d3/d3-time-format#locale_format). UTC date format parsing is supported similarly (e.g., `{foo: \\\"utc:'%m%d%Y'\\\"}`). See more about [UTC time](https://vega.github.io/vega-lite/docs/timeunit.html#utc)\"\n },\n \"type\": {\n \"const\": \"topojson\",\n \"description\": \"Type of input data: `\\\"json\\\"`, `\\\"csv\\\"`, `\\\"tsv\\\"`, `\\\"dsv\\\"`.\\n\\n__Default value:__ The default format type is determined by the extension of the file URL. If no extension is detected, `\\\"json\\\"` will be used by default.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"Transform\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/AggregateTransform\"\n },\n {\n \"$ref\": \"#/definitions/BinTransform\"\n },\n {\n \"$ref\": \"#/definitions/CalculateTransform\"\n },\n {\n \"$ref\": \"#/definitions/DensityTransform\"\n },\n {\n \"$ref\": \"#/definitions/ExtentTransform\"\n },\n {\n \"$ref\": \"#/definitions/FilterTransform\"\n },\n {\n \"$ref\": \"#/definitions/FlattenTransform\"\n },\n {\n \"$ref\": \"#/definitions/FoldTransform\"\n },\n {\n \"$ref\": \"#/definitions/ImputeTransform\"\n },\n {\n \"$ref\": \"#/definitions/JoinAggregateTransform\"\n },\n {\n \"$ref\": \"#/definitions/LoessTransform\"\n },\n {\n \"$ref\": \"#/definitions/LookupTransform\"\n },\n {\n \"$ref\": \"#/definitions/QuantileTransform\"\n },\n {\n \"$ref\": \"#/definitions/RegressionTransform\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitTransform\"\n },\n {\n \"$ref\": \"#/definitions/SampleTransform\"\n },\n {\n \"$ref\": \"#/definitions/StackTransform\"\n },\n {\n \"$ref\": \"#/definitions/WindowTransform\"\n },\n {\n \"$ref\": \"#/definitions/PivotTransform\"\n }\n ]\n },\n \"Type\": {\n \"description\": \"Data type based on level of measurement\",\n \"enum\": [\n \"quantitative\",\n \"ordinal\",\n \"temporal\",\n \"nominal\",\n \"geojson\"\n ],\n \"type\": \"string\"\n },\n \"TypeForShape\": {\n \"enum\": [\n \"nominal\",\n \"ordinal\",\n \"geojson\"\n ],\n \"type\": \"string\"\n },\n \"TypedFieldDef\": {\n \"additionalProperties\": false,\n \"description\": \"Definition object for a data field, its type and transformation of an encoding channel.\",\n \"properties\": {\n \"aggregate\": {\n \"$ref\": \"#/definitions/Aggregate\",\n \"description\": \"Aggregation function for the field (e.g., `\\\"mean\\\"`, `\\\"sum\\\"`, `\\\"median\\\"`, `\\\"min\\\"`, `\\\"max\\\"`, `\\\"count\\\"`).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html) documentation.\"\n },\n \"bandPosition\": {\n \"description\": \"Relative position on a band of a stacked, binned, time unit, or band scale. For example, the marks will be positioned at the beginning of the band if set to `0`, and at the middle of the band if set to `0.5`.\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n \"bin\": {\n \"anyOf\": [\n {\n \"type\": \"boolean\"\n },\n {\n \"$ref\": \"#/definitions/BinParams\"\n },\n {\n \"const\": \"binned\",\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A flag for binning a `quantitative` field, [an object defining binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters), or indicating that the data for `x` or `y` channel are binned before they are imported into Vega-Lite (`\\\"binned\\\"`).\\n\\n- If `true`, default [binning parameters](https://vega.github.io/vega-lite/docs/bin.html#bin-parameters) will be applied.\\n\\n- If `\\\"binned\\\"`, this indicates that the data for the `x` (or `y`) channel are already binned. You can map the bin-start field to `x` (or `y`) and the bin-end field to `x2` (or `y2`). The scale and axis will be formatted similar to binning in Vega-Lite. To adjust the axis ticks based on the bin step, you can also set the axis's [`tickMinStep`](https://vega.github.io/vega-lite/docs/axis.html#ticks) property.\\n\\n__Default value:__ `false`\\n\\n__See also:__ [`bin`](https://vega.github.io/vega-lite/docs/bin.html) documentation.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/Field\",\n \"description\": \"__Required.__ A string defining the name of the field from which to pull a data value or an object defining iterated values from the [`repeat`](https://vega.github.io/vega-lite/docs/repeat.html) operator.\\n\\n__See also:__ [`field`](https://vega.github.io/vega-lite/docs/field.html) documentation.\\n\\n__Notes:__ 1) Dots (`.`) and brackets (`[` and `]`) can be used to access nested objects (e.g., `\\\"field\\\": \\\"foo.bar\\\"` and `\\\"field\\\": \\\"foo['bar']\\\"`). If field names contain dots or brackets but are not nested, you can use `\\\\\\\\` to escape dots and brackets (e.g., `\\\"a\\\\\\\\.b\\\"` and `\\\"a\\\\\\\\[0\\\\\\\\]\\\"`). See more details about escaping in the [field documentation](https://vega.github.io/vega-lite/docs/field.html). 2) `field` is not required if `aggregate` is `count`.\"\n },\n \"timeUnit\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/TimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/BinnedTimeUnit\"\n },\n {\n \"$ref\": \"#/definitions/TimeUnitParams\"\n }\n ],\n \"description\": \"Time unit (e.g., `year`, `yearmonth`, `month`, `hours`) for a temporal field. or [a temporal field that gets casted as ordinal](https://vega.github.io/vega-lite/docs/type.html#cast).\\n\\n__Default value:__ `undefined` (None)\\n\\n__See also:__ [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html) documentation.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"A title for the field. If `null`, the title will be removed.\\n\\n__Default value:__ derived from the field's name and transformation function (`aggregate`, `bin` and `timeUnit`). If the field has an aggregate function, the function is displayed as part of the title (e.g., `\\\"Sum of Profit\\\"`). If the field is binned or has a time unit applied, the applied function is shown in parentheses (e.g., `\\\"Profit (binned)\\\"`, `\\\"Transaction Date (year-month)\\\"`). Otherwise, the title is simply the field name.\\n\\n__Notes__:\\n\\n1) You can customize the default field title format by providing the [`fieldTitle`](https://vega.github.io/vega-lite/docs/config.html#top-level-config) property in the [config](https://vega.github.io/vega-lite/docs/config.html) or [`fieldTitle` function via the `compile` function's options](https://vega.github.io/vega-lite/usage/compile.html#field-title).\\n\\n2) If both field definition's `title` and axis, header, or legend `title` are defined, axis/header/legend title will be used.\"\n },\n \"type\": {\n \"$ref\": \"#/definitions/StandardType\",\n \"description\": \"The type of measurement (`\\\"quantitative\\\"`, `\\\"temporal\\\"`, `\\\"ordinal\\\"`, or `\\\"nominal\\\"`) for the encoded field or constant value (`datum`). It can also be a `\\\"geojson\\\"` type for encoding ['geoshape'](https://vega.github.io/vega-lite/docs/geoshape.html).\\n\\nVega-Lite automatically infers data types in many cases as discussed below. However, type is required for a field if: (1) the field is not nominal and the field encoding has no specified `aggregate` (except `argmin` and `argmax`), `bin`, scale type, custom `sort` order, nor `timeUnit` or (2) if you wish to use an ordinal scale for a field with `bin` or `timeUnit`.\\n\\n__Default value:__\\n\\n1) For a data `field`, `\\\"nominal\\\"` is the default data type unless the field encoding has `aggregate`, `channel`, `bin`, scale type, `sort`, or `timeUnit` that satisfies the following criteria:\\n- `\\\"quantitative\\\"` is the default type if (1) the encoded field contains `bin` or `aggregate` except `\\\"argmin\\\"` and `\\\"argmax\\\"`, (2) the encoding channel is `latitude` or `longitude` channel or (3) if the specified scale type is [a quantitative scale](https://vega.github.io/vega-lite/docs/scale.html#type).\\n- `\\\"temporal\\\"` is the default type if (1) the encoded field contains `timeUnit` or (2) the specified scale type is a time or utc scale\\n- `\\\"ordinal\\\"` is the default type if (1) the encoded field contains a [custom `sort` order](https://vega.github.io/vega-lite/docs/sort.html#specifying-custom-sort-order), (2) the specified scale type is an ordinal/point/band scale, or (3) the encoding channel is `order`.\\n\\n2) For a constant value in data domain (`datum`):\\n- `\\\"quantitative\\\"` if the datum is a number\\n- `\\\"nominal\\\"` if the datum is a string\\n- `\\\"temporal\\\"` if the datum is [a date time object](https://vega.github.io/vega-lite/docs/datetime.html)\\n\\n__Note:__\\n- Data `type` describes the semantics of the data rather than the primitive data types (number, string, etc.). The same primitive data type can have different types of measurement. For example, numeric data can represent quantitative, ordinal, or nominal data.\\n- Data values for a temporal field can be either a date-time string (e.g., `\\\"2015-03-07 12:32:17\\\"`, `\\\"17:01\\\"`, `\\\"2015-03-16\\\"`. `\\\"2015\\\"`) or a timestamp number (e.g., `1552199579097`).\\n- When using with [`bin`](https://vega.github.io/vega-lite/docs/bin.html), the `type` property can be either `\\\"quantitative\\\"` (for using a linear bin scale) or [`\\\"ordinal\\\"` (for using an ordinal bin scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`timeUnit`](https://vega.github.io/vega-lite/docs/timeunit.html), the `type` property can be either `\\\"temporal\\\"` (default, for using a temporal scale) or [`\\\"ordinal\\\"` (for using an ordinal scale)](https://vega.github.io/vega-lite/docs/type.html#cast-bin).\\n- When using with [`aggregate`](https://vega.github.io/vega-lite/docs/aggregate.html), the `type` property refers to the post-aggregation data type. For example, we can calculate count `distinct` of a categorical field `\\\"cat\\\"` using `{\\\"aggregate\\\": \\\"distinct\\\", \\\"field\\\": \\\"cat\\\"}`. The `\\\"type\\\"` of the aggregate output is `\\\"quantitative\\\"`.\\n- Secondary channels (e.g., `x2`, `y2`, `xError`, `yError`) do not have `type` as they must have exactly the same type as their primary channels (e.g., `x`, `y`).\\n\\n__See also:__ [`type`](https://vega.github.io/vega-lite/docs/type.html) documentation.\"\n }\n },\n \"type\": \"object\"\n },\n \"URI\": {\n \"format\": \"uri-reference\",\n \"type\": \"string\"\n },\n \"UnitSpec\": {\n \"$ref\": \"#/definitions/GenericUnitSpec\",\n \"description\": \"A unit specification, which can contain either [primitive marks or composite marks](https://vega.github.io/vega-lite/docs/mark.html#types).\"\n },\n \"UnitSpecWithFrame\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"data\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Data\"\n },\n {\n \"type\": \"null\"\n }\n ],\n \"description\": \"An object describing the data source. Set to `null` to ignore the parent's data source. If no data is set, it is derived from the parent.\"\n },\n \"description\": {\n \"description\": \"Description of this mark for commenting purpose.\",\n \"type\": \"string\"\n },\n \"encoding\": {\n \"$ref\": \"#/definitions/Encoding\",\n \"description\": \"A key-value mapping between encoding channels and definition of fields.\"\n },\n \"height\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"container\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Step\"\n }\n ],\n \"description\": \"The height of a visualization.\\n\\n- For a plot with a continuous y-field, height should be a number.\\n- For a plot with either a discrete y-field or no y-field, height can be either a number indicating a fixed height or an object in the form of `{step: number}` defining the height per discrete step. (No y-field is equivalent to having one discrete step.)\\n- To enable responsive sizing on height, it should be set to `\\\"container\\\"`.\\n\\n__Default value:__ Based on `config.view.continuousHeight` for a plot with a continuous y-field and `config.view.discreteHeight` otherwise.\\n\\n__Note:__ For plots with [`row` and `column` channels](https://vega.github.io/vega-lite/docs/encoding.html#facet), this represents the height of a single view and the `\\\"container\\\"` option cannot be used.\\n\\n__See also:__ [`height`](https://vega.github.io/vega-lite/docs/size.html) documentation.\"\n },\n \"mark\": {\n \"$ref\": \"#/definitions/AnyMark\",\n \"description\": \"A string describing the mark type (one of `\\\"bar\\\"`, `\\\"circle\\\"`, `\\\"square\\\"`, `\\\"tick\\\"`, `\\\"line\\\"`, `\\\"area\\\"`, `\\\"point\\\"`, `\\\"rule\\\"`, `\\\"geoshape\\\"`, and `\\\"text\\\"`) or a [mark definition object](https://vega.github.io/vega-lite/docs/mark.html#mark-def).\"\n },\n \"name\": {\n \"description\": \"Name of the visualization for later reference.\",\n \"type\": \"string\"\n },\n \"params\": {\n \"description\": \"An array of parameters that may either be simple variables, or more complex selections that map user input to data queries.\",\n \"items\": {\n \"$ref\": \"#/definitions/SelectionParameter\"\n },\n \"type\": \"array\"\n },\n \"projection\": {\n \"$ref\": \"#/definitions/Projection\",\n \"description\": \"An object defining properties of geographic projection, which will be applied to `shape` path for `\\\"geoshape\\\"` marks and to `latitude` and `\\\"longitude\\\"` channels for other marks.\"\n },\n \"title\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"$ref\": \"#/definitions/TitleParams\"\n }\n ],\n \"description\": \"Title for the plot.\"\n },\n \"transform\": {\n \"description\": \"An array of data transformations such as filter and new field calculation.\",\n \"items\": {\n \"$ref\": \"#/definitions/Transform\"\n },\n \"type\": \"array\"\n },\n \"view\": {\n \"$ref\": \"#/definitions/ViewBackground\",\n \"description\": \"An object defining the view background's fill and stroke.\\n\\n__Default value:__ none (transparent)\"\n },\n \"width\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"container\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/Step\"\n }\n ],\n \"description\": \"The width of a visualization.\\n\\n- For a plot with a continuous x-field, width should be a number.\\n- For a plot with either a discrete x-field or no x-field, width can be either a number indicating a fixed width or an object in the form of `{step: number}` defining the width per discrete step. (No x-field is equivalent to having one discrete step.)\\n- To enable responsive sizing on width, it should be set to `\\\"container\\\"`.\\n\\n__Default value:__ Based on `config.view.continuousWidth` for a plot with a continuous x-field and `config.view.discreteWidth` otherwise.\\n\\n__Note:__ For plots with [`row` and `column` channels](https://vega.github.io/vega-lite/docs/encoding.html#facet), this represents the width of a single view and the `\\\"container\\\"` option cannot be used.\\n\\n__See also:__ [`width`](https://vega.github.io/vega-lite/docs/size.html) documentation.\"\n }\n },\n \"required\": [\n \"mark\"\n ],\n \"type\": \"object\"\n },\n \"UrlData\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"format\": {\n \"$ref\": \"#/definitions/DataFormat\",\n \"description\": \"An object that specifies the format for parsing the data.\"\n },\n \"name\": {\n \"description\": \"Provide a placeholder name and bind data at runtime.\",\n \"type\": \"string\"\n },\n \"url\": {\n \"description\": \"An URL from which to load the data set. Use the `format.type` property to ensure the loaded data is correctly parsed.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"url\"\n ],\n \"type\": \"object\"\n },\n \"UtcMultiTimeUnit\": {\n \"enum\": [\n \"utcyearquarter\",\n \"utcyearquartermonth\",\n \"utcyearmonth\",\n \"utcyearmonthdate\",\n \"utcyearmonthdatehours\",\n \"utcyearmonthdatehoursminutes\",\n \"utcyearmonthdatehoursminutesseconds\",\n \"utcyearweek\",\n \"utcyearweekday\",\n \"utcyearweekdayhours\",\n \"utcyearweekdayhoursminutes\",\n \"utcyearweekdayhoursminutesseconds\",\n \"utcyeardayofyear\",\n \"utcquartermonth\",\n \"utcmonthdate\",\n \"utcmonthdatehours\",\n \"utcmonthdatehoursminutes\",\n \"utcmonthdatehoursminutesseconds\",\n \"utcweekday\",\n \"utcweeksdayhours\",\n \"utcweekdayhoursminutes\",\n \"utcweekdayhoursminutesseconds\",\n \"utcdayhours\",\n \"utcdayhoursminutes\",\n \"utcdayhoursminutesseconds\",\n \"utchoursminutes\",\n \"utchoursminutesseconds\",\n \"utcminutesseconds\",\n \"utcsecondsmilliseconds\"\n ],\n \"type\": \"string\"\n },\n \"UtcSingleTimeUnit\": {\n \"enum\": [\n \"utcyear\",\n \"utcquarter\",\n \"utcmonth\",\n \"utcweek\",\n \"utcday\",\n \"utcdayofyear\",\n \"utcdate\",\n \"utchours\",\n \"utcminutes\",\n \"utcseconds\",\n \"utcmilliseconds\"\n ],\n \"type\": \"string\"\n },\n \"ValueDef<(number|\\\"width\\\"|\\\"height\\\"|ExprRef)>\": {\n \"additionalProperties\": false,\n \"description\": \"Definition object for a constant value (primitive value or gradient definition) of an encoding channel.\",\n \"properties\": {\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"const\": \"width\",\n \"type\": \"string\"\n },\n {\n \"const\": \"height\",\n \"type\": \"string\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"required\": [\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"ValueDef\": {\n \"additionalProperties\": false,\n \"description\": \"Definition object for a constant value (primitive value or gradient definition) of an encoding channel.\",\n \"properties\": {\n \"value\": {\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\",\n \"type\": \"number\"\n }\n },\n \"required\": [\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"ValueDefWithCondition\": {\n \"additionalProperties\": false,\n \"minProperties\": 1,\n \"properties\": {\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalMarkPropFieldOrDatumDef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Gradient|string|null|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Gradient|string|null|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A field definition or one or more value definition(s) with a parameter predicate.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Gradient\"\n },\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"ValueDefWithCondition\": {\n \"additionalProperties\": false,\n \"minProperties\": 1,\n \"properties\": {\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalMarkPropFieldOrDatumDef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|null|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|null|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A field definition or one or more value definition(s) with a parameter predicate.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"ValueDefWithCondition\": {\n \"additionalProperties\": false,\n \"minProperties\": 1,\n \"properties\": {\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalMarkPropFieldOrDatumDef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A field definition or one or more value definition(s) with a parameter predicate.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"ValueDefWithCondition\": {\n \"additionalProperties\": false,\n \"minProperties\": 1,\n \"properties\": {\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalMarkPropFieldOrDatumDef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number[]|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(number[]|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A field definition or one or more value definition(s) with a parameter predicate.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"ValueDefWithCondition,(string|null)>\": {\n \"additionalProperties\": false,\n \"minProperties\": 1,\n \"properties\": {\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalMarkPropFieldOrDatumDef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|null|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(string|null|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A field definition or one or more value definition(s) with a parameter predicate.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"ValueDefWithCondition\": {\n \"additionalProperties\": false,\n \"minProperties\": 1,\n \"properties\": {\n \"condition\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/ConditionalStringFieldDef\"\n },\n {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Text|ExprRef)>\"\n },\n {\n \"items\": {\n \"$ref\": \"#/definitions/ConditionalValueDef<(Text|ExprRef)>\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A field definition or one or more value definition(s) with a parameter predicate.\"\n },\n \"value\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Text\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"A constant value in visual domain (e.g., `\\\"red\\\"` / `\\\"#0099ff\\\"` / [gradient definition](https://vega.github.io/vega-lite/docs/types.html#gradient) for color, values between `0` to `1` for opacity).\"\n }\n },\n \"type\": \"object\"\n },\n \"VariableParameter\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"bind\": {\n \"$ref\": \"#/definitions/Binding\",\n \"description\": \"Binds the parameter to an external input element such as a slider, selection list or radio button group.\"\n },\n \"expr\": {\n \"$ref\": \"#/definitions/Expr\",\n \"description\": \"An expression for the value of the parameter. This expression may include other parameters, in which case the parameter will automatically update in response to upstream parameter changes.\"\n },\n \"name\": {\n \"$ref\": \"#/definitions/ParameterName\",\n \"description\": \"A unique name for the variable parameter. Parameter names should be valid JavaScript identifiers: they should contain only alphanumeric characters (or \\\"$\\\", or \\\"_\\\") and may not start with a digit. Reserved keywords that may not be used as parameter names are \\\"datum\\\", \\\"event\\\", \\\"item\\\", and \\\"parent\\\".\"\n },\n \"value\": {\n \"description\": \"The [initial value](http://vega.github.io/vega-lite/docs/value.html) of the parameter.\\n\\n__Default value:__ `undefined`\"\n }\n },\n \"required\": [\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"Vector10\": {\n \"items\": {\n \"type\": \"string\"\n },\n \"maxItems\": 10,\n \"minItems\": 10,\n \"type\": \"array\"\n },\n \"Vector12\": {\n \"items\": {\n \"type\": \"string\"\n },\n \"maxItems\": 12,\n \"minItems\": 12,\n \"type\": \"array\"\n },\n \"Vector2\": {\n \"items\": {\n \"$ref\": \"#/definitions/DateTime\"\n },\n \"maxItems\": 2,\n \"minItems\": 2,\n \"type\": \"array\"\n },\n \"Vector2>\": {\n \"items\": {\n \"$ref\": \"#/definitions/Vector2\"\n },\n \"maxItems\": 2,\n \"minItems\": 2,\n \"type\": \"array\"\n },\n \"Vector2\": {\n \"items\": {\n \"type\": \"boolean\"\n },\n \"maxItems\": 2,\n \"minItems\": 2,\n \"type\": \"array\"\n },\n \"Vector2\": {\n \"items\": {\n \"type\": \"number\"\n },\n \"maxItems\": 2,\n \"minItems\": 2,\n \"type\": \"array\"\n },\n \"Vector2\": {\n \"items\": {\n \"type\": \"string\"\n },\n \"maxItems\": 2,\n \"minItems\": 2,\n \"type\": \"array\"\n },\n \"Vector3\": {\n \"items\": {\n \"type\": \"number\"\n },\n \"maxItems\": 3,\n \"minItems\": 3,\n \"type\": \"array\"\n },\n \"Vector7\": {\n \"items\": {\n \"type\": \"string\"\n },\n \"maxItems\": 7,\n \"minItems\": 7,\n \"type\": \"array\"\n },\n \"ViewBackground\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"cornerRadius\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles or arcs' corners.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cursor\": {\n \"$ref\": \"#/definitions/Cursor\",\n \"description\": \"The mouse cursor used over the view. Any valid [CSS cursor type](https://developer.mozilla.org/en-US/docs/Web/CSS/cursor#Values) can be used.\"\n },\n \"fill\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The fill color.\\n\\n__Default value:__ `undefined`\"\n },\n \"fillOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"The fill opacity (value between [0,1]).\\n\\n__Default value:__ `1`\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"opacity\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The overall opacity (value between [0,1]).\\n\\n__Default value:__ `0.7` for non-aggregate plots with `point`, `tick`, `circle`, or `square` marks or layered `bar` charts and `1` otherwise.\",\n \"maximum\": 1,\n \"minimum\": 0\n },\n \"stroke\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The stroke color.\\n\\n__Default value:__ `\\\"#ddd\\\"`\"\n },\n \"strokeCap\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeCap\",\n \"description\": \"The stroke cap for line ending style. One of `\\\"butt\\\"`, `\\\"round\\\"`, or `\\\"square\\\"`.\\n\\n__Default value:__ `\\\"butt\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeDash\": {\n \"anyOf\": [\n {\n \"description\": \"An array of alternating stroke, space lengths for creating dashed or dotted lines.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeDashOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The offset (in pixels) into which to begin drawing with the stroke dash array.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeJoin\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeJoin\",\n \"description\": \"The stroke line join method. One of `\\\"miter\\\"`, `\\\"round\\\"` or `\\\"bevel\\\"`.\\n\\n__Default value:__ `\\\"miter\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeMiterLimit\": {\n \"anyOf\": [\n {\n \"description\": \"The miter limit at which to bevel a line join.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"The stroke opacity (value between [0,1]).\\n\\n__Default value:__ `1`\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeWidth\": {\n \"anyOf\": [\n {\n \"description\": \"The stroke width, in pixels.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"style\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n ],\n \"description\": \"A string or array of strings indicating the name of custom styles to apply to the view background. A style is a named collection of mark property defaults defined within the [style configuration](https://vega.github.io/vega-lite/docs/mark.html#style-config). If style is an array, later styles will override earlier styles.\\n\\n__Default value:__ `\\\"cell\\\"` __Note:__ Any specified view background properties will augment the default style.\"\n }\n },\n \"type\": \"object\"\n },\n \"ViewConfig\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"clip\": {\n \"description\": \"Whether the view should be clipped.\",\n \"type\": \"boolean\"\n },\n \"continuousHeight\": {\n \"description\": \"The default height when the plot has a continuous y-field for x or latitude, or has arc marks.\\n\\n__Default value:__ `200`\",\n \"type\": \"number\"\n },\n \"continuousWidth\": {\n \"description\": \"The default width when the plot has a continuous field for x or longitude, or has arc marks.\\n\\n__Default value:__ `200`\",\n \"type\": \"number\"\n },\n \"cornerRadius\": {\n \"anyOf\": [\n {\n \"description\": \"The radius in pixels of rounded rectangles or arcs' corners.\\n\\n__Default value:__ `0`\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"cursor\": {\n \"$ref\": \"#/definitions/Cursor\",\n \"description\": \"The mouse cursor used over the view. Any valid [CSS cursor type](https://developer.mozilla.org/en-US/docs/Web/CSS/cursor#Values) can be used.\"\n },\n \"discreteHeight\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"step\": {\n \"type\": \"number\"\n }\n },\n \"required\": [\n \"step\"\n ],\n \"type\": \"object\"\n }\n ],\n \"description\": \"The default height when the plot has non arc marks and either a discrete y-field or no y-field. The height can be either a number indicating a fixed height or an object in the form of `{step: number}` defining the height per discrete step.\\n\\n__Default value:__ a step size based on `config.view.step`.\"\n },\n \"discreteWidth\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"additionalProperties\": false,\n \"properties\": {\n \"step\": {\n \"type\": \"number\"\n }\n },\n \"required\": [\n \"step\"\n ],\n \"type\": \"object\"\n }\n ],\n \"description\": \"The default width when the plot has non-arc marks and either a discrete x-field or no x-field. The width can be either a number indicating a fixed width or an object in the form of `{step: number}` defining the width per discrete step.\\n\\n__Default value:__ a step size based on `config.view.step`.\"\n },\n \"fill\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The fill color.\\n\\n__Default value:__ `undefined`\"\n },\n \"fillOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"The fill opacity (value between [0,1]).\\n\\n__Default value:__ `1`\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"opacity\": {\n \"anyOf\": [\n {\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The overall opacity (value between [0,1]).\\n\\n__Default value:__ `0.7` for non-aggregate plots with `point`, `tick`, `circle`, or `square` marks or layered `bar` charts and `1` otherwise.\",\n \"maximum\": 1,\n \"minimum\": 0\n },\n \"step\": {\n \"description\": \"Default step size for x-/y- discrete fields.\",\n \"type\": \"number\"\n },\n \"stroke\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/Color\"\n },\n {\n \"type\": \"null\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ],\n \"description\": \"The stroke color.\\n\\n__Default value:__ `\\\"#ddd\\\"`\"\n },\n \"strokeCap\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeCap\",\n \"description\": \"The stroke cap for line ending style. One of `\\\"butt\\\"`, `\\\"round\\\"`, or `\\\"square\\\"`.\\n\\n__Default value:__ `\\\"butt\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeDash\": {\n \"anyOf\": [\n {\n \"description\": \"An array of alternating stroke, space lengths for creating dashed or dotted lines.\",\n \"items\": {\n \"type\": \"number\"\n },\n \"type\": \"array\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeDashOffset\": {\n \"anyOf\": [\n {\n \"description\": \"The offset (in pixels) into which to begin drawing with the stroke dash array.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeJoin\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/StrokeJoin\",\n \"description\": \"The stroke line join method. One of `\\\"miter\\\"`, `\\\"round\\\"` or `\\\"bevel\\\"`.\\n\\n__Default value:__ `\\\"miter\\\"`\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeMiterLimit\": {\n \"anyOf\": [\n {\n \"description\": \"The miter limit at which to bevel a line join.\",\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeOpacity\": {\n \"anyOf\": [\n {\n \"description\": \"The stroke opacity (value between [0,1]).\\n\\n__Default value:__ `1`\",\n \"maximum\": 1,\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n },\n \"strokeWidth\": {\n \"anyOf\": [\n {\n \"description\": \"The stroke width, in pixels.\",\n \"minimum\": 0,\n \"type\": \"number\"\n },\n {\n \"$ref\": \"#/definitions/ExprRef\"\n }\n ]\n }\n },\n \"type\": \"object\"\n },\n \"WindowEventType\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/EventType\"\n },\n {\n \"type\": \"string\"\n }\n ]\n },\n \"WindowFieldDef\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"as\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"The output name for the window operation.\"\n },\n \"field\": {\n \"$ref\": \"#/definitions/FieldName\",\n \"description\": \"The data field for which to compute the aggregate or window function. This can be omitted for window functions that do not operate over a field such as `\\\"count\\\"`, `\\\"rank\\\"`, `\\\"dense_rank\\\"`.\"\n },\n \"op\": {\n \"anyOf\": [\n {\n \"$ref\": \"#/definitions/AggregateOp\"\n },\n {\n \"$ref\": \"#/definitions/WindowOnlyOp\"\n }\n ],\n \"description\": \"The window or aggregation operation to apply within a window (e.g., `\\\"rank\\\"`, `\\\"lead\\\"`, `\\\"sum\\\"`, `\\\"average\\\"` or `\\\"count\\\"`). See the list of all supported operations [here](https://vega.github.io/vega-lite/docs/window.html#ops).\"\n },\n \"param\": {\n \"description\": \"Parameter values for the window functions. Parameter values can be omitted for operations that do not accept a parameter.\\n\\nSee the list of all supported operations and their parameters [here](https://vega.github.io/vega-lite/docs/transforms/window.html).\",\n \"type\": \"number\"\n }\n },\n \"required\": [\n \"op\",\n \"as\"\n ],\n \"type\": \"object\"\n },\n \"WindowOnlyOp\": {\n \"enum\": [\n \"row_number\",\n \"rank\",\n \"dense_rank\",\n \"percent_rank\",\n \"cume_dist\",\n \"ntile\",\n \"lag\",\n \"lead\",\n \"first_value\",\n \"last_value\",\n \"nth_value\"\n ],\n \"type\": \"string\"\n },\n \"WindowTransform\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"frame\": {\n \"description\": \"A frame specification as a two-element array indicating how the sliding window should proceed. The array entries should either be a number indicating the offset from the current data object, or null to indicate unbounded rows preceding or following the current data object. The default value is `[null, 0]`, indicating that the sliding window includes the current object and all preceding objects. The value `[-5, 5]` indicates that the window should include five objects preceding and five objects following the current object. Finally, `[null, null]` indicates that the window frame should always include all data objects. If you this frame and want to assign the same value to add objects, you can use the simpler [join aggregate transform](https://vega.github.io/vega-lite/docs/joinaggregate.html). The only operators affected are the aggregation operations and the `first_value`, `last_value`, and `nth_value` window operations. The other window operations are not affected by this.\\n\\n__Default value:__: `[null, 0]` (includes the current object and all preceding objects)\",\n \"items\": {\n \"type\": [\n \"null\",\n \"number\"\n ]\n },\n \"type\": \"array\"\n },\n \"groupby\": {\n \"description\": \"The data fields for partitioning the data objects into separate windows. If unspecified, all data points will be in a single window.\",\n \"items\": {\n \"$ref\": \"#/definitions/FieldName\"\n },\n \"type\": \"array\"\n },\n \"ignorePeers\": {\n \"description\": \"Indicates if the sliding window frame should ignore peer values (data that are considered identical by the sort criteria). The default is false, causing the window frame to expand to include all peer values. If set to true, the window frame will be defined by offset values only. This setting only affects those operations that depend on the window frame, namely aggregation operations and the first_value, last_value, and nth_value window operations.\\n\\n__Default value:__ `false`\",\n \"type\": \"boolean\"\n },\n \"sort\": {\n \"description\": \"A sort field definition for sorting data objects within a window. If two data objects are considered equal by the comparator, they are considered \\\"peer\\\" values of equal rank. If sort is not specified, the order is undefined: data objects are processed in the order they are observed and none are considered peers (the ignorePeers parameter is ignored and treated as if set to `true`).\",\n \"items\": {\n \"$ref\": \"#/definitions/SortField\"\n },\n \"type\": \"array\"\n },\n \"window\": {\n \"description\": \"The definition of the fields in the window, and what calculations to use.\",\n \"items\": {\n \"$ref\": \"#/definitions/WindowFieldDef\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"window\"\n ],\n \"type\": \"object\"\n }\n }\n}\n" }, { "path": "docs/_static/js/vega-embed@5.js", "content": "!function(e,t){\"object\"==typeof exports&&\"undefined\"!=typeof module?module.exports=t(require(\"vega\"),require(\"vega-lite\")):\"function\"==typeof define&&define.amd?define([\"vega\",\"vega-lite\"],t):(e=e||self).vegaEmbed=t(e.vega,e.vegaLite)}(this,(function(e,t){\"use strict\";var n=\"http://www.w3.org/1999/xhtml\",r={svg:\"http://www.w3.org/2000/svg\",xhtml:n,xlink:\"http://www.w3.org/1999/xlink\",xml:\"http://www.w3.org/XML/1998/namespace\",xmlns:\"http://www.w3.org/2000/xmlns/\"};function i(e){var t=e+=\"\",n=t.indexOf(\":\");return n>=0&&\"xmlns\"!==(t=e.slice(0,n))&&(e=e.slice(n+1)),r.hasOwnProperty(t)?{space:r[t],local:e}:e}function o(e){return function(){var t=this.ownerDocument,r=this.namespaceURI;return r===n&&t.documentElement.namespaceURI===n?t.createElement(e):t.createElementNS(r,e)}}function a(e){return function(){return this.ownerDocument.createElementNS(e.space,e.local)}}function s(e){var t=i(e);return(t.local?a:o)(t)}function l(){}function c(e){return null==e?l:function(){return this.querySelector(e)}}function u(){return[]}function f(e){return new Array(e.length)}function p(e,t){this.ownerDocument=e.ownerDocument,this.namespaceURI=e.namespaceURI,this._next=null,this._parent=e,this.__data__=t}p.prototype={constructor:p,appendChild:function(e){return this._parent.insertBefore(e,this._next)},insertBefore:function(e,t){return this._parent.insertBefore(e,t)},querySelector:function(e){return this._parent.querySelector(e)},querySelectorAll:function(e){return this._parent.querySelectorAll(e)}};var h=\"$\";function d(e,t,n,r,i,o){for(var a,s=0,l=t.length,c=o.length;st?1:e>=t?0:NaN}function v(e){return function(){this.removeAttribute(e)}}function E(e){return function(){this.removeAttributeNS(e.space,e.local)}}function y(e,t){return function(){this.setAttribute(e,t)}}function b(e,t){return function(){this.setAttributeNS(e.space,e.local,t)}}function I(e,t){return function(){var n=t.apply(this,arguments);null==n?this.removeAttribute(e):this.setAttribute(e,n)}}function O(e,t){return function(){var n=t.apply(this,arguments);null==n?this.removeAttributeNS(e.space,e.local):this.setAttributeNS(e.space,e.local,n)}}function R(e){return e.ownerDocument&&e.ownerDocument.defaultView||e.document&&e||e.defaultView}function w(e){return function(){this.style.removeProperty(e)}}function N(e,t,n){return function(){this.style.setProperty(e,t,n)}}function A(e,t,n){return function(){var r=t.apply(this,arguments);null==r?this.style.removeProperty(e):this.style.setProperty(e,r,n)}}function S(e){return function(){delete this[e]}}function L(e,t){return function(){this[e]=t}}function T(e,t){return function(){var n=t.apply(this,arguments);null==n?delete this[e]:this[e]=n}}function x(e){return e.trim().split(/^|\\s+/)}function _(e){return e.classList||new C(e)}function C(e){this._node=e,this._names=x(e.getAttribute(\"class\")||\"\")}function P(e,t){for(var n=_(e),r=-1,i=t.length;++r=0&&(this._names.splice(t,1),this._node.setAttribute(\"class\",this._names.join(\" \")))},contains:function(e){return this._names.indexOf(e)>=0}};var J={},Z=null;\"undefined\"!=typeof document&&(\"onmouseenter\"in document.documentElement||(J={mouseenter:\"mouseover\",mouseleave:\"mouseout\"}));function K(e,t,n){return e=Q(e,t,n),function(t){var n=t.relatedTarget;n&&(n===this||8&n.compareDocumentPosition(this))||e.call(this,t)}}function Q(e,t,n){return function(r){var i=Z;Z=r;try{e.call(this,this.__data__,t,n)}finally{Z=i}}}function ee(e){return function(){var t=this.__on;if(t){for(var n,r=0,i=-1,o=t.length;r=R&&(R=O+1);!(I=y[R])&&++R=0;)(r=i[o])&&(a&&4^r.compareDocumentPosition(a)&&a.parentNode.insertBefore(r,a),a=r);return this},sort:function(e){function t(t,n){return t&&n?e(t.__data__,n.__data__):!t-!n}e||(e=m);for(var n=this._groups,r=n.length,i=new Array(r),o=0;o1?this.each((null==t?w:\"function\"==typeof t?A:N)(e,t,null==n?\"\":n)):function(e,t){return e.style.getPropertyValue(t)||R(e).getComputedStyle(e,null).getPropertyValue(t)}(this.node(),e)},property:function(e,t){return arguments.length>1?this.each((null==t?S:\"function\"==typeof t?T:L)(e,t)):this.node()[e]},classed:function(e,t){var n=x(e+\"\");if(arguments.length<2){for(var r=_(this.node()),i=-1,o=n.length;++i=0&&(t=e.slice(n+1),e=e.slice(0,n)),{type:e,name:t}}))}(e+\"\"),a=o.length;if(!(arguments.length<2)){for(s=t?te:ee,null==n&&(n=!1),r=0;r0)return[m,n+c.join(\",\\n\"+d),s].join(\"\\n\"+o)}return v}(e,\"\",0)};function be(e,t){return e(t={exports:{}},t.exports),t.exports}var Ie,Oe=be((function(e,t){var n;t=e.exports=p,n=\"object\"==typeof process&&process.env&&process.env.NODE_DEBUG&&/\\bsemver\\b/i.test(process.env.NODE_DEBUG)?function(){var e=Array.prototype.slice.call(arguments,0);e.unshift(\"SEMVER\"),console.log.apply(console,e)}:function(){},t.SEMVER_SPEC_VERSION=\"2.0.0\";var r=256,i=Number.MAX_SAFE_INTEGER||9007199254740991,o=t.re=[],a=t.src=[],s=t.tokens={},l=0;function c(e){s[e]=l++}c(\"NUMERICIDENTIFIER\"),a[s.NUMERICIDENTIFIER]=\"0|[1-9]\\\\d*\",c(\"NUMERICIDENTIFIERLOOSE\"),a[s.NUMERICIDENTIFIERLOOSE]=\"[0-9]+\",c(\"NONNUMERICIDENTIFIER\"),a[s.NONNUMERICIDENTIFIER]=\"\\\\d*[a-zA-Z-][a-zA-Z0-9-]*\",c(\"MAINVERSION\"),a[s.MAINVERSION]=\"(\"+a[s.NUMERICIDENTIFIER]+\")\\\\.(\"+a[s.NUMERICIDENTIFIER]+\")\\\\.(\"+a[s.NUMERICIDENTIFIER]+\")\",c(\"MAINVERSIONLOOSE\"),a[s.MAINVERSIONLOOSE]=\"(\"+a[s.NUMERICIDENTIFIERLOOSE]+\")\\\\.(\"+a[s.NUMERICIDENTIFIERLOOSE]+\")\\\\.(\"+a[s.NUMERICIDENTIFIERLOOSE]+\")\",c(\"PRERELEASEIDENTIFIER\"),a[s.PRERELEASEIDENTIFIER]=\"(?:\"+a[s.NUMERICIDENTIFIER]+\"|\"+a[s.NONNUMERICIDENTIFIER]+\")\",c(\"PRERELEASEIDENTIFIERLOOSE\"),a[s.PRERELEASEIDENTIFIERLOOSE]=\"(?:\"+a[s.NUMERICIDENTIFIERLOOSE]+\"|\"+a[s.NONNUMERICIDENTIFIER]+\")\",c(\"PRERELEASE\"),a[s.PRERELEASE]=\"(?:-(\"+a[s.PRERELEASEIDENTIFIER]+\"(?:\\\\.\"+a[s.PRERELEASEIDENTIFIER]+\")*))\",c(\"PRERELEASELOOSE\"),a[s.PRERELEASELOOSE]=\"(?:-?(\"+a[s.PRERELEASEIDENTIFIERLOOSE]+\"(?:\\\\.\"+a[s.PRERELEASEIDENTIFIERLOOSE]+\")*))\",c(\"BUILDIDENTIFIER\"),a[s.BUILDIDENTIFIER]=\"[0-9A-Za-z-]+\",c(\"BUILD\"),a[s.BUILD]=\"(?:\\\\+(\"+a[s.BUILDIDENTIFIER]+\"(?:\\\\.\"+a[s.BUILDIDENTIFIER]+\")*))\",c(\"FULL\"),c(\"FULLPLAIN\"),a[s.FULLPLAIN]=\"v?\"+a[s.MAINVERSION]+a[s.PRERELEASE]+\"?\"+a[s.BUILD]+\"?\",a[s.FULL]=\"^\"+a[s.FULLPLAIN]+\"$\",c(\"LOOSEPLAIN\"),a[s.LOOSEPLAIN]=\"[v=\\\\s]*\"+a[s.MAINVERSIONLOOSE]+a[s.PRERELEASELOOSE]+\"?\"+a[s.BUILD]+\"?\",c(\"LOOSE\"),a[s.LOOSE]=\"^\"+a[s.LOOSEPLAIN]+\"$\",c(\"GTLT\"),a[s.GTLT]=\"((?:<|>)?=?)\",c(\"XRANGEIDENTIFIERLOOSE\"),a[s.XRANGEIDENTIFIERLOOSE]=a[s.NUMERICIDENTIFIERLOOSE]+\"|x|X|\\\\*\",c(\"XRANGEIDENTIFIER\"),a[s.XRANGEIDENTIFIER]=a[s.NUMERICIDENTIFIER]+\"|x|X|\\\\*\",c(\"XRANGEPLAIN\"),a[s.XRANGEPLAIN]=\"[v=\\\\s]*(\"+a[s.XRANGEIDENTIFIER]+\")(?:\\\\.(\"+a[s.XRANGEIDENTIFIER]+\")(?:\\\\.(\"+a[s.XRANGEIDENTIFIER]+\")(?:\"+a[s.PRERELEASE]+\")?\"+a[s.BUILD]+\"?)?)?\",c(\"XRANGEPLAINLOOSE\"),a[s.XRANGEPLAINLOOSE]=\"[v=\\\\s]*(\"+a[s.XRANGEIDENTIFIERLOOSE]+\")(?:\\\\.(\"+a[s.XRANGEIDENTIFIERLOOSE]+\")(?:\\\\.(\"+a[s.XRANGEIDENTIFIERLOOSE]+\")(?:\"+a[s.PRERELEASELOOSE]+\")?\"+a[s.BUILD]+\"?)?)?\",c(\"XRANGE\"),a[s.XRANGE]=\"^\"+a[s.GTLT]+\"\\\\s*\"+a[s.XRANGEPLAIN]+\"$\",c(\"XRANGELOOSE\"),a[s.XRANGELOOSE]=\"^\"+a[s.GTLT]+\"\\\\s*\"+a[s.XRANGEPLAINLOOSE]+\"$\",c(\"COERCE\"),a[s.COERCE]=\"(^|[^\\\\d])(\\\\d{1,16})(?:\\\\.(\\\\d{1,16}))?(?:\\\\.(\\\\d{1,16}))?(?:$|[^\\\\d])\",c(\"COERCERTL\"),o[s.COERCERTL]=new RegExp(a[s.COERCE],\"g\"),c(\"LONETILDE\"),a[s.LONETILDE]=\"(?:~>?)\",c(\"TILDETRIM\"),a[s.TILDETRIM]=\"(\\\\s*)\"+a[s.LONETILDE]+\"\\\\s+\",o[s.TILDETRIM]=new RegExp(a[s.TILDETRIM],\"g\");c(\"TILDE\"),a[s.TILDE]=\"^\"+a[s.LONETILDE]+a[s.XRANGEPLAIN]+\"$\",c(\"TILDELOOSE\"),a[s.TILDELOOSE]=\"^\"+a[s.LONETILDE]+a[s.XRANGEPLAINLOOSE]+\"$\",c(\"LONECARET\"),a[s.LONECARET]=\"(?:\\\\^)\",c(\"CARETTRIM\"),a[s.CARETTRIM]=\"(\\\\s*)\"+a[s.LONECARET]+\"\\\\s+\",o[s.CARETTRIM]=new RegExp(a[s.CARETTRIM],\"g\");c(\"CARET\"),a[s.CARET]=\"^\"+a[s.LONECARET]+a[s.XRANGEPLAIN]+\"$\",c(\"CARETLOOSE\"),a[s.CARETLOOSE]=\"^\"+a[s.LONECARET]+a[s.XRANGEPLAINLOOSE]+\"$\",c(\"COMPARATORLOOSE\"),a[s.COMPARATORLOOSE]=\"^\"+a[s.GTLT]+\"\\\\s*(\"+a[s.LOOSEPLAIN]+\")$|^$\",c(\"COMPARATOR\"),a[s.COMPARATOR]=\"^\"+a[s.GTLT]+\"\\\\s*(\"+a[s.FULLPLAIN]+\")$|^$\",c(\"COMPARATORTRIM\"),a[s.COMPARATORTRIM]=\"(\\\\s*)\"+a[s.GTLT]+\"\\\\s*(\"+a[s.LOOSEPLAIN]+\"|\"+a[s.XRANGEPLAIN]+\")\",o[s.COMPARATORTRIM]=new RegExp(a[s.COMPARATORTRIM],\"g\");c(\"HYPHENRANGE\"),a[s.HYPHENRANGE]=\"^\\\\s*(\"+a[s.XRANGEPLAIN]+\")\\\\s+-\\\\s+(\"+a[s.XRANGEPLAIN]+\")\\\\s*$\",c(\"HYPHENRANGELOOSE\"),a[s.HYPHENRANGELOOSE]=\"^\\\\s*(\"+a[s.XRANGEPLAINLOOSE]+\")\\\\s+-\\\\s+(\"+a[s.XRANGEPLAINLOOSE]+\")\\\\s*$\",c(\"STAR\"),a[s.STAR]=\"(<|>)?=?\\\\s*\\\\*\";for(var u=0;ur)return null;if(!(t.loose?o[s.LOOSE]:o[s.FULL]).test(e))return null;try{return new p(e,t)}catch(e){return null}}function p(e,t){if(t&&\"object\"==typeof t||(t={loose:!!t,includePrerelease:!1}),e instanceof p){if(e.loose===t.loose)return e;e=e.version}else if(\"string\"!=typeof e)throw new TypeError(\"Invalid Version: \"+e);if(e.length>r)throw new TypeError(\"version is longer than \"+r+\" characters\");if(!(this instanceof p))return new p(e,t);n(\"SemVer\",e,t),this.options=t,this.loose=!!t.loose;var a=e.trim().match(t.loose?o[s.LOOSE]:o[s.FULL]);if(!a)throw new TypeError(\"Invalid Version: \"+e);if(this.raw=e,this.major=+a[1],this.minor=+a[2],this.patch=+a[3],this.major>i||this.major<0)throw new TypeError(\"Invalid major version\");if(this.minor>i||this.minor<0)throw new TypeError(\"Invalid minor version\");if(this.patch>i||this.patch<0)throw new TypeError(\"Invalid patch version\");a[4]?this.prerelease=a[4].split(\".\").map((function(e){if(/^[0-9]+$/.test(e)){var t=+e;if(t>=0&&t=0;)\"number\"==typeof this.prerelease[n]&&(this.prerelease[n]++,n=-2);-1===n&&this.prerelease.push(0)}t&&(this.prerelease[0]===t?isNaN(this.prerelease[1])&&(this.prerelease=[t,0]):this.prerelease=[t,0]);break;default:throw new Error(\"invalid increment argument: \"+e)}return this.format(),this.raw=this.version,this},t.inc=function(e,t,n,r){\"string\"==typeof n&&(r=n,n=void 0);try{return new p(e,n).inc(t,r).version}catch(e){return null}},t.diff=function(e,t){if(E(e,t))return null;var n=f(e),r=f(t),i=\"\";if(n.prerelease.length||r.prerelease.length){i=\"pre\";var o=\"prerelease\"}for(var a in n)if((\"major\"===a||\"minor\"===a||\"patch\"===a)&&n[a]!==r[a])return i+a;return o},t.compareIdentifiers=d;var h=/^[0-9]+$/;function d(e,t){var n=h.test(e),r=h.test(t);return n&&r&&(e=+e,t=+t),e===t?0:n&&!r?-1:r&&!n?1:e0}function v(e,t,n){return g(e,t,n)<0}function E(e,t,n){return 0===g(e,t,n)}function y(e,t,n){return 0!==g(e,t,n)}function b(e,t,n){return g(e,t,n)>=0}function I(e,t,n){return g(e,t,n)<=0}function O(e,t,n,r){switch(t){case\"===\":return\"object\"==typeof e&&(e=e.version),\"object\"==typeof n&&(n=n.version),e===n;case\"!==\":return\"object\"==typeof e&&(e=e.version),\"object\"==typeof n&&(n=n.version),e!==n;case\"\":case\"=\":case\"==\":return E(e,n,r);case\"!=\":return y(e,n,r);case\">\":return m(e,n,r);case\">=\":return b(e,n,r);case\"<\":return v(e,n,r);case\"<=\":return I(e,n,r);default:throw new TypeError(\"Invalid operator: \"+t)}}function R(e,t){if(t&&\"object\"==typeof t||(t={loose:!!t,includePrerelease:!1}),e instanceof R){if(e.loose===!!t.loose)return e;e=e.value}if(!(this instanceof R))return new R(e,t);n(\"comparator\",e,t),this.options=t,this.loose=!!t.loose,this.parse(e),this.semver===w?this.value=\"\":this.value=this.operator+this.semver.version,n(\"comp\",this)}t.rcompareIdentifiers=function(e,t){return d(t,e)},t.major=function(e,t){return new p(e,t).major},t.minor=function(e,t){return new p(e,t).minor},t.patch=function(e,t){return new p(e,t).patch},t.compare=g,t.compareLoose=function(e,t){return g(e,t,!0)},t.compareBuild=function(e,t,n){var r=new p(e,n),i=new p(t,n);return r.compare(i)||r.compareBuild(i)},t.rcompare=function(e,t,n){return g(t,e,n)},t.sort=function(e,n){return e.sort((function(e,r){return t.compareBuild(e,r,n)}))},t.rsort=function(e,n){return e.sort((function(e,r){return t.compareBuild(r,e,n)}))},t.gt=m,t.lt=v,t.eq=E,t.neq=y,t.gte=b,t.lte=I,t.cmp=O,t.Comparator=R;var w={};function N(e,t){if(t&&\"object\"==typeof t||(t={loose:!!t,includePrerelease:!1}),e instanceof N)return e.loose===!!t.loose&&e.includePrerelease===!!t.includePrerelease?e:new N(e.raw,t);if(e instanceof R)return new N(e.value,t);if(!(this instanceof N))return new N(e,t);if(this.options=t,this.loose=!!t.loose,this.includePrerelease=!!t.includePrerelease,this.raw=e,this.set=e.split(/\\s*\\|\\|\\s*/).map((function(e){return this.parseRange(e.trim())}),this).filter((function(e){return e.length})),!this.set.length)throw new TypeError(\"Invalid SemVer Range: \"+e);this.format()}function A(e,t){for(var n=!0,r=e.slice(),i=r.pop();n&&r.length;)n=r.every((function(e){return i.intersects(e,t)})),i=r.pop();return n}function S(e){return!e||\"x\"===e.toLowerCase()||\"*\"===e}function L(e,t,n,r,i,o,a,s,l,c,u,f,p){return((t=S(n)?\"\":S(r)?\">=\"+n+\".0.0\":S(i)?\">=\"+n+\".\"+r+\".0\":\">=\"+t)+\" \"+(s=S(l)?\"\":S(c)?\"<\"+(+l+1)+\".0.0\":S(u)?\"<\"+l+\".\"+(+c+1)+\".0\":f?\"<=\"+l+\".\"+c+\".\"+u+\"-\"+f:\"<=\"+s)).trim()}function T(e,t,r){for(var i=0;i0){var o=e[i].semver;if(o.major===t.major&&o.minor===t.minor&&o.patch===t.patch)return!0}return!1}return!0}function x(e,t,n){try{t=new N(t,n)}catch(e){return!1}return t.test(e)}function _(e,t,n,r){var i,o,a,s,l;switch(e=new p(e,r),t=new N(t,r),n){case\">\":i=m,o=I,a=v,s=\">\",l=\">=\";break;case\"<\":i=v,o=b,a=m,s=\"<\",l=\"<=\";break;default:throw new TypeError('Must provide a hilo val of \"<\" or \">\"')}if(x(e,t,r))return!1;for(var c=0;c=0.0.0\")),f=f||e,h=h||e,i(e.semver,f.semver,r)?f=e:a(e.semver,h.semver,r)&&(h=e)})),f.operator===s||f.operator===l)return!1;if((!h.operator||h.operator===s)&&o(e,h.semver))return!1;if(h.operator===l&&a(e,h.semver))return!1}return!0}R.prototype.parse=function(e){var t=this.options.loose?o[s.COMPARATORLOOSE]:o[s.COMPARATOR],n=e.match(t);if(!n)throw new TypeError(\"Invalid comparator: \"+e);this.operator=void 0!==n[1]?n[1]:\"\",\"=\"===this.operator&&(this.operator=\"\"),n[2]?this.semver=new p(n[2],this.options.loose):this.semver=w},R.prototype.toString=function(){return this.value},R.prototype.test=function(e){if(n(\"Comparator.test\",e,this.options.loose),this.semver===w||e===w)return!0;if(\"string\"==typeof e)try{e=new p(e,this.options)}catch(e){return!1}return O(e,this.operator,this.semver,this.options)},R.prototype.intersects=function(e,t){if(!(e instanceof R))throw new TypeError(\"a Comparator is required\");var n;if(t&&\"object\"==typeof t||(t={loose:!!t,includePrerelease:!1}),\"\"===this.operator)return\"\"===this.value||(n=new N(e.value,t),x(this.value,n,t));if(\"\"===e.operator)return\"\"===e.value||(n=new N(this.value,t),x(e.semver,n,t));var r=!(\">=\"!==this.operator&&\">\"!==this.operator||\">=\"!==e.operator&&\">\"!==e.operator),i=!(\"<=\"!==this.operator&&\"<\"!==this.operator||\"<=\"!==e.operator&&\"<\"!==e.operator),o=this.semver.version===e.semver.version,a=!(\">=\"!==this.operator&&\"<=\"!==this.operator||\">=\"!==e.operator&&\"<=\"!==e.operator),s=O(this.semver,\"<\",e.semver,t)&&(\">=\"===this.operator||\">\"===this.operator)&&(\"<=\"===e.operator||\"<\"===e.operator),l=O(this.semver,\">\",e.semver,t)&&(\"<=\"===this.operator||\"<\"===this.operator)&&(\">=\"===e.operator||\">\"===e.operator);return r||i||o&&a||s||l},t.Range=N,N.prototype.format=function(){return this.range=this.set.map((function(e){return e.join(\" \").trim()})).join(\"||\").trim(),this.range},N.prototype.toString=function(){return this.range},N.prototype.parseRange=function(e){var t=this.options.loose;e=e.trim();var r=t?o[s.HYPHENRANGELOOSE]:o[s.HYPHENRANGE];e=e.replace(r,L),n(\"hyphen replace\",e),e=e.replace(o[s.COMPARATORTRIM],\"$1$2$3\"),n(\"comparator trim\",e,o[s.COMPARATORTRIM]),e=(e=(e=e.replace(o[s.TILDETRIM],\"$1~\")).replace(o[s.CARETTRIM],\"$1^\")).split(/\\s+/).join(\" \");var i=t?o[s.COMPARATORLOOSE]:o[s.COMPARATOR],a=e.split(\" \").map((function(e){return function(e,t){return n(\"comp\",e,t),e=function(e,t){return e.trim().split(/\\s+/).map((function(e){return function(e,t){n(\"caret\",e,t);var r=t.loose?o[s.CARETLOOSE]:o[s.CARET];return e.replace(r,(function(t,r,i,o,a){var s;return n(\"caret\",e,t,r,i,o,a),S(r)?s=\"\":S(i)?s=\">=\"+r+\".0.0 <\"+(+r+1)+\".0.0\":S(o)?s=\"0\"===r?\">=\"+r+\".\"+i+\".0 <\"+r+\".\"+(+i+1)+\".0\":\">=\"+r+\".\"+i+\".0 <\"+(+r+1)+\".0.0\":a?(n(\"replaceCaret pr\",a),s=\"0\"===r?\"0\"===i?\">=\"+r+\".\"+i+\".\"+o+\"-\"+a+\" <\"+r+\".\"+i+\".\"+(+o+1):\">=\"+r+\".\"+i+\".\"+o+\"-\"+a+\" <\"+r+\".\"+(+i+1)+\".0\":\">=\"+r+\".\"+i+\".\"+o+\"-\"+a+\" <\"+(+r+1)+\".0.0\"):(n(\"no pr\"),s=\"0\"===r?\"0\"===i?\">=\"+r+\".\"+i+\".\"+o+\" <\"+r+\".\"+i+\".\"+(+o+1):\">=\"+r+\".\"+i+\".\"+o+\" <\"+r+\".\"+(+i+1)+\".0\":\">=\"+r+\".\"+i+\".\"+o+\" <\"+(+r+1)+\".0.0\"),n(\"caret return\",s),s}))}(e,t)})).join(\" \")}(e,t),n(\"caret\",e),e=function(e,t){return e.trim().split(/\\s+/).map((function(e){return function(e,t){var r=t.loose?o[s.TILDELOOSE]:o[s.TILDE];return e.replace(r,(function(t,r,i,o,a){var s;return n(\"tilde\",e,t,r,i,o,a),S(r)?s=\"\":S(i)?s=\">=\"+r+\".0.0 <\"+(+r+1)+\".0.0\":S(o)?s=\">=\"+r+\".\"+i+\".0 <\"+r+\".\"+(+i+1)+\".0\":a?(n(\"replaceTilde pr\",a),s=\">=\"+r+\".\"+i+\".\"+o+\"-\"+a+\" <\"+r+\".\"+(+i+1)+\".0\"):s=\">=\"+r+\".\"+i+\".\"+o+\" <\"+r+\".\"+(+i+1)+\".0\",n(\"tilde return\",s),s}))}(e,t)})).join(\" \")}(e,t),n(\"tildes\",e),e=function(e,t){return n(\"replaceXRanges\",e,t),e.split(/\\s+/).map((function(e){return function(e,t){e=e.trim();var r=t.loose?o[s.XRANGELOOSE]:o[s.XRANGE];return e.replace(r,(function(r,i,o,a,s,l){n(\"xRange\",e,r,i,o,a,s,l);var c=S(o),u=c||S(a),f=u||S(s),p=f;return\"=\"===i&&p&&(i=\"\"),l=t.includePrerelease?\"-0\":\"\",c?r=\">\"===i||\"<\"===i?\"<0.0.0-0\":\"*\":i&&p?(u&&(a=0),s=0,\">\"===i?(i=\">=\",u?(o=+o+1,a=0,s=0):(a=+a+1,s=0)):\"<=\"===i&&(i=\"<\",u?o=+o+1:a=+a+1),r=i+o+\".\"+a+\".\"+s+l):u?r=\">=\"+o+\".0.0\"+l+\" <\"+(+o+1)+\".0.0\"+l:f&&(r=\">=\"+o+\".\"+a+\".0\"+l+\" <\"+o+\".\"+(+a+1)+\".0\"+l),n(\"xRange return\",r),r}))}(e,t)})).join(\" \")}(e,t),n(\"xrange\",e),e=function(e,t){return n(\"replaceStars\",e,t),e.trim().replace(o[s.STAR],\"\")}(e,t),n(\"stars\",e),e}(e,this.options)}),this).join(\" \").split(/\\s+/);return this.options.loose&&(a=a.filter((function(e){return!!e.match(i)}))),a=a.map((function(e){return new R(e,this.options)}),this)},N.prototype.intersects=function(e,t){if(!(e instanceof N))throw new TypeError(\"a Range is required\");return this.set.some((function(n){return A(n,t)&&e.set.some((function(e){return A(e,t)&&n.every((function(n){return e.every((function(e){return n.intersects(e,t)}))}))}))}))},t.toComparators=function(e,t){return new N(e,t).set.map((function(e){return e.map((function(e){return e.value})).join(\" \").trim().split(\" \")}))},N.prototype.test=function(e){if(!e)return!1;if(\"string\"==typeof e)try{e=new p(e,this.options)}catch(e){return!1}for(var t=0;t\":0===t.prerelease.length?t.patch++:t.prerelease.push(0),t.raw=t.format();case\"\":case\">=\":n&&!m(n,t)||(n=t);break;case\"<\":case\"<=\":break;default:throw new Error(\"Unexpected operation: \"+e.operator)}}))}if(n&&e.test(n))return n;return null},t.validRange=function(e,t){try{return new N(e,t).range||\"*\"}catch(e){return null}},t.ltr=function(e,t,n){return _(e,t,\"<\",n)},t.gtr=function(e,t,n){return _(e,t,\">\",n)},t.outside=_,t.prerelease=function(e,t){var n=f(e,t);return n&&n.prerelease.length?n.prerelease:null},t.intersects=function(e,t,n){return e=new N(e,n),t=new N(t,n),e.intersects(t)},t.coerce=function(e,t){if(e instanceof p)return e;\"number\"==typeof e&&(e=String(e));if(\"string\"!=typeof e)return null;var n=null;if((t=t||{}).rtl){for(var r;(r=o[s.COERCERTL].exec(e))&&(!n||n.index+n[0].length!==e.length);)n&&r.index+r[0].length===n.index+n[0].length||(n=r),o[s.COERCERTL].lastIndex=r.index+r[1].length+r[2].length;o[s.COERCERTL].lastIndex=-1}else n=e.match(o[s.COERCE]);if(null===n)return null;return f(n[2]+\".\"+(n[3]||\"0\")+\".\"+(n[4]||\"0\"),t)}})),Re=(Oe.SEMVER_SPEC_VERSION,Oe.re,Oe.src,Oe.tokens,Oe.parse,Oe.valid,Oe.clean,Oe.SemVer,Oe.inc,Oe.diff,Oe.compareIdentifiers,Oe.rcompareIdentifiers,Oe.major,Oe.minor,Oe.patch,Oe.compare,Oe.compareLoose,Oe.compareBuild,Oe.rcompare,Oe.sort,Oe.rsort,Oe.gt,Oe.lt,Oe.eq,Oe.neq,Oe.gte,Oe.lte,Oe.cmp,Oe.Comparator,Oe.Range,Oe.toComparators,Oe.satisfies),we=(Oe.maxSatisfying,Oe.minSatisfying,Oe.minVersion,Oe.validRange,Oe.ltr,Oe.gtr,Oe.outside,Oe.prerelease,Oe.intersects,Oe.coerce,be((function(e,t){Object.defineProperty(t,\"__esModule\",{value:!0}),t.default=function(e){var t=/\\/schema\\/([\\w-]+)\\/([\\w\\.\\-]+)\\.json$/g.exec(e).slice(1,3);return{library:t[0],version:t[1]}}}))),Ne=(Ie=we)&&Ie.__esModule&&Object.prototype.hasOwnProperty.call(Ie,\"default\")?Ie.default:Ie;const Ae={background:\"#333\",title:{color:\"#fff\"},style:{\"guide-label\":{fill:\"#fff\"},\"guide-title\":{fill:\"#fff\"}},axis:{domainColor:\"#fff\",gridColor:\"#888\",tickColor:\"#fff\"}},Se={background:\"#fff\",arc:{fill:\"#4572a7\"},area:{fill:\"#4572a7\"},line:{stroke:\"#4572a7\",strokeWidth:2},path:{stroke:\"#4572a7\"},rect:{fill:\"#4572a7\"},shape:{stroke:\"#4572a7\"},symbol:{fill:\"#4572a7\",strokeWidth:1.5,size:50},axis:{bandPosition:.5,grid:!0,gridColor:\"#000000\",gridOpacity:1,gridWidth:.5,labelPadding:10,tickSize:5,tickWidth:.5},axisBand:{grid:!1,tickExtra:!0},legend:{labelBaseline:\"middle\",labelFontSize:11,symbolSize:50,symbolType:\"square\"},range:{category:[\"#4572a7\",\"#aa4643\",\"#8aa453\",\"#71598e\",\"#4598ae\",\"#d98445\",\"#94aace\",\"#d09393\",\"#b9cc98\",\"#a99cbc\"]}},Le={arc:{fill:\"#30a2da\"},area:{fill:\"#30a2da\"},axis:{domainColor:\"#cbcbcb\",grid:!0,gridColor:\"#cbcbcb\",gridWidth:1,labelColor:\"#999\",labelFontSize:10,titleColor:\"#333\",tickColor:\"#cbcbcb\",tickSize:10,titleFontSize:14,titlePadding:10,labelPadding:4},axisBand:{grid:!1},background:\"#f0f0f0\",group:{fill:\"#f0f0f0\"},legend:{labelColor:\"#333\",labelFontSize:11,padding:1,symbolSize:30,symbolType:\"square\",titleColor:\"#333\",titleFontSize:14,titlePadding:10},line:{stroke:\"#30a2da\",strokeWidth:2},path:{stroke:\"#30a2da\",strokeWidth:.5},rect:{fill:\"#30a2da\"},range:{category:[\"#30a2da\",\"#fc4f30\",\"#e5ae38\",\"#6d904f\",\"#8b8b8b\",\"#b96db8\",\"#ff9e27\",\"#56cc60\",\"#52d2ca\",\"#52689e\",\"#545454\",\"#9fe4f8\"],diverging:[\"#cc0020\",\"#e77866\",\"#f6e7e1\",\"#d6e8ed\",\"#91bfd9\",\"#1d78b5\"],heatmap:[\"#d6e8ed\",\"#cee0e5\",\"#91bfd9\",\"#549cc6\",\"#1d78b5\"]},point:{filled:!0,shape:\"circle\"},shape:{stroke:\"#30a2da\"},style:{bar:{binSpacing:2,fill:\"#30a2da\",stroke:null}},title:{anchor:\"start\",fontSize:24,fontWeight:600,offset:20}},Te={group:{fill:\"#e5e5e5\"},arc:{fill:\"#000\"},area:{fill:\"#000\"},line:{stroke:\"#000\"},path:{stroke:\"#000\"},rect:{fill:\"#000\"},shape:{stroke:\"#000\"},symbol:{fill:\"#000\",size:40},axis:{domain:!1,grid:!0,gridColor:\"#FFFFFF\",gridOpacity:1,labelColor:\"#7F7F7F\",labelPadding:4,tickColor:\"#7F7F7F\",tickSize:5.67,titleFontSize:16,titleFontWeight:\"normal\"},legend:{labelBaseline:\"middle\",labelFontSize:11,symbolSize:40},range:{category:[\"#000000\",\"#7F7F7F\",\"#1A1A1A\",\"#999999\",\"#333333\",\"#B0B0B0\",\"#4D4D4D\",\"#C9C9C9\",\"#666666\",\"#DCDCDC\"]}},xe=\"Benton Gothic Bold, sans-serif\",_e={\"category-6\":[\"#ec8431\",\"#829eb1\",\"#c89d29\",\"#3580b1\",\"#adc839\",\"#ab7fb4\"],\"fire-7\":[\"#fbf2c7\",\"#f9e39c\",\"#f8d36e\",\"#f4bb6a\",\"#e68a4f\",\"#d15a40\",\"#ab4232\"],\"fireandice-6\":[\"#e68a4f\",\"#f4bb6a\",\"#f9e39c\",\"#dadfe2\",\"#a6b7c6\",\"#849eae\"],\"ice-7\":[\"#edefee\",\"#dadfe2\",\"#c4ccd2\",\"#a6b7c6\",\"#849eae\",\"#607785\",\"#47525d\"]},Ce={background:\"#ffffff\",title:{anchor:\"start\",color:\"#000000\",font:xe,fontSize:22,fontWeight:\"normal\"},arc:{fill:\"#82c6df\"},area:{fill:\"#82c6df\"},line:{stroke:\"#82c6df\",strokeWidth:2},path:{stroke:\"#82c6df\"},rect:{fill:\"#82c6df\"},shape:{stroke:\"#82c6df\"},symbol:{fill:\"#82c6df\",size:30},axis:{labelFont:\"Benton Gothic, sans-serif\",labelFontSize:11.5,labelFontWeight:\"normal\",titleFont:xe,titleFontSize:13,titleFontWeight:\"normal\"},axisX:{labelAngle:0,labelPadding:4,tickSize:3},axisY:{labelBaseline:\"middle\",maxExtent:45,minExtent:45,tickSize:2,titleAlign:\"left\",titleAngle:0,titleX:-45,titleY:-11},legend:{labelFont:\"Benton Gothic, sans-serif\",labelFontSize:11.5,symbolType:\"square\",titleFont:xe,titleFontSize:13,titleFontWeight:\"normal\"},range:{category:_e[\"category-6\"],diverging:_e[\"fireandice-6\"],heatmap:_e[\"fire-7\"],ordinal:_e[\"fire-7\"],ramp:_e[\"fire-7\"]}},Pe={background:\"#f9f9f9\",arc:{fill:\"#ab5787\"},area:{fill:\"#ab5787\"},line:{stroke:\"#ab5787\"},path:{stroke:\"#ab5787\"},rect:{fill:\"#ab5787\"},shape:{stroke:\"#ab5787\"},symbol:{fill:\"#ab5787\",size:30},axis:{domainColor:\"#979797\",domainWidth:.5,gridWidth:.2,labelColor:\"#979797\",tickColor:\"#979797\",tickWidth:.2,titleColor:\"#979797\"},axisBand:{grid:!1},axisX:{grid:!0,tickSize:10},axisY:{domain:!1,grid:!0,tickSize:0},legend:{labelFontSize:11,padding:1,symbolSize:30,symbolType:\"square\"},range:{category:[\"#ab5787\",\"#51b2e5\",\"#703c5c\",\"#168dd9\",\"#d190b6\",\"#00609f\",\"#d365ba\",\"#154866\",\"#666666\",\"#c4c4c4\"]}},De={background:\"#fff\",arc:{fill:\"#3e5c69\"},area:{fill:\"#3e5c69\"},line:{stroke:\"#3e5c69\"},path:{stroke:\"#3e5c69\"},rect:{fill:\"#3e5c69\"},shape:{stroke:\"#3e5c69\"},symbol:{fill:\"#3e5c69\"},axis:{domainWidth:.5,grid:!0,labelPadding:2,tickSize:5,tickWidth:.5,titleFontWeight:\"normal\"},axisBand:{grid:!1},axisX:{gridWidth:.2},axisY:{gridDash:[3],gridWidth:.4},legend:{labelFontSize:11,padding:1,symbolType:\"square\"},range:{category:[\"#3e5c69\",\"#6793a6\",\"#182429\",\"#0570b0\",\"#3690c0\",\"#74a9cf\",\"#a6bddb\",\"#e2ddf2\"]}},ke=\"2.4.0\";var Fe=Object.freeze({version:ke,dark:Ae,excel:Se,fivethirtyeight:Le,ggplot2:Te,latimes:Ce,quartz:Pe,vox:De}),Me=\"#vg-tooltip-element {\\n visibility: hidden;\\n padding: 8px;\\n position: fixed;\\n z-index: 1000;\\n font-family: sans-serif;\\n font-size: 11px;\\n border-radius: 3px;\\n box-shadow: 2px 2px 4px rgba(0, 0, 0, 0.1);\\n /* The default theme is the light theme. */\\n background-color: rgba(255, 255, 255, 0.95);\\n border: 1px solid #d9d9d9;\\n color: black; }\\n #vg-tooltip-element.visible {\\n visibility: visible; }\\n #vg-tooltip-element h2 {\\n margin-top: 0;\\n margin-bottom: 10px;\\n font-size: 13px; }\\n #vg-tooltip-element table {\\n border-spacing: 0; }\\n #vg-tooltip-element table tr {\\n border: none; }\\n #vg-tooltip-element table tr td {\\n overflow: hidden;\\n text-overflow: ellipsis;\\n padding-top: 2px;\\n padding-bottom: 2px; }\\n #vg-tooltip-element table tr td.key {\\n color: #808080;\\n max-width: 150px;\\n text-align: right;\\n padding-right: 4px; }\\n #vg-tooltip-element table tr td.value {\\n display: block;\\n max-width: 300px;\\n max-height: 7em;\\n text-align: left; }\\n #vg-tooltip-element.dark-theme {\\n background-color: rgba(32, 32, 32, 0.9);\\n border: 1px solid #f5f5f5;\\n color: white; }\\n #vg-tooltip-element.dark-theme td.key {\\n color: #bfbfbf; }\\n\";const je=\"vg-tooltip-element\",$e={offsetX:10,offsetY:10,id:je,styleId:\"vega-tooltip-style\",theme:\"light\",disableDefaultStyle:!1,sanitize:function(e){return String(e).replace(/&/g,\"&\").replace(/t&&c(),a=t=n+1):\"]\"===r&&(a||ze(\"Access path missing open bracket: \"+e),a>0&&c(),a=0,t=n+1):n>t?c():t=n+1}return a&&ze(\"Access path missing closing bracket: \"+e),o&&ze(\"Access path missing closing quote: \"+e),n>t&&(n++,c()),i}(e),r=\"return _[\"+n.map(Ve).join(\"][\")+\"];\";Ge(Function(\"_\",r),[e=1===n.length?n[0]:e],t||e)}(\"id\"),Ge((function(e){return e}),We,\"identity\"),Ge((function(){return 0}),We,\"zero\"),Ge((function(){return 1}),We,\"one\"),Ge((function(){return!0}),We,\"true\"),Ge((function(){return!1}),We,\"false\"),function(e,t){var n={};for(var r in e)Object.prototype.hasOwnProperty.call(e,r)&&t.indexOf(r)<0&&(n[r]=e[r]);if(null!=e&&\"function\"==typeof Object.getOwnPropertySymbols){var i=0;for(r=Object.getOwnPropertySymbols(e);ie?\"[Object]\":t.indexOf(r)>=0?\"[Circular]\":(t.push(r),r)}}(t))}class Ye{constructor(e){this.options=Object.assign(Object.assign({},$e),e);const t=this.options.id;if(this.call=this.tooltipHandler.bind(this),!this.options.disableDefaultStyle&&!document.getElementById(this.options.styleId)){const e=document.createElement(\"style\");e.setAttribute(\"id\",this.options.styleId),e.innerHTML=function(e){if(!/^[A-Za-z]+[-:.\\w]*$/.test(e))throw new Error(\"Invalid HTML ID\");return Me.toString().replace(je,e)}(t);const n=document.head;n.childNodes.length>0?n.insertBefore(e,n.childNodes[0]):n.appendChild(e)}this.el=document.getElementById(t),this.el||(this.el=document.createElement(\"div\"),this.el.setAttribute(\"id\",t),this.el.classList.add(\"vg-tooltip\"),document.body.appendChild(this.el))}tooltipHandler(e,t,n,r){if(null==r||\"\"===r)return void this.el.classList.remove(\"visible\",`${this.options.theme}-theme`);this.el.innerHTML=function(e,t,n){if(Ue(e))return`[${e.map(e=>t(Xe(e)?e:qe(e,n))).join(\", \")}]`;if(Be(e)){let r=\"\";const i=e,{title:o}=i,a=He(i,[\"title\"]);o&&(r+=`

${t(o)}

`);const s=Object.keys(a);if(s.length>0){r+=\"\";for(const e of s){let i=a[e];void 0!==i&&(Be(i)&&(i=qe(i,n)),r+=``)}r+=\"
${t(e)}:${t(i)}
\"}return r||\"{}\"}return t(e)}(r,this.options.sanitize,this.options.maxDepth),this.el.classList.add(\"visible\",`${this.options.theme}-theme`);const{x:i,y:o}=function(e,t,n,r){let i=e.clientX+n;i+t.width>window.innerWidth&&(i=+e.clientX-n-t.width);let o=e.clientY+r;return o+t.height>window.innerHeight&&(o=+e.clientY-r-t.height),{x:i,y:o}}(t,this.el.getBoundingClientRect(),this.options.offsetX,this.options.offsetY);this.el.setAttribute(\"style\",`top: ${o}px; left: ${i}px`)}}var Je='.vega-embed {\\n position: relative;\\n display: inline-block;\\n padding-right: 38px; }\\n .vega-embed details:not([open]) > :not(summary) {\\n display: none !important; }\\n .vega-embed summary {\\n list-style: none;\\n position: absolute;\\n top: 0;\\n right: 0;\\n padding: 6px;\\n z-index: 1000;\\n background: white;\\n box-shadow: 1px 1px 3px rgba(0, 0, 0, 0.1);\\n color: #1b1e23;\\n border: 1px solid #aaa;\\n border-radius: 999px;\\n opacity: 0.2;\\n transition: opacity 0.4s ease-in;\\n outline: none;\\n cursor: pointer;\\n line-height: 0px; }\\n .vega-embed summary::-webkit-details-marker {\\n display: none; }\\n .vega-embed summary:active {\\n box-shadow: #aaa 0px 0px 0px 1px inset; }\\n .vega-embed summary svg {\\n width: 14px;\\n height: 14px; }\\n .vega-embed details[open] summary {\\n opacity: 0.7; }\\n .vega-embed:hover summary,\\n .vega-embed:focus summary {\\n opacity: 1 !important;\\n transition: opacity 0.2s ease; }\\n .vega-embed .vega-actions {\\n position: absolute;\\n top: 35px;\\n right: -9px;\\n display: flex;\\n flex-direction: column;\\n padding-bottom: 8px;\\n padding-top: 8px;\\n border-radius: 4px;\\n box-shadow: 0 2px 8px 0 rgba(0, 0, 0, 0.2);\\n border: 1px solid #d9d9d9;\\n background: white;\\n animation-duration: 0.15s;\\n animation-name: scale-in;\\n animation-timing-function: cubic-bezier(0.2, 0, 0.13, 1.5); }\\n .vega-embed .vega-actions a {\\n padding: 8px 16px;\\n font-family: sans-serif;\\n font-size: 14px;\\n font-weight: 600;\\n white-space: nowrap;\\n color: #434a56;\\n text-decoration: none; }\\n .vega-embed .vega-actions a:hover {\\n background-color: #f7f7f9;\\n color: black; }\\n .vega-embed .vega-actions::before, .vega-embed .vega-actions::after {\\n content: \"\";\\n display: inline-block;\\n position: absolute; }\\n .vega-embed .vega-actions::before {\\n left: auto;\\n right: 14px;\\n top: -16px;\\n border: 8px solid #0000;\\n border-bottom-color: #d9d9d9; }\\n .vega-embed .vega-actions::after {\\n left: auto;\\n right: 15px;\\n top: -14px;\\n border: 7px solid #0000;\\n border-bottom-color: #fff; }\\n\\n.vega-embed-wrapper {\\n max-width: 100%;\\n overflow: scroll;\\n padding-right: 14px; }\\n\\n@keyframes scale-in {\\n from {\\n opacity: 0;\\n transform: scale(0.6); }\\n to {\\n opacity: 1;\\n transform: scale(1); } }\\n';const Ze=e;let Ke=t;const Qe=window;void 0===Ke&&Qe.vl&&Qe.vl.compile&&(Ke=Qe.vl);const et={CLICK_TO_VIEW_ACTIONS:\"Click to view actions\",COMPILED_ACTION:\"View Compiled Vega\",EDITOR_ACTION:\"Open in Vega Editor\",PNG_ACTION:\"Save as PNG\",SOURCE_ACTION:\"View Source\",SVG_ACTION:\"Save as SVG\"},tt={vega:\"Vega\",\"vega-lite\":\"Vega-Lite\"},nt={vega:Ze.version,\"vega-lite\":Ke?Ke.version:\"not available\"},rt={vega:e=>e,\"vega-lite\":(e,t)=>Ke.compile(e,{config:t}).spec},it='\\n\\n \\n \\n \\n';function ot(e,t,n,r){const i=`${t}
`,o=`
${n}`,a=window.open(\"\");a.document.write(i+e+o),a.document.title=`${tt[r]} JSON Source`}function at(t,n,r={}){return ce(this,void 0,void 0,(function*(){const i=(o=r.loader)&&\"load\"in o?r.loader:Ze.loader(r.loader);var o;if(Ze.isString(n)){const e=yield i.load(n);return at(t,JSON.parse(e),r)}let a=(r=ve(r,n.usermeta&&n.usermeta.embedOptions||{})).config||{};if(Ze.isString(a)){const e=yield i.load(a);return at(t,n,Object.assign(Object.assign({},r),{config:JSON.parse(e)}))}const s=e.isBoolean(r.actions)?r.actions:ve({export:{svg:!0,png:!0},source:!0,compiled:!0,editor:!0},r.actions||{}),l=Object.assign(Object.assign({},et),r.i18n),c=r.renderer||\"canvas\",u=r.logLevel||Ze.Warn,f=r.downloadFileName||\"visualization\";if(!1!==r.defaultStyle){const e=\"vega-embed-style\";if(!document.getElementById(e)){const t=document.createElement(\"style\");t.id=e,t.innerText=void 0===r.defaultStyle||!0===r.defaultStyle?Je.toString():r.defaultStyle,document.head.appendChild(t)}}r.theme&&(a=ve(Fe[r.theme],a));const p=function(e,t){if(e.$schema){const n=Ne(e.$schema);t&&t!==n.library&&console.warn(`The given visualization spec is written in ${tt[n.library]}, but mode argument sets ${tt[t]||t}.`);const r=n.library;return Re(nt[r],`^${n.version.slice(1)}`)||console.warn(`The input spec uses ${tt[r]} ${n.version}, but the current version of ${tt[r]} is v${nt[r]}.`),r}return\"mark\"in e||\"encoding\"in e||\"layer\"in e||\"hconcat\"in e||\"vconcat\"in e||\"facet\"in e||\"repeat\"in e?\"vega-lite\":\"marks\"in e||\"signals\"in e||\"scales\"in e||\"axes\"in e?\"vega\":t||\"vega\"}(n,r.mode);let h=rt[p](n,a);if(\"vega-lite\"===p&&h.$schema){const e=Ne(h.$schema);Re(nt.vega,`^${e.version.slice(1)}`)||console.warn(`The compiled spec uses Vega ${e.version}, but current version is v${nt.vega}.`)}const d=function(e){return\"string\"==typeof e?new ae([[document.querySelector(e)]],[document.documentElement]):new ae([[e]],oe)}(t).classed(\"vega-embed\",!0).html(\"\"),g=r.patch;if(g)if(g instanceof Function)h=g(h);else if(Ze.isString(g)){const e=yield i.load(g);h=ve(h,JSON.parse(e))}else h=ve(h,g);const m=Ze.parse(h,\"vega-lite\"===p?{}:a),v=new Ze.View(m,{loader:i,logLevel:u,renderer:c});if(!1!==r.tooltip){let e;e=\"function\"==typeof r.tooltip?r.tooltip:new Ye(!0===r.tooltip?{}:r.tooltip).call,v.tooltip(e)}let{hover:E}=r;if(void 0===E&&(E=\"vega\"===p),E){const{hoverSet:e,updateSet:t}=\"boolean\"==typeof E?{}:E;v.hover(e,t)}if(r&&(r.width&&v.width(r.width),r.height&&v.height(r.height),r.padding&&v.padding(r.padding)),yield v.initialize(t).runAsync(),!1!==s){let e=d;if(!1!==r.defaultStyle){const t=d.append(\"details\").attr(\"title\",l.CLICK_TO_VIEW_ACTIONS);e=t,t.insert(\"summary\").html(it);const n=t.node();document.addEventListener(\"click\",e=>{n.contains(e.target)||n.removeAttribute(\"open\")})}const t=e.insert(\"div\").attr(\"class\",\"vega-actions\");if(!0===s||!1!==s.export)for(const e of[\"svg\",\"png\"])if(!0===s||!0===s.export||s.export[e]){const n=l[`${e.toUpperCase()}_ACTION`];t.append(\"a\").text(n).attr(\"href\",\"#\").attr(\"target\",\"_blank\").attr(\"download\",`${f}.${e}`).on(\"mousedown\",(function(){v.toImageURL(e,r.scaleFactor).then(e=>{this.href=e}).catch(e=>{throw e}),Z.preventDefault()}))}if(!0!==s&&!1===s.source||t.append(\"a\").text(l.SOURCE_ACTION).attr(\"href\",\"#\").on(\"mousedown\",()=>{ot(ye(n),r.sourceHeader||\"\",r.sourceFooter||\"\",p),Z.preventDefault()}),\"vega-lite\"!==p||!0!==s&&!1===s.compiled||t.append(\"a\").text(l.COMPILED_ACTION).attr(\"href\",\"#\").on(\"mousedown\",()=>{ot(ye(h),r.sourceHeader||\"\",r.sourceFooter||\"\",\"vega\"),Z.preventDefault()}),!0===s||!1!==s.editor){const e=r.editorUrl||\"https://vega.github.io/editor/\";t.append(\"a\").text(l.EDITOR_ACTION).attr(\"href\",\"#\").on(\"mousedown\",()=>{!function(e,t,n){const r=e.open(t),i=250;let o=~~(1e4/i);e.addEventListener(\"message\",(function t(n){n.source===r&&(o=0,e.removeEventListener(\"message\",t,!1))}),!1),setTimeout((function e(){o<=0||(r.postMessage(n,\"*\"),setTimeout(e,i),o-=1)}),i)}(window,e,{config:a,mode:p,renderer:c,spec:ye(n)}),Z.preventDefault()})}}return{view:v,spec:n,vgSpec:h}}))}function st(e,t={}){return ce(this,void 0,void 0,(function*(){const n=document.createElement(\"div\");n.classList.add(\"vega-embed-wrapper\");const r=document.createElement(\"div\");n.appendChild(r);const i=!0===t.actions||!1===t.actions?t.actions:Object.assign({export:!0,source:!1,compiled:!0,editor:!0},t.actions||{}),o=yield at(r,e,Object.assign({actions:i},t||{}));return n.value=o.view,n}))}String.prototype.startsWith||(String.prototype.startsWith=function(e,t){return this.substr(!t||t<0?0:+t,e.length)===e});const lt=(...t)=>t.length>1&&(e.isString(t[0])&&!function(e){return e.startsWith(\"http://\")||e.startsWith(\"https://\")||e.startsWith(\"//\")}(t[0])||function(e){return e instanceof se||\"object\"==typeof HTMLElement?e instanceof HTMLElement:e&&\"object\"==typeof e&&null!==e&&1===e.nodeType&&\"string\"==typeof e.nodeName}(t[0])||3===t.length)?at(t[0],t[1],t[2]):st(t[0],t[1]);return lt.vegaLite=Ke,lt.vl=Ke,lt.container=st,lt.embed=at,lt.vega=Ze,lt.default=at,lt.version=le,lt}));\n" }, { "path": "docs/_static/js/vega-lite@5.js", "content": "!function(e,t){\"object\"==typeof exports&&\"undefined\"!=typeof module?t(exports,require(\"vega\")):\"function\"==typeof define&&define.amd?define([\"exports\",\"vega\"],t):t((e=\"undefined\"!=typeof globalThis?globalThis:e||self).vegaLite={},e.vega)}(this,(function(e,t){\"use strict\";var n=\"5.16.3\";function i(e){return!!e.or}function r(e){return!!e.and}function o(e){return!!e.not}function a(e,t){if(o(e))a(e.not,t);else if(r(e))for(const n of e.and)a(n,t);else if(i(e))for(const n of e.or)a(n,t);else t(e)}function s(e,t){return o(e)?{not:s(e.not,t)}:r(e)?{and:e.and.map((e=>s(e,t)))}:i(e)?{or:e.or.map((e=>s(e,t)))}:t(e)}const l=structuredClone;function c(e){throw new Error(e)}function u(e,n){const i={};for(const r of n)t.hasOwnProperty(e,r)&&(i[r]=e[r]);return i}function f(e,t){const n={...e};for(const e of t)delete n[e];return n}function d(e){if(t.isNumber(e))return e;const n=t.isString(e)?e:X(e);if(n.length<250)return n;let i=0;for(let e=0;e1?t-1:0),i=1;i0===t?e:`[${e}]`)),r=e.map(((t,n)=>e.slice(0,n+1).join(\"\")));for(const e of r)n.add(e)}return n}function k(e,t){return void 0===e||void 0===t||$(w(e),w(t))}function S(e){return 0===D(e).length}Set.prototype.toJSON=function(){return`Set(${[...this].map((e=>X(e))).join(\",\")})`};const D=Object.keys,F=Object.values,z=Object.entries;function O(e){return!0===e||!1===e}function _(e){const t=e.replace(/\\W/g,\"_\");return(e.match(/^\\d+/)?\"_\":\"\")+t}function N(e,t){return o(e)?`!(${N(e.not,t)})`:r(e)?`(${e.and.map((e=>N(e,t))).join(\") && (\")})`:i(e)?`(${e.or.map((e=>N(e,t))).join(\") || (\")})`:t(e)}function C(e,t){if(0===t.length)return!0;const n=t.shift();return n in e&&C(e[n],t)&&delete e[n],S(e)}function P(e){return e.charAt(0).toUpperCase()+e.substr(1)}function A(e){let n=arguments.length>1&&void 0!==arguments[1]?arguments[1]:\"datum\";const i=t.splitAccessPath(e),r=[];for(let e=1;e<=i.length;e++){const o=`[${i.slice(0,e).map(t.stringValue).join(\"][\")}]`;r.push(`${n}${o}`)}return r.join(\" && \")}function j(e){return`${arguments.length>1&&void 0!==arguments[1]?arguments[1]:\"datum\"}[${t.stringValue(t.splitAccessPath(e).join(\".\"))}]`}function T(e){return e.replace(/(\\[|\\]|\\.|'|\")/g,\"\\\\$1\")}function E(e){return`${t.splitAccessPath(e).map(T).join(\"\\\\.\")}`}function M(e,t,n){return e.replace(new RegExp(t.replace(/[-/\\\\^$*+?.()|[\\]{}]/g,\"\\\\$&\"),\"g\"),n)}function L(e){return`${t.splitAccessPath(e).join(\".\")}`}function q(e){return e?t.splitAccessPath(e).length:0}function U(){for(var e=arguments.length,t=new Array(e),n=0;nfn(e[t])?_(`_${t}_${z(e[t])}`):_(`_${t}_${e[t]}`))).join(\"\")}function ln(e){return!0===e||un(e)&&!e.binned}function cn(e){return\"binned\"===e||un(e)&&!0===e.binned}function un(e){return t.isObject(e)}function fn(e){return e?.param}function dn(e){switch(e){case Q:case J:case ye:case me:case pe:case ge:case we:case be:case xe:case $e:case he:return 6;case ke:return 4;default:return 10}}function mn(e){return!!e?.expr}function pn(e){const t=D(e||{}),n={};for(const i of t)n[i]=Sn(e[i]);return n}function gn(e){const{anchor:t,frame:n,offset:i,orient:r,angle:o,limit:a,color:s,subtitleColor:l,subtitleFont:c,subtitleFontSize:f,subtitleFontStyle:d,subtitleFontWeight:m,subtitleLineHeight:p,subtitlePadding:g,...h}=e,y={...t?{anchor:t}:{},...n?{frame:n}:{},...i?{offset:i}:{},...r?{orient:r}:{},...void 0!==o?{angle:o}:{},...void 0!==a?{limit:a}:{}},v={...l?{subtitleColor:l}:{},...c?{subtitleFont:c}:{},...f?{subtitleFontSize:f}:{},...d?{subtitleFontStyle:d}:{},...m?{subtitleFontWeight:m}:{},...p?{subtitleLineHeight:p}:{},...g?{subtitlePadding:g}:{}};return{titleMarkConfig:{...h,...s?{fill:s}:{}},subtitleMarkConfig:u(e,[\"align\",\"baseline\",\"dx\",\"dy\",\"limit\"]),nonMarkTitleProperties:y,subtitle:v}}function hn(e){return t.isString(e)||t.isArray(e)&&t.isString(e[0])}function yn(e){return!!e?.signal}function vn(e){return!!e.step}function bn(e){return!t.isArray(e)&&(\"field\"in e&&\"data\"in e)}const xn=D({aria:1,description:1,ariaRole:1,ariaRoleDescription:1,blend:1,opacity:1,fill:1,fillOpacity:1,stroke:1,strokeCap:1,strokeWidth:1,strokeOpacity:1,strokeDash:1,strokeDashOffset:1,strokeJoin:1,strokeOffset:1,strokeMiterLimit:1,startAngle:1,endAngle:1,padAngle:1,innerRadius:1,outerRadius:1,size:1,shape:1,interpolate:1,tension:1,orient:1,align:1,baseline:1,text:1,dir:1,dx:1,dy:1,ellipsis:1,limit:1,radius:1,theta:1,angle:1,font:1,fontSize:1,fontWeight:1,fontStyle:1,lineBreak:1,lineHeight:1,cursor:1,href:1,tooltip:1,cornerRadius:1,cornerRadiusTopLeft:1,cornerRadiusTopRight:1,cornerRadiusBottomLeft:1,cornerRadiusBottomRight:1,aspect:1,width:1,height:1,url:1,smooth:1}),$n={arc:1,area:1,group:1,image:1,line:1,path:1,rect:1,rule:1,shape:1,symbol:1,text:1,trail:1},wn=[\"cornerRadius\",\"cornerRadiusTopLeft\",\"cornerRadiusTopRight\",\"cornerRadiusBottomLeft\",\"cornerRadiusBottomRight\"];function kn(e){const n=t.isArray(e.condition)?e.condition.map(Dn):Dn(e.condition);return{...Sn(e),condition:n}}function Sn(e){if(mn(e)){const{expr:t,...n}=e;return{signal:t,...n}}return e}function Dn(e){if(mn(e)){const{expr:t,...n}=e;return{signal:t,...n}}return e}function Fn(e){if(mn(e)){const{expr:t,...n}=e;return{signal:t,...n}}return yn(e)?e:void 0!==e?{value:e}:void 0}function zn(e){return yn(e)?e.signal:t.stringValue(e.value)}function On(e){return yn(e)?e.signal:null==e?null:t.stringValue(e)}function _n(e,t,n){for(const i of n){const n=Pn(i,t.markDef,t.config);void 0!==n&&(e[i]=Fn(n))}return e}function Nn(e){return[].concat(e.type,e.style??[])}function Cn(e,t,n){let i=arguments.length>3&&void 0!==arguments[3]?arguments[3]:{};const{vgChannel:r,ignoreVgConfig:o}=i;return r&&void 0!==t[r]?t[r]:void 0!==t[e]?t[e]:!o||r&&r!==e?Pn(e,t,n,i):void 0}function Pn(e,t,n){let{vgChannel:i}=arguments.length>3&&void 0!==arguments[3]?arguments[3]:{};return U(i?An(e,t,n.style):void 0,An(e,t,n.style),i?n[t.type][i]:void 0,n[t.type][e],i?n.mark[i]:n.mark[e])}function An(e,t,n){return jn(e,Nn(t),n)}function jn(e,n,i){let r;n=t.array(n);for(const t of n){const n=i[t];n&&void 0!==n[e]&&(r=n[e])}return r}function Tn(e,n){return t.array(e).reduce(((e,t)=>(e.field.push(oa(t,n)),e.order.push(t.sort??\"ascending\"),e)),{field:[],order:[]})}function En(e,t){const n=[...e];return t.forEach((e=>{for(const t of n)if(Y(t,e))return;n.push(e)})),n}function Mn(e,n){return Y(e,n)||!n?e:e?[...t.array(e),...t.array(n)].join(\", \"):n}function Ln(e,t){const n=e.value,i=t.value;if(null==n||null===i)return{explicit:e.explicit,value:null};if((hn(n)||yn(n))&&(hn(i)||yn(i)))return{explicit:e.explicit,value:Mn(n,i)};if(hn(n)||yn(n))return{explicit:e.explicit,value:n};if(hn(i)||yn(i))return{explicit:e.explicit,value:i};if(!(hn(n)||yn(n)||hn(i)||yn(i)))return{explicit:e.explicit,value:En(n,i)};throw new Error(\"It should never reach here\")}function qn(e,t,n){return(t=function(e){var t=function(e,t){if(\"object\"!=typeof e||null===e)return e;var n=e[Symbol.toPrimitive];if(void 0!==n){var i=n.call(e,t||\"default\");if(\"object\"!=typeof i)return i;throw new TypeError(\"@@toPrimitive must return a primitive value.\")}return(\"string\"===t?String:Number)(e)}(e,\"string\");return\"symbol\"==typeof t?t:String(t)}(t))in e?Object.defineProperty(e,t,{value:n,enumerable:!0,configurable:!0,writable:!0}):e[t]=n,e}function Un(e,t,n){return function(e,t,n){if(t.set)t.set.call(e,n);else{if(!t.writable)throw new TypeError(\"attempted to set read only private field\");t.value=n}}(e,Rn(e,t,\"set\"),n),n}function Rn(e,t,n){if(!t.has(e))throw new TypeError(\"attempted to \"+n+\" private field on non-instance\");return t.get(e)}function Wn(e,t,n){!function(e,t){if(t.has(e))throw new TypeError(\"Cannot initialize the same private elements twice on an object\")}(e,t),t.set(e,n)}function Bn(e){return`Invalid specification ${X(e)}. Make sure the specification includes at least one of the following properties: \"mark\", \"layer\", \"facet\", \"hconcat\", \"vconcat\", \"concat\", or \"repeat\".`}const In='Autosize \"fit\" only works for single views and layered views.';function Hn(e){return`${\"width\"==e?\"Width\":\"Height\"} \"container\" only works for single views and layered views.`}function Vn(e){return`${\"width\"==e?\"Width\":\"Height\"} \"container\" only works well with autosize \"fit\" or \"fit-${\"width\"==e?\"x\":\"y\"}\".`}function Gn(e){return e?`Dropping \"fit-${e}\" because spec has discrete ${rt(e)}.`:'Dropping \"fit\" because spec has discrete size.'}function Yn(e){return`Unknown field for ${e}. Cannot calculate view size.`}function Xn(e){return`Cannot project a selection on encoding channel \"${e}\", which has no field.`}function Qn(e,t){return`Cannot project a selection on encoding channel \"${e}\" as it uses an aggregate function (\"${t}\").`}function Jn(e){return`Selection not supported for ${e} yet.`}const Kn=\"The same selection must be used to override scale domains in a layered view.\";function Zn(e){return`The \"columns\" property cannot be used when \"${e}\" has nested row/column.`}function ei(e,t,n){return`An ancestor parsed field \"${e}\" as ${n} but a child wants to parse the field as ${t}.`}function ti(e){return`Config.customFormatTypes is not true, thus custom format type and format for channel ${e} are dropped.`}function ni(e){return`${e}Offset dropped because ${e} is continuous`}function ii(e){return`Invalid field type \"${e}\".`}function ri(e,t){const{fill:n,stroke:i}=t;return`Dropping color ${e} as the plot also has ${n&&i?\"fill and stroke\":n?\"fill\":\"stroke\"}.`}function oi(e,t){return`Dropping ${X(e)} from channel \"${t}\" since it does not contain any data field, datum, value, or signal.`}function ai(e,t,n){return`${e} dropped as it is incompatible with \"${t}\"${n?` when ${n}`:\"\"}.`}function si(e){return`${e} encoding should be discrete (ordinal / nominal / binned).`}function li(e){return`${e} encoding should be discrete (ordinal / nominal / binned) or use a discretizing scale (e.g. threshold).`}function ci(e,t){return`Using discrete channel \"${e}\" to encode \"${t}\" field can be misleading as it does not encode ${\"ordinal\"===t?\"order\":\"magnitude\"}.`}function ui(e){return`Using unaggregated domain with raw field has no effect (${X(e)}).`}function fi(e){return`Unaggregated domain not applicable for \"${e}\" since it produces values outside the origin domain of the source data.`}function di(e){return`Unaggregated domain is currently unsupported for log scale (${X(e)}).`}function mi(e,t,n){return`${n}-scale's \"${t}\" is dropped as it does not work with ${e} scale.`}function pi(e){return`The step for \"${e}\" is dropped because the ${\"width\"===e?\"x\":\"y\"} is continuous.`}const gi=\"Domains that should be unioned has conflicting sort properties. Sort will be set to true.\";function hi(e,t){return`Invalid ${e}: ${X(t)}.`}function yi(e){return`1D error band does not support ${e}.`}function vi(e){return`Channel ${e} is required for \"binned\" bin.`}const bi=t.logger(t.Warn);let xi=bi;function $i(){xi.warn(...arguments)}function wi(e){if(e&&t.isObject(e))for(const t of Ni)if(t in e)return!0;return!1}const ki=[\"january\",\"february\",\"march\",\"april\",\"may\",\"june\",\"july\",\"august\",\"september\",\"october\",\"november\",\"december\"],Si=ki.map((e=>e.substr(0,3))),Di=[\"sunday\",\"monday\",\"tuesday\",\"wednesday\",\"thursday\",\"friday\",\"saturday\"],Fi=Di.map((e=>e.substr(0,3)));function zi(e,n){const i=[];if(n&&void 0!==e.day&&D(e).length>1&&($i(function(e){return`Dropping day from datetime ${X(e)} as day cannot be combined with other units.`}(e)),delete(e=l(e)).day),void 0!==e.year?i.push(e.year):i.push(2012),void 0!==e.month){const r=n?function(e){if(V(e)&&(e=+e),t.isNumber(e))return e-1;{const t=e.toLowerCase(),n=ki.indexOf(t);if(-1!==n)return n;const i=t.substr(0,3),r=Si.indexOf(i);if(-1!==r)return r;throw new Error(hi(\"month\",e))}}(e.month):e.month;i.push(r)}else if(void 0!==e.quarter){const r=n?function(e){if(V(e)&&(e=+e),t.isNumber(e))return e>4&&$i(hi(\"quarter\",e)),e-1;throw new Error(hi(\"quarter\",e))}(e.quarter):e.quarter;i.push(t.isNumber(r)?3*r:`${r}*3`)}else i.push(0);if(void 0!==e.date)i.push(e.date);else if(void 0!==e.day){const r=n?function(e){if(V(e)&&(e=+e),t.isNumber(e))return e%7;{const t=e.toLowerCase(),n=Di.indexOf(t);if(-1!==n)return n;const i=t.substr(0,3),r=Fi.indexOf(i);if(-1!==r)return r;throw new Error(hi(\"day\",e))}}(e.day):e.day;i.push(t.isNumber(r)?r+1:`${r}+1`)}else i.push(1);for(const t of[\"hours\",\"minutes\",\"seconds\",\"milliseconds\"]){const n=e[t];i.push(void 0===n?0:n)}return i}function Oi(e){const t=zi(e,!0).join(\", \");return e.utc?`utc(${t})`:`datetime(${t})`}const _i={year:1,quarter:1,month:1,week:1,day:1,dayofyear:1,date:1,hours:1,minutes:1,seconds:1,milliseconds:1},Ni=D(_i);function Ci(e){return t.isObject(e)?e.binned:Pi(e)}function Pi(e){return e&&e.startsWith(\"binned\")}function Ai(e){return e.startsWith(\"utc\")}const ji={\"year-month\":\"%b %Y \",\"year-month-date\":\"%b %d, %Y \"};function Ti(e){return Ni.filter((t=>Mi(e,t)))}function Ei(e){const t=Ti(e);return t[t.length-1]}function Mi(e,t){const n=e.indexOf(t);return!(n<0)&&(!(n>0&&\"seconds\"===t&&\"i\"===e.charAt(n-1))&&(!(e.length>n+3&&\"day\"===t&&\"o\"===e.charAt(n+3))&&!(n>0&&\"year\"===t&&\"f\"===e.charAt(n-1))))}function Li(e,t){let{end:n}=arguments.length>2&&void 0!==arguments[2]?arguments[2]:{end:!1};const i=A(t),r=Ai(e)?\"utc\":\"\";let o;const a={};for(const t of Ni)Mi(e,t)&&(a[t]=\"quarter\"===(s=t)?`(${r}quarter(${i})-1)`:`${r}${s}(${i})`,o=t);var s;return n&&(a[o]+=\"+1\"),function(e){const t=zi(e,!1).join(\", \");return e.utc?`utc(${t})`:`datetime(${t})`}(a)}function qi(e){if(!e)return;return`timeUnitSpecifier(${X(Ti(e))}, ${X(ji)})`}function Ui(e){if(!e)return;let n;return t.isString(e)?n=Pi(e)?{unit:e.substring(6),binned:!0}:{unit:e}:t.isObject(e)&&(n={...e,...e.unit?{unit:e.unit}:{}}),Ai(n.unit)&&(n.utc=!0,n.unit=n.unit.substring(3)),n}function Ri(e){let t=arguments.length>1&&void 0!==arguments[1]?arguments[1]:e=>e;const n=Ui(e),i=Ei(n.unit);if(i&&\"day\"!==i){const e={year:2001,month:1,date:1,hours:0,minutes:0,seconds:0,milliseconds:0},{step:r,part:o}=Bi(i,n.step);return`${t(Oi({...e,[o]:+e[o]+r}))} - ${t(Oi(e))}`}}const Wi={year:1,month:1,date:1,hours:1,minutes:1,seconds:1,milliseconds:1};function Bi(e){let t=arguments.length>1&&void 0!==arguments[1]?arguments[1]:1;if(function(e){return!!Wi[e]}(e))return{part:e,step:t};switch(e){case\"day\":case\"dayofyear\":return{part:\"date\",step:t};case\"quarter\":return{part:\"month\",step:3*t};case\"week\":return{part:\"date\",step:7*t}}}function Ii(e){return!!e?.field&&void 0!==e.equal}function Hi(e){return!!e?.field&&void 0!==e.lt}function Vi(e){return!!e?.field&&void 0!==e.lte}function Gi(e){return!!e?.field&&void 0!==e.gt}function Yi(e){return!!e?.field&&void 0!==e.gte}function Xi(e){if(e?.field){if(t.isArray(e.range)&&2===e.range.length)return!0;if(yn(e.range))return!0}return!1}function Qi(e){return!!e?.field&&(t.isArray(e.oneOf)||t.isArray(e.in))}function Ji(e){return Qi(e)||Ii(e)||Xi(e)||Hi(e)||Gi(e)||Vi(e)||Yi(e)}function Ki(e,t){return wa(e,{timeUnit:t,wrapTime:!0})}function Zi(e){let t=!(arguments.length>1&&void 0!==arguments[1])||arguments[1];const{field:n}=e,i=Ui(e.timeUnit),{unit:r,binned:o}=i||{},a=oa(e,{expr:\"datum\"}),s=r?`time(${o?a:Li(r,n)})`:a;if(Ii(e))return`${s}===${Ki(e.equal,r)}`;if(Hi(e)){return`${s}<${Ki(e.lt,r)}`}if(Gi(e)){return`${s}>${Ki(e.gt,r)}`}if(Vi(e)){return`${s}<=${Ki(e.lte,r)}`}if(Yi(e)){return`${s}>=${Ki(e.gte,r)}`}if(Qi(e))return`indexof([${function(e,t){return e.map((e=>Ki(e,t)))}(e.oneOf,r).join(\",\")}], ${s}) !== -1`;if(function(e){return!!e?.field&&void 0!==e.valid}(e))return er(s,e.valid);if(Xi(e)){const{range:n}=e,i=yn(n)?{signal:`${n.signal}[0]`}:n[0],o=yn(n)?{signal:`${n.signal}[1]`}:n[1];if(null!==i&&null!==o&&t)return\"inrange(\"+s+\", [\"+Ki(i,r)+\", \"+Ki(o,r)+\"])\";const a=[];return null!==i&&a.push(`${s} >= ${Ki(i,r)}`),null!==o&&a.push(`${s} <= ${Ki(o,r)}`),a.length>0?a.join(\" && \"):\"true\"}throw new Error(`Invalid field predicate: ${X(e)}`)}function er(e){return!(arguments.length>1&&void 0!==arguments[1])||arguments[1]?`isValid(${e}) && isFinite(+${e})`:`!isValid(${e}) || !isFinite(+${e})`}function tr(e){return Ji(e)&&e.timeUnit?{...e,timeUnit:Ui(e.timeUnit)}:e}function nr(e){return\"quantitative\"===e||\"temporal\"===e}function ir(e){return\"ordinal\"===e||\"nominal\"===e}const rr=\"quantitative\",or=\"ordinal\",ar=\"temporal\",sr=\"nominal\",lr=\"geojson\";const cr={LINEAR:\"linear\",LOG:\"log\",POW:\"pow\",SQRT:\"sqrt\",SYMLOG:\"symlog\",IDENTITY:\"identity\",SEQUENTIAL:\"sequential\",TIME:\"time\",UTC:\"utc\",QUANTILE:\"quantile\",QUANTIZE:\"quantize\",THRESHOLD:\"threshold\",BIN_ORDINAL:\"bin-ordinal\",ORDINAL:\"ordinal\",POINT:\"point\",BAND:\"band\"},ur={linear:\"numeric\",log:\"numeric\",pow:\"numeric\",sqrt:\"numeric\",symlog:\"numeric\",identity:\"numeric\",sequential:\"numeric\",time:\"time\",utc:\"time\",ordinal:\"ordinal\",\"bin-ordinal\":\"bin-ordinal\",point:\"ordinal-position\",band:\"ordinal-position\",quantile:\"discretizing\",quantize:\"discretizing\",threshold:\"discretizing\"};function fr(e,t){const n=ur[e],i=ur[t];return n===i||\"ordinal-position\"===n&&\"time\"===i||\"ordinal-position\"===i&&\"time\"===n}const dr={linear:0,log:1,pow:1,sqrt:1,symlog:1,identity:1,sequential:1,time:0,utc:0,point:10,band:11,ordinal:0,\"bin-ordinal\":0,quantile:0,quantize:0,threshold:0};function mr(e){return dr[e]}const pr=new Set([\"linear\",\"log\",\"pow\",\"sqrt\",\"symlog\"]),gr=new Set([...pr,\"time\",\"utc\"]);function hr(e){return pr.has(e)}const yr=new Set([\"quantile\",\"quantize\",\"threshold\"]),vr=new Set([...gr,...yr,\"sequential\",\"identity\"]),br=new Set([\"ordinal\",\"bin-ordinal\",\"point\",\"band\"]);function xr(e){return br.has(e)}function $r(e){return vr.has(e)}function wr(e){return gr.has(e)}function kr(e){return yr.has(e)}function Sr(e){return e?.param}const{type:Dr,domain:Fr,range:zr,rangeMax:Or,rangeMin:_r,scheme:Nr,...Cr}={type:1,domain:1,domainMax:1,domainMin:1,domainMid:1,domainRaw:1,align:1,range:1,rangeMax:1,rangeMin:1,scheme:1,bins:1,reverse:1,round:1,clamp:1,nice:1,base:1,exponent:1,constant:1,interpolate:1,zero:1,padding:1,paddingInner:1,paddingOuter:1},Pr=D(Cr);function Ar(e,t){switch(t){case\"type\":case\"domain\":case\"reverse\":case\"range\":return!0;case\"scheme\":case\"interpolate\":return![\"point\",\"band\",\"identity\"].includes(e);case\"bins\":return![\"point\",\"band\",\"identity\",\"ordinal\"].includes(e);case\"round\":return wr(e)||\"band\"===e||\"point\"===e;case\"padding\":case\"rangeMin\":case\"rangeMax\":return wr(e)||[\"point\",\"band\"].includes(e);case\"paddingOuter\":case\"align\":return[\"point\",\"band\"].includes(e);case\"paddingInner\":return\"band\"===e;case\"domainMax\":case\"domainMid\":case\"domainMin\":case\"domainRaw\":case\"clamp\":return wr(e);case\"nice\":return wr(e)||\"quantize\"===e||\"threshold\"===e;case\"exponent\":return\"pow\"===e;case\"base\":return\"log\"===e;case\"constant\":return\"symlog\"===e;case\"zero\":return $r(e)&&!p([\"log\",\"time\",\"utc\",\"threshold\",\"quantile\"],e)}}function jr(e,t){switch(t){case\"interpolate\":case\"scheme\":case\"domainMid\":return qe(e)?void 0:`Cannot use the scale property \"${t}\" with non-color channel.`;case\"align\":case\"type\":case\"bins\":case\"domain\":case\"domainMax\":case\"domainMin\":case\"domainRaw\":case\"range\":case\"base\":case\"exponent\":case\"constant\":case\"nice\":case\"padding\":case\"paddingInner\":case\"paddingOuter\":case\"rangeMax\":case\"rangeMin\":case\"reverse\":case\"round\":case\"clamp\":case\"zero\":return}}const Tr={arc:\"arc\",area:\"area\",bar:\"bar\",image:\"image\",line:\"line\",point:\"point\",rect:\"rect\",rule:\"rule\",text:\"text\",tick:\"tick\",trail:\"trail\",circle:\"circle\",square:\"square\",geoshape:\"geoshape\"},Er=Tr.arc,Mr=Tr.area,Lr=Tr.bar,qr=Tr.image,Ur=Tr.line,Rr=Tr.point,Wr=Tr.rect,Br=Tr.rule,Ir=Tr.text,Hr=Tr.tick,Vr=Tr.trail,Gr=Tr.circle,Yr=Tr.square,Xr=Tr.geoshape;function Qr(e){return[\"line\",\"area\",\"trail\"].includes(e)}function Jr(e){return[\"rect\",\"bar\",\"image\",\"arc\"].includes(e)}const Kr=new Set(D(Tr));function Zr(e){return e.type}const eo=[\"stroke\",\"strokeWidth\",\"strokeDash\",\"strokeDashOffset\",\"strokeOpacity\",\"strokeJoin\",\"strokeMiterLimit\",\"fill\",\"fillOpacity\"],to=D({color:1,filled:1,invalid:1,order:1,radius2:1,theta2:1,timeUnitBandSize:1,timeUnitBandPosition:1}),no=D({mark:1,arc:1,area:1,bar:1,circle:1,image:1,line:1,point:1,rect:1,rule:1,square:1,text:1,tick:1,trail:1,geoshape:1});function io(e){return e&&null!=e.band}const ro={horizontal:[\"cornerRadiusTopRight\",\"cornerRadiusBottomRight\"],vertical:[\"cornerRadiusTopLeft\",\"cornerRadiusTopRight\"]},oo={binSpacing:1,continuousBandSize:5,minBandSize:.25,timeUnitBandPosition:.5},ao={binSpacing:0,continuousBandSize:5,minBandSize:.25,timeUnitBandPosition:.5};function so(e){const{channel:t,channelDef:n,markDef:i,scale:r,config:o}=e,a=mo(e);return Ho(n)&&!rn(n.aggregate)&&r&&wr(r.get(\"type\"))?function(e){let{fieldDef:t,channel:n,markDef:i,ref:r,config:o}=e;if(Qr(i.type))return r;const a=Cn(\"invalid\",i,o);if(null===a)return[lo(t,n),r];return r}({fieldDef:n,channel:t,markDef:i,ref:a,config:o}):a}function lo(e,t){return{test:co(e,!0),...\"y\"===tt(t)?{field:{group:\"height\"}}:{value:0}}}function co(e){let n=!(arguments.length>1&&void 0!==arguments[1])||arguments[1];return er(t.isString(e)?e:oa(e,{expr:\"datum\"}),!n)}function uo(e,t,n,i){const r={};if(t&&(r.scale=t),Go(e)){const{datum:t}=e;wi(t)?r.signal=Oi(t):yn(t)?r.signal=t.signal:mn(t)?r.signal=t.expr:r.value=t}else r.field=oa(e,n);if(i){const{offset:e,band:t}=i;e&&(r.offset=e),t&&(r.band=t)}return r}function fo(e){let{scaleName:t,fieldOrDatumDef:n,fieldOrDatumDef2:i,offset:r,startSuffix:o,endSuffix:a=\"end\",bandPosition:s=.5}=e;const l=!yn(s)&&01&&void 0!==arguments[1]?arguments[1]:{},n=e.field;const i=t.prefix;let r=t.suffix,o=\"\";if(function(e){return\"count\"===e.aggregate}(e))n=B(\"count\");else{let i;if(!t.nofn)if(function(e){return\"op\"in e}(e))i=e.op;else{const{bin:a,aggregate:s,timeUnit:l}=e;ln(a)?(i=sn(a),r=(t.binSuffix??\"\")+(t.suffix??\"\")):s?en(s)?(o=`[\"${n}\"]`,n=`argmax_${s.argmax}`):Zt(s)?(o=`[\"${n}\"]`,n=`argmin_${s.argmin}`):i=String(s):l&&!Ci(l)&&(i=function(e){const{utc:t,...n}=Ui(e);return n.unit?(t?\"utc\":\"\")+D(n).map((e=>_(`${\"unit\"===e?\"\":`_${e}_`}${n[e]}`))).join(\"\"):(t?\"utc\":\"\")+\"timeunit\"+D(n).map((e=>_(`_${e}_${n[e]}`))).join(\"\")}(l),r=(![\"range\",\"mid\"].includes(t.binSuffix)&&t.binSuffix||\"\")+(t.suffix??\"\"))}i&&(n=n?`${i}_${n}`:i)}return r&&(n=`${n}_${r}`),i&&(n=`${i}_${n}`),t.forAs?L(n):t.expr?j(n,t.expr)+o:E(n)+o}function aa(e){switch(e.type){case\"nominal\":case\"ordinal\":case\"geojson\":return!0;case\"quantitative\":return Ho(e)&&!!e.bin;case\"temporal\":return!1}throw new Error(ii(e.type))}const sa=(e,t)=>{switch(t.fieldTitle){case\"plain\":return e.field;case\"functional\":return function(e){const{aggregate:t,bin:n,timeUnit:i,field:r}=e;if(en(t))return`${r} for argmax(${t.argmax})`;if(Zt(t))return`${r} for argmin(${t.argmin})`;const o=i&&!Ci(i)?Ui(i):void 0,a=t||o?.unit||o?.maxbins&&\"timeunit\"||ln(n)&&\"bin\";return a?`${a.toUpperCase()}(${r})`:r}(e);default:return function(e,t){const{field:n,bin:i,timeUnit:r,aggregate:o}=e;if(\"count\"===o)return t.countTitle;if(ln(i))return`${n} (binned)`;if(r&&!Ci(r)){const e=Ui(r)?.unit;if(e)return`${n} (${Ti(e).join(\"-\")})`}else if(o)return en(o)?`${n} for max ${o.argmax}`:Zt(o)?`${n} for min ${o.argmin}`:`${P(o)} of ${n}`;return n}(e,t)}};let la=sa;function ca(e){la=e}function ua(e,t,n){let{allowDisabling:i,includeDefault:r=!0}=n;const o=fa(e)?.title;if(!Ho(e))return o??e.title;const a=e,s=r?da(a,t):void 0;return i?U(o,a.title,s):o??a.title??s}function fa(e){return ta(e)&&e.axis?e.axis:na(e)&&e.legend?e.legend:jo(e)&&e.header?e.header:void 0}function da(e,t){return la(e,t)}function ma(e){if(ia(e)){const{format:t,formatType:n}=e;return{format:t,formatType:n}}{const t=fa(e)??{},{format:n,formatType:i}=t;return{format:n,formatType:i}}}function pa(e){return Ho(e)?e:Bo(e)?e.condition:void 0}function ga(e){return Jo(e)?e:Io(e)?e.condition:void 0}function ha(e,n,i){let r=arguments.length>3&&void 0!==arguments[3]?arguments[3]:{};if(t.isString(e)||t.isNumber(e)||t.isBoolean(e)){return $i(function(e,t,n){return`Channel ${e} is a ${t}. Converted to {value: ${X(n)}}.`}(n,t.isString(e)?\"string\":t.isNumber(e)?\"number\":\"boolean\",e)),{value:e}}return Jo(e)?ya(e,n,i,r):Io(e)?{...e,condition:ya(e.condition,n,i,r)}:e}function ya(e,n,i,r){if(ia(e)){const{format:t,formatType:o,...a}=e;if(go(o)&&!i.customFormatTypes)return $i(ti(n)),ya(a,n,i,r)}else{const t=ta(e)?\"axis\":na(e)?\"legend\":jo(e)?\"header\":null;if(t&&e[t]){const{format:o,formatType:a,...s}=e[t];if(go(a)&&!i.customFormatTypes)return $i(ti(n)),ya({...e,[t]:s},n,i,r)}}return Ho(e)?va(e,n,r):function(e){let n=e.type;if(n)return e;const{datum:i}=e;return n=t.isNumber(i)?\"quantitative\":t.isString(i)?\"nominal\":wi(i)?\"temporal\":void 0,{...e,type:n}}(e)}function va(e,n){let{compositeMark:i=!1}=arguments.length>2&&void 0!==arguments[2]?arguments[2]:{};const{aggregate:r,timeUnit:o,bin:a,field:s}=e,l={...e};if(i||!r||tn(r)||en(r)||Zt(r)||($i(function(e){return`Invalid aggregation operator \"${e}\".`}(r)),delete l.aggregate),o&&(l.timeUnit=Ui(o)),s&&(l.field=`${s}`),ln(a)&&(l.bin=ba(a,n)),cn(a)&&!zt(n)&&$i(function(e){return`Channel ${e} should not be used with \"binned\" bin.`}(n)),Ko(l)){const{type:e}=l,t=function(e){if(e)switch(e=e.toLowerCase()){case\"q\":case rr:return\"quantitative\";case\"t\":case ar:return\"temporal\";case\"o\":case or:return\"ordinal\";case\"n\":case sr:return\"nominal\";case lr:return\"geojson\"}}(e);e!==t&&(l.type=t),\"quantitative\"!==e&&rn(r)&&($i(function(e,t){return`Invalid field type \"${e}\" for aggregate: \"${t}\", using \"quantitative\" instead.`}(e,r)),l.type=\"quantitative\")}else if(!et(n)){const e=function(e,n){switch(n){case\"latitude\":case\"longitude\":return\"quantitative\";case\"row\":case\"column\":case\"facet\":case\"shape\":case\"strokeDash\":return\"nominal\";case\"order\":return\"ordinal\"}if(Mo(e)&&t.isArray(e.sort))return\"ordinal\";const{aggregate:i,bin:r,timeUnit:o}=e;if(o)return\"temporal\";if(r||i&&!en(i)&&!Zt(i))return\"quantitative\";if(ea(e)&&e.scale?.type)switch(ur[e.scale.type]){case\"numeric\":case\"discretizing\":return\"quantitative\";case\"time\":return\"temporal\"}return\"nominal\"}(l,n);l.type=e}if(Ko(l)){const{compatible:e,warning:t}=function(e,t){const n=e.type;if(\"geojson\"===n&&\"shape\"!==t)return{compatible:!1,warning:`Channel ${t} should not be used with a geojson data.`};switch(t){case Q:case J:case K:return aa(e)?xa:{compatible:!1,warning:si(t)};case Z:case ee:case ie:case re:case me:case pe:case ge:case Se:case Fe:case ze:case Oe:case _e:case Ne:case ve:case se:case oe:case Ce:return xa;case ue:case de:case ce:case fe:return n!==rr?{compatible:!1,warning:`Channel ${t} should be used with a quantitative field only, not ${e.type} field.`}:xa;case be:case xe:case $e:case we:case ye:case le:case ae:case te:case ne:return\"nominal\"!==n||e.sort?xa:{compatible:!1,warning:`Channel ${t} should not be used with an unsorted discrete field.`};case he:case ke:return aa(e)||ea(i=e)&&kr(i.scale?.type)?xa:{compatible:!1,warning:li(t)};case De:return\"nominal\"!==e.type||\"sort\"in e?xa:{compatible:!1,warning:\"Channel order is inappropriate for nominal field, which has no inherent order.\"}}var i}(l,n)||{};!1===e&&$i(t)}if(Mo(l)&&t.isString(l.sort)){const{sort:e}=l;if(_o(e))return{...l,sort:{encoding:e}};const t=e.substr(1);if(\"-\"===e.charAt(0)&&_o(t))return{...l,sort:{encoding:t,order:\"descending\"}}}if(jo(l)){const{header:e}=l;if(e){const{orient:t,...n}=e;if(t)return{...l,header:{...n,labelOrient:e.labelOrient||t,titleOrient:e.titleOrient||t}}}}return l}function ba(e,n){return t.isBoolean(e)?{maxbins:dn(n)}:\"binned\"===e?{binned:!0}:e.maxbins||e.step?e:{...e,maxbins:dn(n)}}const xa={compatible:!0};function $a(e){const{formatType:t}=ma(e);return\"time\"===t||!t&&((n=e)&&(\"temporal\"===n.type||Ho(n)&&!!n.timeUnit));var n}function wa(e,n){let{timeUnit:i,type:r,wrapTime:o,undefinedIfExprNotRequired:a}=n;const s=i&&Ui(i)?.unit;let l,c=s||\"temporal\"===r;return mn(e)?l=e.expr:yn(e)?l=e.signal:wi(e)?(c=!0,l=Oi(e)):(t.isString(e)||t.isNumber(e))&&c&&(l=`datetime(${X(e)})`,function(e){return!!_i[e]}(s)&&(t.isNumber(e)&&e<1e4||t.isString(e)&&isNaN(Date.parse(e)))&&(l=Oi({[s]:e}))),l?o&&c?`time(${l})`:l:a?void 0:X(e)}function ka(e,t){const{type:n}=e;return t.map((t=>{const i=wa(t,{timeUnit:Ho(e)&&!Ci(e.timeUnit)?e.timeUnit:void 0,type:n,undefinedIfExprNotRequired:!0});return void 0!==i?{signal:i}:t}))}function Sa(e,t){return ln(e.bin)?Ht(t)&&[\"ordinal\",\"nominal\"].includes(e.type):(console.warn(\"Only call this method for binned field defs.\"),!1)}const Da={labelAlign:{part:\"labels\",vgProp:\"align\"},labelBaseline:{part:\"labels\",vgProp:\"baseline\"},labelColor:{part:\"labels\",vgProp:\"fill\"},labelFont:{part:\"labels\",vgProp:\"font\"},labelFontSize:{part:\"labels\",vgProp:\"fontSize\"},labelFontStyle:{part:\"labels\",vgProp:\"fontStyle\"},labelFontWeight:{part:\"labels\",vgProp:\"fontWeight\"},labelOpacity:{part:\"labels\",vgProp:\"opacity\"},labelOffset:null,labelPadding:null,gridColor:{part:\"grid\",vgProp:\"stroke\"},gridDash:{part:\"grid\",vgProp:\"strokeDash\"},gridDashOffset:{part:\"grid\",vgProp:\"strokeDashOffset\"},gridOpacity:{part:\"grid\",vgProp:\"opacity\"},gridWidth:{part:\"grid\",vgProp:\"strokeWidth\"},tickColor:{part:\"ticks\",vgProp:\"stroke\"},tickDash:{part:\"ticks\",vgProp:\"strokeDash\"},tickDashOffset:{part:\"ticks\",vgProp:\"strokeDashOffset\"},tickOpacity:{part:\"ticks\",vgProp:\"opacity\"},tickSize:null,tickWidth:{part:\"ticks\",vgProp:\"strokeWidth\"}};function Fa(e){return e?.condition}const za=[\"domain\",\"grid\",\"labels\",\"ticks\",\"title\"],Oa={grid:\"grid\",gridCap:\"grid\",gridColor:\"grid\",gridDash:\"grid\",gridDashOffset:\"grid\",gridOpacity:\"grid\",gridScale:\"grid\",gridWidth:\"grid\",orient:\"main\",bandPosition:\"both\",aria:\"main\",description:\"main\",domain:\"main\",domainCap:\"main\",domainColor:\"main\",domainDash:\"main\",domainDashOffset:\"main\",domainOpacity:\"main\",domainWidth:\"main\",format:\"main\",formatType:\"main\",labelAlign:\"main\",labelAngle:\"main\",labelBaseline:\"main\",labelBound:\"main\",labelColor:\"main\",labelFlush:\"main\",labelFlushOffset:\"main\",labelFont:\"main\",labelFontSize:\"main\",labelFontStyle:\"main\",labelFontWeight:\"main\",labelLimit:\"main\",labelLineHeight:\"main\",labelOffset:\"main\",labelOpacity:\"main\",labelOverlap:\"main\",labelPadding:\"main\",labels:\"main\",labelSeparation:\"main\",maxExtent:\"main\",minExtent:\"main\",offset:\"both\",position:\"main\",tickCap:\"main\",tickColor:\"main\",tickDash:\"main\",tickDashOffset:\"main\",tickMinStep:\"both\",tickOffset:\"both\",tickOpacity:\"main\",tickRound:\"both\",ticks:\"main\",tickSize:\"main\",tickWidth:\"both\",title:\"main\",titleAlign:\"main\",titleAnchor:\"main\",titleAngle:\"main\",titleBaseline:\"main\",titleColor:\"main\",titleFont:\"main\",titleFontSize:\"main\",titleFontStyle:\"main\",titleFontWeight:\"main\",titleLimit:\"main\",titleLineHeight:\"main\",titleOpacity:\"main\",titlePadding:\"main\",titleX:\"main\",titleY:\"main\",encode:\"both\",scale:\"both\",tickBand:\"both\",tickCount:\"both\",tickExtra:\"both\",translate:\"both\",values:\"both\",zindex:\"both\"},_a={orient:1,aria:1,bandPosition:1,description:1,domain:1,domainCap:1,domainColor:1,domainDash:1,domainDashOffset:1,domainOpacity:1,domainWidth:1,format:1,formatType:1,grid:1,gridCap:1,gridColor:1,gridDash:1,gridDashOffset:1,gridOpacity:1,gridWidth:1,labelAlign:1,labelAngle:1,labelBaseline:1,labelBound:1,labelColor:1,labelFlush:1,labelFlushOffset:1,labelFont:1,labelFontSize:1,labelFontStyle:1,labelFontWeight:1,labelLimit:1,labelLineHeight:1,labelOffset:1,labelOpacity:1,labelOverlap:1,labelPadding:1,labels:1,labelSeparation:1,maxExtent:1,minExtent:1,offset:1,position:1,tickBand:1,tickCap:1,tickColor:1,tickCount:1,tickDash:1,tickDashOffset:1,tickExtra:1,tickMinStep:1,tickOffset:1,tickOpacity:1,tickRound:1,ticks:1,tickSize:1,tickWidth:1,title:1,titleAlign:1,titleAnchor:1,titleAngle:1,titleBaseline:1,titleColor:1,titleFont:1,titleFontSize:1,titleFontStyle:1,titleFontWeight:1,titleLimit:1,titleLineHeight:1,titleOpacity:1,titlePadding:1,titleX:1,titleY:1,translate:1,values:1,zindex:1},Na={..._a,style:1,labelExpr:1,encoding:1};function Ca(e){return!!Na[e]}const Pa=D({axis:1,axisBand:1,axisBottom:1,axisDiscrete:1,axisLeft:1,axisPoint:1,axisQuantitative:1,axisRight:1,axisTemporal:1,axisTop:1,axisX:1,axisXBand:1,axisXDiscrete:1,axisXPoint:1,axisXQuantitative:1,axisXTemporal:1,axisY:1,axisYBand:1,axisYDiscrete:1,axisYPoint:1,axisYQuantitative:1,axisYTemporal:1});function Aa(e){return\"mark\"in e}class ja{constructor(e,t){this.name=e,this.run=t}hasMatchingType(e){return!!Aa(e)&&(Zr(t=e.mark)?t.type:t)===this.name;var t}}function Ta(e,n){const i=e&&e[n];return!!i&&(t.isArray(i)?g(i,(e=>!!e.field)):Ho(i)||Bo(i))}function Ea(e,n){const i=e&&e[n];return!!i&&(t.isArray(i)?g(i,(e=>!!e.field)):Ho(i)||Go(i)||Io(i))}function Ma(e,t){if(zt(t)){const n=e[t];if((Ho(n)||Go(n))&&(ir(n.type)||Ho(n)&&n.timeUnit)){return Ea(e,at(t))}}return!1}function La(e){return g(Be,(n=>{if(Ta(e,n)){const i=e[n];if(t.isArray(i))return g(i,(e=>!!e.aggregate));{const e=pa(i);return e&&!!e.aggregate}}return!1}))}function qa(e,t){const n=[],i=[],r=[],o=[],a={};return Wa(e,((s,l)=>{if(Ho(s)){const{field:c,aggregate:u,bin:f,timeUnit:d,...m}=s;if(u||d||f){const e=fa(s),p=e?.title;let g=oa(s,{forAs:!0});const h={...p?[]:{title:ua(s,t,{allowDisabling:!0})},...m,field:g};if(u){let e;if(en(u)?(e=\"argmax\",g=oa({op:\"argmax\",field:u.argmax},{forAs:!0}),h.field=`${g}.${c}`):Zt(u)?(e=\"argmin\",g=oa({op:\"argmin\",field:u.argmin},{forAs:!0}),h.field=`${g}.${c}`):\"boxplot\"!==u&&\"errorbar\"!==u&&\"errorband\"!==u&&(e=u),e){const t={op:e,as:g};c&&(t.field=c),o.push(t)}}else if(n.push(g),Ko(s)&&ln(f)){if(i.push({bin:f,field:c,as:g}),n.push(oa(s,{binSuffix:\"end\"})),Sa(s,l)&&n.push(oa(s,{binSuffix:\"range\"})),zt(l)){const e={field:`${g}_end`};a[`${l}2`]=e}h.bin=\"binned\",et(l)||(h.type=rr)}else if(d&&!Ci(d)){r.push({timeUnit:d,field:c,as:g});const e=Ko(s)&&s.type!==ar&&\"time\";e&&(l===Se||l===Oe?h.formatType=e:!function(e){return!!kt[e]}(l)?zt(l)&&(h.axis={formatType:e,...h.axis}):h.legend={formatType:e,...h.legend})}a[l]=h}else n.push(c),a[l]=e[l]}else a[l]=e[l]})),{bins:i,timeUnits:r,aggregate:o,groupby:n,encoding:a}}function Ua(e,t,n){const i=Vt(t,n);if(!i)return!1;if(\"binned\"===i){const n=e[t===te?Z:ee];return!!(Ho(n)&&Ho(e[t])&&cn(n.bin))}return!0}function Ra(e,t){const n={};for(const i of D(e)){const r=ha(e[i],i,t,{compositeMark:!0});n[i]=r}return n}function Wa(e,n,i){if(e)for(const r of D(e)){const o=e[r];if(t.isArray(o))for(const e of o)n.call(i,e,r);else n.call(i,o,r)}}function Ba(e,n){return D(n).reduce(((i,r)=>{switch(r){case Z:case ee:case _e:case Ce:case Ne:case te:case ne:case ie:case re:case se:case le:case oe:case ae:case ce:case ue:case fe:case de:case Se:case he:case ve:case Oe:return i;case De:if(\"line\"===e||\"trail\"===e)return i;case Fe:case ze:{const e=n[r];if(t.isArray(e)||Ho(e))for(const n of t.array(e))n.aggregate||i.push(oa(n,{}));return i}case ye:if(\"trail\"===e)return i;case me:case pe:case ge:case be:case xe:case $e:case ke:case we:{const e=pa(n[r]);return e&&!e.aggregate&&i.push(oa(e,{})),i}}}),[])}function Ia(e,n,i){let r=!(arguments.length>3&&void 0!==arguments[3])||arguments[3];if(\"tooltip\"in i)return{tooltip:i.tooltip};return{tooltip:[...e.map((e=>{let{fieldPrefix:t,titlePrefix:i}=e;const o=r?` of ${Ha(n)}`:\"\";return{field:t+n.field,type:n.type,title:yn(i)?{signal:`${i}\"${escape(o)}\"`}:i+o}})),...b(function(e){const n=[];for(const i of D(e))if(Ta(e,i)){const r=e[i],o=t.array(r);for(const e of o)Ho(e)?n.push(e):Bo(e)&&n.push(e.condition)}return n}(i).map(ra),d)]}}function Ha(e){const{title:t,field:n}=e;return U(t,n)}function Va(e,n,i,r,o){const{scale:a,axis:s}=i;return l=>{let{partName:c,mark:u,positionPrefix:f,endPositionPrefix:d,extraEncoding:m={}}=l;const p=Ha(i);return Ga(e,c,o,{mark:u,encoding:{[n]:{field:`${f}_${i.field}`,type:i.type,...void 0!==p?{title:p}:{},...void 0!==a?{scale:a}:{},...void 0!==s?{axis:s}:{}},...t.isString(d)?{[`${n}2`]:{field:`${d}_${i.field}`}}:{},...r,...m}})}}function Ga(e,n,i,r){const{clip:o,color:a,opacity:s}=e,l=e.type;return e[n]||void 0===e[n]&&i[n]?[{...r,mark:{...i[n],...o?{clip:o}:{},...a?{color:a}:{},...s?{opacity:s}:{},...Zr(r.mark)?r.mark:{type:r.mark},style:`${l}-${String(n)}`,...t.isBoolean(e[n])?{}:e[n]}}]:[]}function Ya(e,t,n){const{encoding:i}=e,r=\"vertical\"===t?\"y\":\"x\",o=i[r],a=i[`${r}2`],s=i[`${r}Error`],l=i[`${r}Error2`];return{continuousAxisChannelDef:Xa(o,n),continuousAxisChannelDef2:Xa(a,n),continuousAxisChannelDefError:Xa(s,n),continuousAxisChannelDefError2:Xa(l,n),continuousAxis:r}}function Xa(e,t){if(e?.aggregate){const{aggregate:n,...i}=e;return n!==t&&$i(function(e,t){return`Continuous axis should not have customized aggregation function ${e}; ${t} already agregates the axis.`}(n,t)),i}return e}function Qa(e,t){const{mark:n,encoding:i}=e,{x:r,y:o}=i;if(Zr(n)&&n.orient)return n.orient;if(Yo(r)){if(Yo(o)){const e=Ho(r)&&r.aggregate,n=Ho(o)&&o.aggregate;if(e||n!==t){if(n||e!==t){if(e===t&&n===t)throw new Error(\"Both x and y cannot have aggregate\");return $a(o)&&!$a(r)?\"horizontal\":\"vertical\"}return\"horizontal\"}return\"vertical\"}return\"horizontal\"}if(Yo(o))return\"vertical\";throw new Error(`Need a valid continuous axis for ${t}s`)}const Ja=\"boxplot\",Ka=new ja(Ja,es);function Za(e){return t.isNumber(e)?\"tukey\":e}function es(e,n){let{config:i}=n;e={...e,encoding:Ra(e.encoding,i)};const{mark:r,encoding:o,params:a,projection:s,...l}=e,c=Zr(r)?r:{type:r};a&&$i(Jn(\"boxplot\"));const u=c.extent??i.boxplot.extent,d=Cn(\"size\",c,i),m=c.invalid,p=Za(u),{bins:g,timeUnits:h,transform:y,continuousAxisChannelDef:v,continuousAxis:b,groupby:x,aggregate:$,encodingWithoutContinuousAxis:w,ticksOrient:k,boxOrient:D,customTooltipWithoutAggregatedField:F}=function(e,n,i){const r=Qa(e,Ja),{continuousAxisChannelDef:o,continuousAxis:a}=Ya(e,r,Ja),s=o.field,l=Za(n),c=[...ts(s),{op:\"median\",field:s,as:`mid_box_${s}`},{op:\"min\",field:s,as:(\"min-max\"===l?\"lower_whisker_\":\"min_\")+s},{op:\"max\",field:s,as:(\"min-max\"===l?\"upper_whisker_\":\"max_\")+s}],u=\"min-max\"===l||\"tukey\"===l?[]:[{calculate:`datum[\"upper_box_${s}\"] - datum[\"lower_box_${s}\"]`,as:`iqr_${s}`},{calculate:`min(datum[\"upper_box_${s}\"] + datum[\"iqr_${s}\"] * ${n}, datum[\"max_${s}\"])`,as:`upper_whisker_${s}`},{calculate:`max(datum[\"lower_box_${s}\"] - datum[\"iqr_${s}\"] * ${n}, datum[\"min_${s}\"])`,as:`lower_whisker_${s}`}],{[a]:f,...d}=e.encoding,{customTooltipWithoutAggregatedField:m,filteredEncoding:p}=function(e){const{tooltip:n,...i}=e;if(!n)return{filteredEncoding:i};let r,o;if(t.isArray(n)){for(const e of n)e.aggregate?(r||(r=[]),r.push(e)):(o||(o=[]),o.push(e));r&&(i.tooltip=r)}else n.aggregate?i.tooltip=n:o=n;return t.isArray(o)&&1===o.length&&(o=o[0]),{customTooltipWithoutAggregatedField:o,filteredEncoding:i}}(d),{bins:g,timeUnits:h,aggregate:y,groupby:v,encoding:b}=qa(p,i),x=\"vertical\"===r?\"horizontal\":\"vertical\",$=r,w=[...g,...h,{aggregate:[...y,...c],groupby:v},...u];return{bins:g,timeUnits:h,transform:w,groupby:v,aggregate:y,continuousAxisChannelDef:o,continuousAxis:a,encodingWithoutContinuousAxis:b,ticksOrient:x,boxOrient:$,customTooltipWithoutAggregatedField:m}}(e,u,i),{color:z,size:O,..._}=w,N=e=>Va(c,b,v,e,i.boxplot),C=N(_),P=N(w),A=N({..._,...O?{size:O}:{}}),j=Ia([{fieldPrefix:\"min-max\"===p?\"upper_whisker_\":\"max_\",titlePrefix:\"Max\"},{fieldPrefix:\"upper_box_\",titlePrefix:\"Q3\"},{fieldPrefix:\"mid_box_\",titlePrefix:\"Median\"},{fieldPrefix:\"lower_box_\",titlePrefix:\"Q1\"},{fieldPrefix:\"min-max\"===p?\"lower_whisker_\":\"min_\",titlePrefix:\"Min\"}],v,w),T={type:\"tick\",color:\"black\",opacity:1,orient:k,invalid:m,aria:!1},E=\"min-max\"===p?j:Ia([{fieldPrefix:\"upper_whisker_\",titlePrefix:\"Upper Whisker\"},{fieldPrefix:\"lower_whisker_\",titlePrefix:\"Lower Whisker\"}],v,w),M=[...C({partName:\"rule\",mark:{type:\"rule\",invalid:m,aria:!1},positionPrefix:\"lower_whisker\",endPositionPrefix:\"lower_box\",extraEncoding:E}),...C({partName:\"rule\",mark:{type:\"rule\",invalid:m,aria:!1},positionPrefix:\"upper_box\",endPositionPrefix:\"upper_whisker\",extraEncoding:E}),...C({partName:\"ticks\",mark:T,positionPrefix:\"lower_whisker\",extraEncoding:E}),...C({partName:\"ticks\",mark:T,positionPrefix:\"upper_whisker\",extraEncoding:E})],L=[...\"tukey\"!==p?M:[],...P({partName:\"box\",mark:{type:\"bar\",...d?{size:d}:{},orient:D,invalid:m,ariaRoleDescription:\"box\"},positionPrefix:\"lower_box\",endPositionPrefix:\"upper_box\",extraEncoding:j}),...A({partName:\"median\",mark:{type:\"tick\",invalid:m,...t.isObject(i.boxplot.median)&&i.boxplot.median.color?{color:i.boxplot.median.color}:{},...d?{size:d}:{},orient:k,aria:!1},positionPrefix:\"mid_box\",extraEncoding:j})];if(\"min-max\"===p)return{...l,transform:(l.transform??[]).concat(y),layer:L};const q=`datum[\"lower_box_${v.field}\"]`,U=`datum[\"upper_box_${v.field}\"]`,R=`(${U} - ${q})`,W=`${q} - ${u} * ${R}`,B=`${U} + ${u} * ${R}`,I=`datum[\"${v.field}\"]`,H={joinaggregate:ts(v.field),groupby:x},V={transform:[{filter:`(${W} <= ${I}) && (${I} <= ${B})`},{aggregate:[{op:\"min\",field:v.field,as:`lower_whisker_${v.field}`},{op:\"max\",field:v.field,as:`upper_whisker_${v.field}`},{op:\"min\",field:`lower_box_${v.field}`,as:`lower_box_${v.field}`},{op:\"max\",field:`upper_box_${v.field}`,as:`upper_box_${v.field}`},...$],groupby:x}],layer:M},{tooltip:G,...Y}=_,{scale:X,axis:Q}=v,J=Ha(v),K=f(Q,[\"title\"]),Z=Ga(c,\"outliers\",i.boxplot,{transform:[{filter:`(${I} < ${W}) || (${I} > ${B})`}],mark:\"point\",encoding:{[b]:{field:v.field,type:v.type,...void 0!==J?{title:J}:{},...void 0!==X?{scale:X}:{},...S(K)?{}:{axis:K}},...Y,...z?{color:z}:{},...F?{tooltip:F}:{}}})[0];let ee;const te=[...g,...h,H];return Z?ee={transform:te,layer:[Z,V]}:(ee=V,ee.transform.unshift(...te)),{...l,layer:[ee,{transform:y,layer:L}]}}function ts(e){return[{op:\"q1\",field:e,as:`lower_box_${e}`},{op:\"q3\",field:e,as:`upper_box_${e}`}]}const ns=\"errorbar\",is=new ja(ns,rs);function rs(e,t){let{config:n}=t;e={...e,encoding:Ra(e.encoding,n)};const{transform:i,continuousAxisChannelDef:r,continuousAxis:o,encodingWithoutContinuousAxis:a,ticksOrient:s,markDef:l,outerSpec:c,tooltipEncoding:u}=as(e,ns,n);delete a.size;const f=Va(l,o,r,a,n.errorbar),d=l.thickness,m=l.size,p={type:\"tick\",orient:s,aria:!1,...void 0!==d?{thickness:d}:{},...void 0!==m?{size:m}:{}},g=[...f({partName:\"ticks\",mark:p,positionPrefix:\"lower\",extraEncoding:u}),...f({partName:\"ticks\",mark:p,positionPrefix:\"upper\",extraEncoding:u}),...f({partName:\"rule\",mark:{type:\"rule\",ariaRoleDescription:\"errorbar\",...void 0!==d?{size:d}:{}},positionPrefix:\"lower\",endPositionPrefix:\"upper\",extraEncoding:u})];return{...c,transform:i,...g.length>1?{layer:g}:{...g[0]}}}function os(e,t){const{encoding:n}=e;if(function(e){return(Jo(e.x)||Jo(e.y))&&!Jo(e.x2)&&!Jo(e.y2)&&!Jo(e.xError)&&!Jo(e.xError2)&&!Jo(e.yError)&&!Jo(e.yError2)}(n))return{orient:Qa(e,t),inputType:\"raw\"};const i=function(e){return Jo(e.x2)||Jo(e.y2)}(n),r=function(e){return Jo(e.xError)||Jo(e.xError2)||Jo(e.yError)||Jo(e.yError2)}(n),o=n.x,a=n.y;if(i){if(r)throw new Error(`${t} cannot be both type aggregated-upper-lower and aggregated-error`);const e=n.x2,i=n.y2;if(Jo(e)&&Jo(i))throw new Error(`${t} cannot have both x2 and y2`);if(Jo(e)){if(Yo(o))return{orient:\"horizontal\",inputType:\"aggregated-upper-lower\"};throw new Error(`Both x and x2 have to be quantitative in ${t}`)}if(Jo(i)){if(Yo(a))return{orient:\"vertical\",inputType:\"aggregated-upper-lower\"};throw new Error(`Both y and y2 have to be quantitative in ${t}`)}throw new Error(\"No ranged axis\")}{const e=n.xError,i=n.xError2,r=n.yError,s=n.yError2;if(Jo(i)&&!Jo(e))throw new Error(`${t} cannot have xError2 without xError`);if(Jo(s)&&!Jo(r))throw new Error(`${t} cannot have yError2 without yError`);if(Jo(e)&&Jo(r))throw new Error(`${t} cannot have both xError and yError with both are quantiative`);if(Jo(e)){if(Yo(o))return{orient:\"horizontal\",inputType:\"aggregated-error\"};throw new Error(\"All x, xError, and xError2 (if exist) have to be quantitative\")}if(Jo(r)){if(Yo(a))return{orient:\"vertical\",inputType:\"aggregated-error\"};throw new Error(\"All y, yError, and yError2 (if exist) have to be quantitative\")}throw new Error(\"No ranged axis\")}}function as(e,t,n){const{mark:i,encoding:r,params:o,projection:a,...s}=e,l=Zr(i)?i:{type:i};o&&$i(Jn(t));const{orient:c,inputType:u}=os(e,t),{continuousAxisChannelDef:f,continuousAxisChannelDef2:d,continuousAxisChannelDefError:m,continuousAxisChannelDefError2:p,continuousAxis:g}=Ya(e,c,t),{errorBarSpecificAggregate:h,postAggregateCalculates:y,tooltipSummary:v,tooltipTitleWithFieldName:b}=function(e,t,n,i,r,o,a,s){let l=[],c=[];const u=t.field;let f,d=!1;if(\"raw\"===o){const t=e.center?e.center:e.extent?\"iqr\"===e.extent?\"median\":\"mean\":s.errorbar.center,n=e.extent?e.extent:\"mean\"===t?\"stderr\":\"iqr\";if(\"median\"===t!=(\"iqr\"===n)&&$i(function(e,t,n){return`${e} is not usually used with ${t} for ${n}.`}(t,n,a)),\"stderr\"===n||\"stdev\"===n)l=[{op:n,field:u,as:`extent_${u}`},{op:t,field:u,as:`center_${u}`}],c=[{calculate:`datum[\"center_${u}\"] + datum[\"extent_${u}\"]`,as:`upper_${u}`},{calculate:`datum[\"center_${u}\"] - datum[\"extent_${u}\"]`,as:`lower_${u}`}],f=[{fieldPrefix:\"center_\",titlePrefix:P(t)},{fieldPrefix:\"upper_\",titlePrefix:ss(t,n,\"+\")},{fieldPrefix:\"lower_\",titlePrefix:ss(t,n,\"-\")}],d=!0;else{let e,t,i;\"ci\"===n?(e=\"mean\",t=\"ci0\",i=\"ci1\"):(e=\"median\",t=\"q1\",i=\"q3\"),l=[{op:t,field:u,as:`lower_${u}`},{op:i,field:u,as:`upper_${u}`},{op:e,field:u,as:`center_${u}`}],f=[{fieldPrefix:\"upper_\",titlePrefix:ua({field:u,aggregate:i,type:\"quantitative\"},s,{allowDisabling:!1})},{fieldPrefix:\"lower_\",titlePrefix:ua({field:u,aggregate:t,type:\"quantitative\"},s,{allowDisabling:!1})},{fieldPrefix:\"center_\",titlePrefix:ua({field:u,aggregate:e,type:\"quantitative\"},s,{allowDisabling:!1})}]}}else{(e.center||e.extent)&&$i((m=e.center,`${(p=e.extent)?\"extent \":\"\"}${p&&m?\"and \":\"\"}${m?\"center \":\"\"}${p&&m?\"are \":\"is \"}not needed when data are aggregated.`)),\"aggregated-upper-lower\"===o?(f=[],c=[{calculate:`datum[\"${n.field}\"]`,as:`upper_${u}`},{calculate:`datum[\"${u}\"]`,as:`lower_${u}`}]):\"aggregated-error\"===o&&(f=[{fieldPrefix:\"\",titlePrefix:u}],c=[{calculate:`datum[\"${u}\"] + datum[\"${i.field}\"]`,as:`upper_${u}`}],r?c.push({calculate:`datum[\"${u}\"] + datum[\"${r.field}\"]`,as:`lower_${u}`}):c.push({calculate:`datum[\"${u}\"] - datum[\"${i.field}\"]`,as:`lower_${u}`}));for(const e of c)f.push({fieldPrefix:e.as.substring(0,6),titlePrefix:M(M(e.calculate,'datum[\"',\"\"),'\"]',\"\")})}var m,p;return{postAggregateCalculates:c,errorBarSpecificAggregate:l,tooltipSummary:f,tooltipTitleWithFieldName:d}}(l,f,d,m,p,u,t,n),{[g]:x,[\"x\"===g?\"x2\":\"y2\"]:$,[\"x\"===g?\"xError\":\"yError\"]:w,[\"x\"===g?\"xError2\":\"yError2\"]:k,...S}=r,{bins:D,timeUnits:F,aggregate:z,groupby:O,encoding:_}=qa(S,n),N=[...z,...h],C=\"raw\"!==u?[]:O,A=Ia(v,f,_,b);return{transform:[...s.transform??[],...D,...F,...0===N.length?[]:[{aggregate:N,groupby:C}],...y],groupby:C,continuousAxisChannelDef:f,continuousAxis:g,encodingWithoutContinuousAxis:_,ticksOrient:\"vertical\"===c?\"horizontal\":\"vertical\",markDef:l,outerSpec:s,tooltipEncoding:A}}function ss(e,t,n){return`${P(e)} ${n} ${t}`}const ls=\"errorband\",cs=new ja(ls,us);function us(e,t){let{config:n}=t;e={...e,encoding:Ra(e.encoding,n)};const{transform:i,continuousAxisChannelDef:r,continuousAxis:o,encodingWithoutContinuousAxis:a,markDef:s,outerSpec:l,tooltipEncoding:c}=as(e,ls,n),u=s,f=Va(u,o,r,a,n.errorband),d=void 0!==e.encoding.x&&void 0!==e.encoding.y;let m={type:d?\"area\":\"rect\"},p={type:d?\"line\":\"rule\"};const g={...u.interpolate?{interpolate:u.interpolate}:{},...u.tension&&u.interpolate?{tension:u.tension}:{}};return d?(m={...m,...g,ariaRoleDescription:\"errorband\"},p={...p,...g,aria:!1}):u.interpolate?$i(yi(\"interpolate\")):u.tension&&$i(yi(\"tension\")),{...l,transform:i,layer:[...f({partName:\"band\",mark:m,positionPrefix:\"lower\",endPositionPrefix:\"upper\",extraEncoding:c}),...f({partName:\"borders\",mark:p,positionPrefix:\"lower\",extraEncoding:c}),...f({partName:\"borders\",mark:p,positionPrefix:\"upper\",extraEncoding:c})]}}const fs={};function ds(e,t,n){const i=new ja(e,t);fs[e]={normalizer:i,parts:n}}ds(Ja,es,[\"box\",\"median\",\"outliers\",\"rule\",\"ticks\"]),ds(ns,rs,[\"ticks\",\"rule\"]),ds(ls,us,[\"band\",\"borders\"]);const ms=[\"gradientHorizontalMaxLength\",\"gradientHorizontalMinLength\",\"gradientVerticalMaxLength\",\"gradientVerticalMinLength\",\"unselectedOpacity\"],ps={titleAlign:\"align\",titleAnchor:\"anchor\",titleAngle:\"angle\",titleBaseline:\"baseline\",titleColor:\"color\",titleFont:\"font\",titleFontSize:\"fontSize\",titleFontStyle:\"fontStyle\",titleFontWeight:\"fontWeight\",titleLimit:\"limit\",titleLineHeight:\"lineHeight\",titleOrient:\"orient\",titlePadding:\"offset\"},gs={labelAlign:\"align\",labelAnchor:\"anchor\",labelAngle:\"angle\",labelBaseline:\"baseline\",labelColor:\"color\",labelFont:\"font\",labelFontSize:\"fontSize\",labelFontStyle:\"fontStyle\",labelFontWeight:\"fontWeight\",labelLimit:\"limit\",labelLineHeight:\"lineHeight\",labelOrient:\"orient\",labelPadding:\"offset\"},hs=D(ps),ys=D(gs),vs=D({header:1,headerRow:1,headerColumn:1,headerFacet:1}),bs=[\"size\",\"shape\",\"fill\",\"stroke\",\"strokeDash\",\"strokeWidth\",\"opacity\"],xs=\"_vgsid_\",$s={point:{on:\"click\",fields:[xs],toggle:\"event.shiftKey\",resolve:\"global\",clear:\"dblclick\"},interval:{on:\"[pointerdown, window:pointerup] > window:pointermove!\",encodings:[\"x\",\"y\"],translate:\"[pointerdown, window:pointerup] > window:pointermove!\",zoom:\"wheel!\",mark:{fill:\"#333\",fillOpacity:.125,stroke:\"white\"},resolve:\"global\",clear:\"dblclick\"}};function ws(e){return\"legend\"===e||!!e?.legend}function ks(e){return ws(e)&&t.isObject(e)}function Ss(e){return!!e?.select}function Ds(e){const t=[];for(const n of e||[]){if(Ss(n))continue;const{expr:e,bind:i,...r}=n;if(i&&e){const n={...r,bind:i,init:e};t.push(n)}else{const n={...r,...e?{update:e}:{},...i?{bind:i}:{}};t.push(n)}}return t}function Fs(e){return\"concat\"in e}function zs(e){return\"vconcat\"in e}function Os(e){return\"hconcat\"in e}function _s(e){let{step:t,offsetIsDiscrete:n}=e;return n?t.for??\"offset\":\"position\"}function Ns(e){return t.isObject(e)&&void 0!==e.step}function Cs(e){return e.view||e.width||e.height}const Ps=D({align:1,bounds:1,center:1,columns:1,spacing:1});function As(e,t){return e[t]??e[\"width\"===t?\"continuousWidth\":\"continuousHeight\"]}function js(e,t){const n=Ts(e,t);return Ns(n)?n.step:Es}function Ts(e,t){return U(e[t]??e[\"width\"===t?\"discreteWidth\":\"discreteHeight\"],{step:e.step})}const Es=20,Ms={background:\"white\",padding:5,timeFormat:\"%b %d, %Y\",countTitle:\"Count of Records\",view:{continuousWidth:200,continuousHeight:200,step:Es},mark:{color:\"#4c78a8\",invalid:\"filter\",timeUnitBandSize:1},arc:{},area:{},bar:oo,circle:{},geoshape:{},image:{},line:{},point:{},rect:ao,rule:{color:\"black\"},square:{},text:{color:\"black\"},tick:{thickness:1},trail:{},boxplot:{size:14,extent:1.5,box:{},median:{color:\"white\"},outliers:{},rule:{},ticks:null},errorbar:{center:\"mean\",rule:!0,ticks:!1},errorband:{band:{opacity:.3},borders:!1},scale:{pointPadding:.5,barBandPaddingInner:.1,rectBandPaddingInner:0,bandWithNestedOffsetPaddingInner:.2,bandWithNestedOffsetPaddingOuter:.2,minBandSize:2,minFontSize:8,maxFontSize:40,minOpacity:.3,maxOpacity:.8,minSize:9,minStrokeWidth:1,maxStrokeWidth:4,quantileCount:4,quantizeCount:4,zero:!0},projection:{},legend:{gradientHorizontalMaxLength:200,gradientHorizontalMinLength:100,gradientVerticalMaxLength:200,gradientVerticalMinLength:64,unselectedOpacity:.35},header:{titlePadding:10,labelPadding:10},headerColumn:{},headerRow:{},headerFacet:{},selection:$s,style:{},title:{},facet:{spacing:20},concat:{spacing:20},normalizedNumberFormat:\".0%\"},Ls=[\"#4c78a8\",\"#f58518\",\"#e45756\",\"#72b7b2\",\"#54a24b\",\"#eeca3b\",\"#b279a2\",\"#ff9da6\",\"#9d755d\",\"#bab0ac\"],qs={text:11,guideLabel:10,guideTitle:11,groupTitle:13,groupSubtitle:12},Us={blue:Ls[0],orange:Ls[1],red:Ls[2],teal:Ls[3],green:Ls[4],yellow:Ls[5],purple:Ls[6],pink:Ls[7],brown:Ls[8],gray0:\"#000\",gray1:\"#111\",gray2:\"#222\",gray3:\"#333\",gray4:\"#444\",gray5:\"#555\",gray6:\"#666\",gray7:\"#777\",gray8:\"#888\",gray9:\"#999\",gray10:\"#aaa\",gray11:\"#bbb\",gray12:\"#ccc\",gray13:\"#ddd\",gray14:\"#eee\",gray15:\"#fff\"};function Rs(e){const t=D(e||{}),n={};for(const i of t){const t=e[i];n[i]=Fa(t)?kn(t):Sn(t)}return n}const Ws=[...no,...Pa,...vs,\"background\",\"padding\",\"legend\",\"lineBreak\",\"scale\",\"style\",\"title\",\"view\"];function Bs(){let e=arguments.length>0&&void 0!==arguments[0]?arguments[0]:{};const{color:n,font:i,fontSize:r,selection:o,...a}=e,s=t.mergeConfig({},l(Ms),i?function(e){return{text:{font:e},style:{\"guide-label\":{font:e},\"guide-title\":{font:e},\"group-title\":{font:e},\"group-subtitle\":{font:e}}}}(i):{},n?function(){let e=arguments.length>0&&void 0!==arguments[0]?arguments[0]:{};return{signals:[{name:\"color\",value:t.isObject(e)?{...Us,...e}:Us}],mark:{color:{signal:\"color.blue\"}},rule:{color:{signal:\"color.gray0\"}},text:{color:{signal:\"color.gray0\"}},style:{\"guide-label\":{fill:{signal:\"color.gray0\"}},\"guide-title\":{fill:{signal:\"color.gray0\"}},\"group-title\":{fill:{signal:\"color.gray0\"}},\"group-subtitle\":{fill:{signal:\"color.gray0\"}},cell:{stroke:{signal:\"color.gray8\"}}},axis:{domainColor:{signal:\"color.gray13\"},gridColor:{signal:\"color.gray8\"},tickColor:{signal:\"color.gray13\"}},range:{category:[{signal:\"color.blue\"},{signal:\"color.orange\"},{signal:\"color.red\"},{signal:\"color.teal\"},{signal:\"color.green\"},{signal:\"color.yellow\"},{signal:\"color.purple\"},{signal:\"color.pink\"},{signal:\"color.brown\"},{signal:\"color.grey8\"}]}}}(n):{},r?function(e){return{signals:[{name:\"fontSize\",value:t.isObject(e)?{...qs,...e}:qs}],text:{fontSize:{signal:\"fontSize.text\"}},style:{\"guide-label\":{fontSize:{signal:\"fontSize.guideLabel\"}},\"guide-title\":{fontSize:{signal:\"fontSize.guideTitle\"}},\"group-title\":{fontSize:{signal:\"fontSize.groupTitle\"}},\"group-subtitle\":{fontSize:{signal:\"fontSize.groupSubtitle\"}}}}}(r):{},a||{});o&&t.writeConfig(s,\"selection\",o,!0);const c=f(s,Ws);for(const e of[\"background\",\"lineBreak\",\"padding\"])s[e]&&(c[e]=Sn(s[e]));for(const e of no)s[e]&&(c[e]=pn(s[e]));for(const e of Pa)s[e]&&(c[e]=Rs(s[e]));for(const e of vs)s[e]&&(c[e]=pn(s[e]));return s.legend&&(c.legend=pn(s.legend)),s.scale&&(c.scale=pn(s.scale)),s.style&&(c.style=function(e){const t=D(e),n={};for(const i of t)n[i]=Rs(e[i]);return n}(s.style)),s.title&&(c.title=pn(s.title)),s.view&&(c.view=pn(s.view)),c}const Is=new Set([\"view\",...Kr]),Hs=[\"color\",\"fontSize\",\"background\",\"padding\",\"facet\",\"concat\",\"numberFormat\",\"numberFormatType\",\"normalizedNumberFormat\",\"normalizedNumberFormatType\",\"timeFormat\",\"countTitle\",\"header\",\"axisQuantitative\",\"axisTemporal\",\"axisDiscrete\",\"axisPoint\",\"axisXBand\",\"axisXPoint\",\"axisXDiscrete\",\"axisXQuantitative\",\"axisXTemporal\",\"axisYBand\",\"axisYPoint\",\"axisYDiscrete\",\"axisYQuantitative\",\"axisYTemporal\",\"scale\",\"selection\",\"overlay\"],Vs={view:[\"continuousWidth\",\"continuousHeight\",\"discreteWidth\",\"discreteHeight\",\"step\"],area:[\"line\",\"point\"],bar:[\"binSpacing\",\"continuousBandSize\",\"discreteBandSize\",\"minBandSize\"],rect:[\"binSpacing\",\"continuousBandSize\",\"discreteBandSize\",\"minBandSize\"],line:[\"point\"],tick:[\"bandSize\",\"thickness\"]};function Gs(e){e=l(e);for(const t of Hs)delete e[t];if(e.axis)for(const t in e.axis)Fa(e.axis[t])&&delete e.axis[t];if(e.legend)for(const t of ms)delete e.legend[t];if(e.mark){for(const t of to)delete e.mark[t];e.mark.tooltip&&t.isObject(e.mark.tooltip)&&delete e.mark.tooltip}e.params&&(e.signals=(e.signals||[]).concat(Ds(e.params)),delete e.params);for(const t of Is){for(const n of to)delete e[t][n];const n=Vs[t];if(n)for(const i of n)delete e[t][i];Ys(e,t)}for(const t of D(fs))delete e[t];!function(e){const{titleMarkConfig:t,subtitleMarkConfig:n,subtitle:i}=gn(e.title);S(t)||(e.style[\"group-title\"]={...e.style[\"group-title\"],...t});S(n)||(e.style[\"group-subtitle\"]={...e.style[\"group-subtitle\"],...n});S(i)?delete e.title:e.title=i}(e);for(const n in e)t.isObject(e[n])&&S(e[n])&&delete e[n];return S(e)?void 0:e}function Ys(e,t,n,i){\"view\"===t&&(n=\"cell\");const r={...i?e[t][i]:e[t],...e.style[n??t]};S(r)||(e.style[n??t]=r),i||delete e[t]}function Xs(e){return\"layer\"in e}class Qs{map(e,t){return To(e)?this.mapFacet(e,t):function(e){return\"repeat\"in e}(e)?this.mapRepeat(e,t):Os(e)?this.mapHConcat(e,t):zs(e)?this.mapVConcat(e,t):Fs(e)?this.mapConcat(e,t):this.mapLayerOrUnit(e,t)}mapLayerOrUnit(e,t){if(Xs(e))return this.mapLayer(e,t);if(Aa(e))return this.mapUnit(e,t);throw new Error(Bn(e))}mapLayer(e,t){return{...e,layer:e.layer.map((e=>this.mapLayerOrUnit(e,t)))}}mapHConcat(e,t){return{...e,hconcat:e.hconcat.map((e=>this.map(e,t)))}}mapVConcat(e,t){return{...e,vconcat:e.vconcat.map((e=>this.map(e,t)))}}mapConcat(e,t){const{concat:n,...i}=e;return{...i,concat:n.map((e=>this.map(e,t)))}}mapFacet(e,t){return{...e,spec:this.map(e.spec,t)}}mapRepeat(e,t){return{...e,spec:this.map(e.spec,t)}}}const Js={zero:1,center:1,normalize:1};const Ks=new Set([Er,Lr,Mr,Br,Rr,Gr,Yr,Ur,Ir,Hr]),Zs=new Set([Lr,Mr,Er]);function el(e){return Ho(e)&&\"quantitative\"===Vo(e)&&!e.bin}function tl(e,t,n){let{orient:i,type:r}=n;const o=\"x\"===t?\"y\":\"radius\",a=\"x\"===t&&[\"bar\",\"area\"].includes(r),s=e[t],l=e[o];if(Ho(s)&&Ho(l))if(el(s)&&el(l)){if(s.stack)return t;if(l.stack)return o;const e=Ho(s)&&!!s.aggregate;if(e!==(Ho(l)&&!!l.aggregate))return e?t:o;if(a){if(\"vertical\"===i)return o;if(\"horizontal\"===i)return t}}else{if(el(s))return t;if(el(l))return o}else{if(el(s)){if(a&&\"vertical\"===i)return;return t}if(el(l)){if(a&&\"horizontal\"===i)return;return o}}}function nl(e,n){const i=Zr(e)?e:{type:e},r=i.type;if(!Ks.has(r))return null;const o=tl(n,\"x\",i)||tl(n,\"theta\",i);if(!o)return null;const a=n[o],s=Ho(a)?oa(a,{}):void 0,l=function(e){switch(e){case\"x\":return\"y\";case\"y\":return\"x\";case\"theta\":return\"radius\";case\"radius\":return\"theta\"}}(o),c=[],u=new Set;if(n[l]){const e=n[l],t=Ho(e)?oa(e,{}):void 0;t&&t!==s&&(c.push(l),u.add(t))}const f=\"x\"===l?\"xOffset\":\"yOffset\",d=n[f],m=Ho(d)?oa(d,{}):void 0;m&&m!==s&&(c.push(f),u.add(m));const p=St.reduce(((e,i)=>{if(\"tooltip\"!==i&&Ta(n,i)){const r=n[i];for(const n of t.array(r)){const t=pa(n);if(t.aggregate)continue;const r=oa(t,{});r&&u.has(r)||e.push({channel:i,fieldDef:t})}}return e}),[]);let g;return void 0!==a.stack?g=t.isBoolean(a.stack)?a.stack?\"zero\":null:a.stack:Zs.has(r)&&(g=\"zero\"),g&&g in Js?La(n)&&0===p.length?null:a?.scale?.type&&a?.scale?.type!==cr.LINEAR?(a?.stack&&$i(function(e){return`Cannot stack non-linear scale (${e}).`}(a.scale.type)),null):Jo(n[it(o)])?(void 0!==a.stack&&$i(`Cannot stack \"${h=o}\" if there is already \"${h}2\".`),null):(Ho(a)&&a.aggregate&&!on.has(a.aggregate)&&$i(`Stacking is applied even though the aggregate function is non-summative (\"${a.aggregate}\").`),{groupbyChannels:c,groupbyFields:u,fieldChannel:o,impute:null!==a.impute&&Qr(r),stackBy:p,offset:g}):null;var h}function il(e,t,n){const i=pn(e),r=Cn(\"orient\",i,n);if(i.orient=function(e,t,n){switch(e){case Rr:case Gr:case Yr:case Ir:case Wr:case qr:return}const{x:i,y:r,x2:o,y2:a}=t;switch(e){case Lr:if(Ho(i)&&(cn(i.bin)||Ho(r)&&r.aggregate&&!i.aggregate))return\"vertical\";if(Ho(r)&&(cn(r.bin)||Ho(i)&&i.aggregate&&!r.aggregate))return\"horizontal\";if(a||o){if(n)return n;if(!o)return(Ho(i)&&i.type===rr&&!ln(i.bin)||Qo(i))&&Ho(r)&&cn(r.bin)?\"horizontal\":\"vertical\";if(!a)return(Ho(r)&&r.type===rr&&!ln(r.bin)||Qo(r))&&Ho(i)&&cn(i.bin)?\"vertical\":\"horizontal\"}case Br:if(o&&(!Ho(i)||!cn(i.bin))&&a&&(!Ho(r)||!cn(r.bin)))return;case Mr:if(a)return Ho(r)&&cn(r.bin)?\"horizontal\":\"vertical\";if(o)return Ho(i)&&cn(i.bin)?\"vertical\":\"horizontal\";if(e===Br){if(i&&!r)return\"vertical\";if(r&&!i)return\"horizontal\"}case Ur:case Hr:{const t=Xo(i),o=Xo(r);if(n)return n;if(t&&!o)return\"tick\"!==e?\"horizontal\":\"vertical\";if(!t&&o)return\"tick\"!==e?\"vertical\":\"horizontal\";if(t&&o)return\"vertical\";{const e=Ko(i)&&i.type===ar,t=Ko(r)&&r.type===ar;if(e&&!t)return\"vertical\";if(!e&&t)return\"horizontal\"}return}}return\"vertical\"}(i.type,t,r),void 0!==r&&r!==i.orient&&$i(`Specified orient \"${i.orient}\" overridden with \"${r}\".`),\"bar\"===i.type&&i.orient){const e=Cn(\"cornerRadiusEnd\",i,n);if(void 0!==e){const n=\"horizontal\"===i.orient&&t.x2||\"vertical\"===i.orient&&t.y2?[\"cornerRadius\"]:ro[i.orient];for(const t of n)i[t]=e;void 0!==i.cornerRadiusEnd&&delete i.cornerRadiusEnd}}void 0===Cn(\"opacity\",i,n)&&(i.opacity=function(e,t){if(p([Rr,Hr,Gr,Yr],e)&&!La(t))return.7;return}(i.type,t));return void 0===Cn(\"cursor\",i,n)&&(i.cursor=function(e,t,n){if(t.href||e.href||Cn(\"href\",e,n))return\"pointer\";return e.cursor}(i,t,n)),i}function rl(e){const{point:t,line:n,...i}=e;return D(i).length>1?i:i.type}function ol(e){for(const t of[\"line\",\"area\",\"rule\",\"trail\"])e[t]&&(e={...e,[t]:f(e[t],[\"point\",\"line\"])});return e}function al(e){let n=arguments.length>1&&void 0!==arguments[1]?arguments[1]:{},i=arguments.length>2?arguments[2]:void 0;return\"transparent\"===e.point?{opacity:0}:e.point?t.isObject(e.point)?e.point:{}:void 0!==e.point?null:n.point||i.shape?t.isObject(n.point)?n.point:{}:void 0}function sl(e){let t=arguments.length>1&&void 0!==arguments[1]?arguments[1]:{};return e.line?!0===e.line?{}:e.line:void 0!==e.line?null:t.line?!0===t.line?{}:t.line:void 0}class ll{constructor(){qn(this,\"name\",\"path-overlay\")}hasMatchingType(e,t){if(Aa(e)){const{mark:n,encoding:i}=e,r=Zr(n)?n:{type:n};switch(r.type){case\"line\":case\"rule\":case\"trail\":return!!al(r,t[r.type],i);case\"area\":return!!al(r,t[r.type],i)||!!sl(r,t[r.type])}}return!1}run(e,t,n){const{config:i}=t,{params:r,projection:o,mark:a,name:s,encoding:l,...c}=e,d=Ra(l,i),m=Zr(a)?a:{type:a},p=al(m,i[m.type],d),g=\"area\"===m.type&&sl(m,i[m.type]),h=[{name:s,...r?{params:r}:{},mark:rl({...\"area\"===m.type&&void 0===m.opacity&&void 0===m.fillOpacity?{opacity:.7}:{},...m}),encoding:f(d,[\"shape\"])}],y=nl(il(m,d,i),d);let v=d;if(y){const{fieldChannel:e,offset:t}=y;v={...d,[e]:{...d[e],...t?{stack:t}:{}}}}return v=f(v,[\"y2\",\"x2\"]),g&&h.push({...o?{projection:o}:{},mark:{type:\"line\",...u(m,[\"clip\",\"interpolate\",\"tension\",\"tooltip\"]),...g},encoding:v}),p&&h.push({...o?{projection:o}:{},mark:{type:\"point\",opacity:1,filled:!0,...u(m,[\"clip\",\"tooltip\"]),...p},encoding:v}),n({...c,layer:h},{...t,config:ol(i)})}}function cl(e,t){return t?Ao(e)?gl(e,t):dl(e,t):e}function ul(e,t){return t?gl(e,t):e}function fl(e,n,i){const r=n[e];return(o=r)&&!t.isString(o)&&\"repeat\"in o?r.repeat in i?{...n,[e]:i[r.repeat]}:void $i(function(e){return`Unknown repeated value \"${e}\".`}(r.repeat)):n;var o}function dl(e,t){if(void 0!==(e=fl(\"field\",e,t))){if(null===e)return null;if(Mo(e)&&Co(e.sort)){const n=fl(\"field\",e.sort,t);e={...e,...n?{sort:n}:{}}}return e}}function ml(e,t){if(Ho(e))return dl(e,t);{const n=fl(\"datum\",e,t);return n===e||n.type||(n.type=\"nominal\"),n}}function pl(e,t){if(!Jo(e)){if(Io(e)){const n=ml(e.condition,t);if(n)return{...e,condition:n};{const{condition:t,...n}=e;return n}}return e}{const n=ml(e,t);if(n)return n;if(Wo(e))return{condition:e.condition}}}function gl(e,n){const i={};for(const r in e)if(t.hasOwnProperty(e,r)){const o=e[r];if(t.isArray(o))i[r]=o.map((e=>pl(e,n))).filter((e=>e));else{const e=pl(o,n);void 0!==e&&(i[r]=e)}}return i}class hl{constructor(){qn(this,\"name\",\"RuleForRangedLine\")}hasMatchingType(e){if(Aa(e)){const{encoding:t,mark:n}=e;if(\"line\"===n||Zr(n)&&\"line\"===n.type)for(const e of Ze){const n=t[tt(e)];if(t[e]&&(Ho(n)&&!cn(n.bin)||Go(n)))return!0}}return!1}run(e,n,i){const{encoding:r,mark:o}=e;var a,s;return $i((a=!!r.x2,s=!!r.y2,`Line mark is for continuous lines and thus cannot be used with ${a&&s?\"x2 and y2\":a?\"x2\":\"y2\"}. We will use the rule mark (line segments) instead.`)),i({...e,mark:t.isObject(o)?{...o,type:\"rule\"}:\"rule\"},n)}}function yl(e){let{parentEncoding:n,encoding:i={},layer:r}=e,o={};if(n){const e=new Set([...D(n),...D(i)]);for(const a of e){const e=i[a],s=n[a];if(Jo(e)){const t={...s,...e};o[a]=t}else Io(e)?o[a]={...e,condition:{...s,...e.condition}}:e||null===e?o[a]=e:(r||Zo(s)||yn(s)||Jo(s)||t.isArray(s))&&(o[a]=s)}}else o=i;return!o||S(o)?void 0:o}function vl(e){const{parentProjection:t,projection:n}=e;return t&&n&&$i(function(e){const{parentProjection:t,projection:n}=e;return`Layer's shared projection ${X(t)} is overridden by a child projection ${X(n)}.`}({parentProjection:t,projection:n})),n??t}function bl(e){return\"filter\"in e}function xl(e){return\"lookup\"in e}function $l(e){return\"pivot\"in e}function wl(e){return\"density\"in e}function kl(e){return\"quantile\"in e}function Sl(e){return\"regression\"in e}function Dl(e){return\"loess\"in e}function Fl(e){return\"sample\"in e}function zl(e){return\"window\"in e}function Ol(e){return\"joinaggregate\"in e}function _l(e){return\"flatten\"in e}function Nl(e){return\"calculate\"in e}function Cl(e){return\"bin\"in e}function Pl(e){return\"impute\"in e}function Al(e){return\"timeUnit\"in e}function jl(e){return\"aggregate\"in e}function Tl(e){return\"stack\"in e}function El(e){return\"fold\"in e}function Ml(e){return\"extent\"in e&&!(\"density\"in e)}function Ll(e,t){const{transform:n,...i}=e;if(n){return{...i,transform:n.map((e=>{if(bl(e))return{filter:Rl(e,t)};if(Cl(e)&&un(e.bin))return{...e,bin:Ul(e.bin)};if(xl(e)){const{selection:t,...n}=e.from;return t?{...e,from:{param:t,...n}}:e}return e}))}}return e}function ql(e,n){const i=l(e);if(Ho(i)&&un(i.bin)&&(i.bin=Ul(i.bin)),ea(i)&&i.scale?.domain?.selection){const{selection:e,...t}=i.scale.domain;i.scale.domain={...t,...e?{param:e}:{}}}if(Wo(i))if(t.isArray(i.condition))i.condition=i.condition.map((e=>{const{selection:t,param:i,test:r,...o}=e;return i?e:{...o,test:Rl(e,n)}}));else{const{selection:e,param:t,test:r,...o}=ql(i.condition,n);i.condition=t?i.condition:{...o,test:Rl(i.condition,n)}}return i}function Ul(e){const t=e.extent;if(t?.selection){const{selection:n,...i}=t;return{...e,extent:{...i,param:n}}}return e}function Rl(e,t){const n=e=>s(e,(e=>{const n={param:e,empty:t.emptySelections[e]??!0};return t.selectionPredicates[e]??=[],t.selectionPredicates[e].push(n),n}));return e.selection?n(e.selection):s(e.test||e.filter,(e=>e.selection?n(e.selection):e))}class Wl extends Qs{map(e,t){const n=t.selections??[];if(e.params&&!Aa(e)){const t=[];for(const i of e.params)Ss(i)?n.push(i):t.push(i);e.params=t}return t.selections=n,super.map(e,t)}mapUnit(e,n){const i=n.selections;if(!i||!i.length)return e;const r=(n.path??[]).concat(e.name),o=[];for(const n of i)if(n.views&&n.views.length)for(const i of n.views)(t.isString(i)&&(i===e.name||r.includes(i))||t.isArray(i)&&i.map((e=>r.indexOf(e))).every(((e,t,n)=>-1!==e&&(0===t||e>n[t-1]))))&&o.push(n);else o.push(n);return o.length&&(e.params=o),e}}for(const e of[\"mapFacet\",\"mapRepeat\",\"mapHConcat\",\"mapVConcat\",\"mapLayer\"]){const t=Wl.prototype[e];Wl.prototype[e]=function(e,n){return t.call(this,e,Bl(e,n))}}function Bl(e,t){return e.name?{...t,path:(t.path??[]).concat(e.name)}:t}function Il(e,t){void 0===t&&(t=Bs(e.config));const n=function(e){let t=arguments.length>1&&void 0!==arguments[1]?arguments[1]:{};const n={config:t};return Gl.map(Hl.map(Vl.map(e,n),n),n)}(e,t),{width:i,height:r}=e,o=function(e,t,n){let{width:i,height:r}=t;const o=Aa(e)||Xs(e),a={};o?\"container\"==i&&\"container\"==r?(a.type=\"fit\",a.contains=\"padding\"):\"container\"==i?(a.type=\"fit-x\",a.contains=\"padding\"):\"container\"==r&&(a.type=\"fit-y\",a.contains=\"padding\"):(\"container\"==i&&($i(Hn(\"width\")),i=void 0),\"container\"==r&&($i(Hn(\"height\")),r=void 0));const s={type:\"pad\",...a,...n?Yl(n.autosize):{},...Yl(e.autosize)};\"fit\"!==s.type||o||($i(In),s.type=\"pad\");\"container\"==i&&\"fit\"!=s.type&&\"fit-x\"!=s.type&&$i(Vn(\"width\"));\"container\"==r&&\"fit\"!=s.type&&\"fit-y\"!=s.type&&$i(Vn(\"height\"));if(Y(s,{type:\"pad\"}))return;return s}(n,{width:i,height:r,autosize:e.autosize},t);return{...n,...o?{autosize:o}:{}}}const Hl=new class extends Qs{constructor(){super(...arguments),qn(this,\"nonFacetUnitNormalizers\",[Ka,is,cs,new ll,new hl])}map(e,t){if(Aa(e)){const n=Ta(e.encoding,Q),i=Ta(e.encoding,J),r=Ta(e.encoding,K);if(n||i||r)return this.mapFacetedUnit(e,t)}return super.map(e,t)}mapUnit(e,t){const{parentEncoding:n,parentProjection:i}=t,r=ul(e.encoding,t.repeater),o={...e,...e.name?{name:[t.repeaterPrefix,e.name].filter((e=>e)).join(\"_\")}:{},...r?{encoding:r}:{}};if(n||i)return this.mapUnitWithParentEncodingOrProjection(o,t);const a=this.mapLayerOrUnit.bind(this);for(const e of this.nonFacetUnitNormalizers)if(e.hasMatchingType(o,t.config))return e.run(o,t,a);return o}mapRepeat(e,n){return function(e){return!t.isArray(e.repeat)&&e.repeat.layer}(e)?this.mapLayerRepeat(e,n):this.mapNonLayerRepeat(e,n)}mapLayerRepeat(e,t){const{repeat:n,spec:i,...r}=e,{row:o,column:a,layer:s}=n,{repeater:l={},repeaterPrefix:c=\"\"}=t;return o||a?this.mapRepeat({...e,repeat:{...o?{row:o}:{},...a?{column:a}:{}},spec:{repeat:{layer:s},spec:i}},t):{...r,layer:s.map((e=>{const n={...l,layer:e},r=`${(i.name?`${i.name}_`:\"\")+c}child__layer_${_(e)}`,o=this.mapLayerOrUnit(i,{...t,repeater:n,repeaterPrefix:r});return o.name=r,o}))}}mapNonLayerRepeat(e,n){const{repeat:i,spec:r,data:o,...a}=e;!t.isArray(i)&&e.columns&&(e=f(e,[\"columns\"]),$i(Zn(\"repeat\")));const s=[],{repeater:l={},repeaterPrefix:c=\"\"}=n,u=!t.isArray(i)&&i.row||[l?l.row:null],d=!t.isArray(i)&&i.column||[l?l.column:null],m=t.isArray(i)&&i||[l?l.repeat:null];for(const e of m)for(const o of u)for(const a of d){const u={repeat:e,row:o,column:a,layer:l.layer},d=(r.name?`${r.name}_`:\"\")+c+\"child__\"+(t.isArray(i)?`${_(e)}`:(i.row?`row_${_(o)}`:\"\")+(i.column?`column_${_(a)}`:\"\")),m=this.map(r,{...n,repeater:u,repeaterPrefix:d});m.name=d,s.push(f(m,[\"data\"]))}const p=t.isArray(i)?e.columns:i.column?i.column.length:1;return{data:r.data??o,align:\"all\",...a,columns:p,concat:s}}mapFacet(e,t){const{facet:n}=e;return Ao(n)&&e.columns&&(e=f(e,[\"columns\"]),$i(Zn(\"facet\"))),super.mapFacet(e,t)}mapUnitWithParentEncodingOrProjection(e,t){const{encoding:n,projection:i}=e,{parentEncoding:r,parentProjection:o,config:a}=t,s=vl({parentProjection:o,projection:i}),l=yl({parentEncoding:r,encoding:ul(n,t.repeater)});return this.mapUnit({...e,...s?{projection:s}:{},...l?{encoding:l}:{}},{config:a})}mapFacetedUnit(e,t){const{row:n,column:i,facet:r,...o}=e.encoding,{mark:a,width:s,projection:l,height:c,view:u,params:f,encoding:d,...m}=e,{facetMapping:p,layout:g}=this.getFacetMappingAndLayout({row:n,column:i,facet:r},t),h=ul(o,t.repeater);return this.mapFacet({...m,...g,facet:p,spec:{...s?{width:s}:{},...c?{height:c}:{},...u?{view:u}:{},...l?{projection:l}:{},mark:a,encoding:h,...f?{params:f}:{}}},t)}getFacetMappingAndLayout(e,t){const{row:n,column:i,facet:r}=e;if(n||i){r&&$i(`Facet encoding dropped as ${(o=[...n?[Q]:[],...i?[J]:[]]).join(\" and \")} ${o.length>1?\"are\":\"is\"} also specified.`);const t={},a={};for(const n of[Q,J]){const i=e[n];if(i){const{align:e,center:r,spacing:o,columns:s,...l}=i;t[n]=l;for(const e of[\"align\",\"center\",\"spacing\"])void 0!==i[e]&&(a[e]??={},a[e][n]=i[e])}}return{facetMapping:t,layout:a}}{const{align:e,center:n,spacing:i,columns:o,...a}=r;return{facetMapping:cl(a,t.repeater),layout:{...e?{align:e}:{},...n?{center:n}:{},...i?{spacing:i}:{},...o?{columns:o}:{}}}}var o}mapLayer(e,t){let{parentEncoding:n,parentProjection:i,...r}=t;const{encoding:o,projection:a,...s}=e,l={...r,parentEncoding:yl({parentEncoding:n,encoding:o,layer:!0}),parentProjection:vl({parentProjection:i,projection:a})};return super.mapLayer({...s,...e.name?{name:[l.repeaterPrefix,e.name].filter((e=>e)).join(\"_\")}:{}},l)}},Vl=new class extends Qs{map(e,t){return t.emptySelections??={},t.selectionPredicates??={},e=Ll(e,t),super.map(e,t)}mapLayerOrUnit(e,t){if((e=Ll(e,t)).encoding){const n={};for(const[i,r]of z(e.encoding))n[i]=ql(r,t);e={...e,encoding:n}}return super.mapLayerOrUnit(e,t)}mapUnit(e,t){const{selection:n,...i}=e;return n?{...i,params:z(n).map((e=>{let[n,i]=e;const{init:r,bind:o,empty:a,...s}=i;\"single\"===s.type?(s.type=\"point\",s.toggle=!1):\"multi\"===s.type&&(s.type=\"point\"),t.emptySelections[n]=\"none\"!==a;for(const e of F(t.selectionPredicates[n]??{}))e.empty=\"none\"!==a;return{name:n,value:r,select:s,bind:o}}))}:e}},Gl=new Wl;function Yl(e){return t.isString(e)?{type:e}:e??{}}const Xl=[\"background\",\"padding\"];function Ql(e,t){const n={};for(const t of Xl)e&&void 0!==e[t]&&(n[t]=Sn(e[t]));return t&&(n.params=e.params),n}class Jl{constructor(){let e=arguments.length>0&&void 0!==arguments[0]?arguments[0]:{},t=arguments.length>1&&void 0!==arguments[1]?arguments[1]:{};this.explicit=e,this.implicit=t}clone(){return new Jl(l(this.explicit),l(this.implicit))}combine(){return{...this.explicit,...this.implicit}}get(e){return U(this.explicit[e],this.implicit[e])}getWithExplicit(e){return void 0!==this.explicit[e]?{explicit:!0,value:this.explicit[e]}:void 0!==this.implicit[e]?{explicit:!1,value:this.implicit[e]}:{explicit:!1,value:void 0}}setWithExplicit(e,t){let{value:n,explicit:i}=t;void 0!==n&&this.set(e,n,i)}set(e,t,n){return delete this[n?\"implicit\":\"explicit\"][e],this[n?\"explicit\":\"implicit\"][e]=t,this}copyKeyFromSplit(e,t){let{explicit:n,implicit:i}=t;void 0!==n[e]?this.set(e,n[e],!0):void 0!==i[e]&&this.set(e,i[e],!1)}copyKeyFromObject(e,t){void 0!==t[e]&&this.set(e,t[e],!0)}copyAll(e){for(const t of D(e.combine())){const n=e.getWithExplicit(t);this.setWithExplicit(t,n)}}}function Kl(e){return{explicit:!0,value:e}}function Zl(e){return{explicit:!1,value:e}}function ec(e){return(t,n,i,r)=>{const o=e(t.value,n.value);return o>0?t:o<0?n:tc(t,n,i,r)}}function tc(e,t,n,i){return e.explicit&&t.explicit&&$i(function(e,t,n,i){return`Conflicting ${t.toString()} property \"${e.toString()}\" (${X(n)} and ${X(i)}). Using ${X(n)}.`}(n,i,e.value,t.value)),e}function nc(e,t,n,i){let r=arguments.length>4&&void 0!==arguments[4]?arguments[4]:tc;return void 0===e||void 0===e.value?t:e.explicit&&!t.explicit?e:t.explicit&&!e.explicit?t:Y(e.value,t.value)?e:r(e,t,n,i)}class ic extends Jl{constructor(){let e=arguments.length>0&&void 0!==arguments[0]?arguments[0]:{},t=arguments.length>1&&void 0!==arguments[1]?arguments[1]:{},n=arguments.length>2&&void 0!==arguments[2]&&arguments[2];super(e,t),this.explicit=e,this.implicit=t,this.parseNothing=n}clone(){const e=super.clone();return e.parseNothing=this.parseNothing,e}}function rc(e){return\"url\"in e}function oc(e){return\"values\"in e}function ac(e){return\"name\"in e&&!rc(e)&&!oc(e)&&!sc(e)}function sc(e){return e&&(lc(e)||cc(e)||uc(e))}function lc(e){return\"sequence\"in e}function cc(e){return\"sphere\"in e}function uc(e){return\"graticule\"in e}let fc=function(e){return e[e.Raw=0]=\"Raw\",e[e.Main=1]=\"Main\",e[e.Row=2]=\"Row\",e[e.Column=3]=\"Column\",e[e.Lookup=4]=\"Lookup\",e}({});function dc(e){const{signals:t,hasLegend:n,index:i,...r}=e;return r.field=E(r.field),r}function mc(e){let n=!(arguments.length>1&&void 0!==arguments[1])||arguments[1],i=arguments.length>2&&void 0!==arguments[2]?arguments[2]:t.identity;if(t.isArray(e)){const t=e.map((e=>mc(e,n,i)));return n?`[${t.join(\", \")}]`:t}return wi(e)?i(n?Oi(e):function(e){const t=zi(e,!0);return e.utc?+new Date(Date.UTC(...t)):+new Date(...t)}(e)):n?i(X(e)):e}function pc(e,n){for(const i of F(e.component.selection??{})){const r=i.name;let o=`${r}${Au}, ${\"global\"===i.resolve?\"true\":`{unit: ${Mu(e)}}`}`;for(const t of Eu)t.defined(i)&&(t.signals&&(n=t.signals(e,i,n)),t.modifyExpr&&(o=t.modifyExpr(e,i,o)));n.push({name:r+ju,on:[{events:{signal:i.name+Au},update:`modify(${t.stringValue(i.name+Pu)}, ${o})`}]})}return yc(n)}function gc(e,n){if(e.component.selection&&D(e.component.selection).length){const i=t.stringValue(e.getName(\"cell\"));n.unshift({name:\"facet\",value:{},on:[{events:t.parseSelector(\"pointermove\",\"scope\"),update:`isTuple(facet) ? facet : group(${i}).datum`}]})}return yc(n)}function hc(e,t){for(const n of F(e.component.selection??{}))for(const i of Eu)i.defined(n)&&i.marks&&(t=i.marks(e,n,t));return t}function yc(e){return e.map((e=>(e.on&&!e.on.length&&delete e.on,e)))}class vc{constructor(e,t){this.debugName=t,qn(this,\"_children\",[]),qn(this,\"_parent\",null),qn(this,\"_hash\",void 0),e&&(this.parent=e)}clone(){throw new Error(\"Cannot clone node\")}get parent(){return this._parent}set parent(e){this._parent=e,e&&e.addChild(this)}get children(){return this._children}numChildren(){return this._children.length}addChild(e,t){this._children.includes(e)?$i(\"Attempt to add the same child twice.\"):void 0!==t?this._children.splice(t,0,e):this._children.push(e)}removeChild(e){const t=this._children.indexOf(e);return this._children.splice(t,1),t}remove(){let e=this._parent.removeChild(this);for(const t of this._children)t._parent=this._parent,this._parent.addChild(t,e++)}insertAsParentOf(e){const t=e.parent;t.removeChild(this),this.parent=t,e.parent=this}swapWithParent(){const e=this._parent,t=e.parent;for(const t of this._children)t.parent=e;this._children=[],e.removeChild(this);const n=e.parent.removeChild(e);this._parent=t,t.addChild(this,n),e.parent=this}}class bc extends vc{clone(){const e=new this.constructor;return e.debugName=`clone_${this.debugName}`,e._source=this._source,e._name=`clone_${this._name}`,e.type=this.type,e.refCounts=this.refCounts,e.refCounts[e._name]=0,e}constructor(e,t,n,i){super(e,t),this.type=n,this.refCounts=i,qn(this,\"_source\",void 0),qn(this,\"_name\",void 0),this._source=this._name=t,this.refCounts&&!(this._name in this.refCounts)&&(this.refCounts[this._name]=0)}dependentFields(){return new Set}producedFields(){return new Set}hash(){return void 0===this._hash&&(this._hash=`Output ${W()}`),this._hash}getSource(){return this.refCounts[this._name]++,this._source}isRequired(){return!!this.refCounts[this._name]}setSource(e){this._source=e}}function xc(e){return void 0!==e.as}function $c(e){return`${e}_end`}class wc extends vc{clone(){return new wc(null,l(this.timeUnits))}constructor(e,t){super(e),this.timeUnits=t}static makeFromEncoding(e,t){const n=t.reduceFieldDef(((e,n,i)=>{const{field:r,timeUnit:o}=n;if(o){let a;if(Ci(o)){if(xm(t)){const{mark:e,markDef:i,config:s}=t,l=Lo({fieldDef:n,markDef:i,config:s});(Jr(e)||l)&&(a={timeUnit:Ui(o),field:r})}}else a={as:oa(n,{forAs:!0}),field:r,timeUnit:o};if(xm(t)){const{mark:e,markDef:r,config:o}=t,s=Lo({fieldDef:n,markDef:r,config:o});Jr(e)&&zt(i)&&.5!==s&&(a.rectBandPosition=s)}a&&(e[d(a)]=a)}return e}),{});return S(n)?null:new wc(e,n)}static makeFromTransform(e,t){const{timeUnit:n,...i}={...t},r={...i,timeUnit:Ui(n)};return new wc(e,{[d(r)]:r})}merge(e){this.timeUnits={...this.timeUnits};for(const t in e.timeUnits)this.timeUnits[t]||(this.timeUnits[t]=e.timeUnits[t]);for(const t of e.children)e.removeChild(t),t.parent=this;e.remove()}removeFormulas(e){const t={};for(const[n,i]of z(this.timeUnits)){const r=xc(i)?i.as:`${i.field}_end`;e.has(r)||(t[n]=i)}this.timeUnits=t}producedFields(){return new Set(F(this.timeUnits).map((e=>xc(e)?e.as:$c(e.field))))}dependentFields(){return new Set(F(this.timeUnits).map((e=>e.field)))}hash(){return`TimeUnit ${d(this.timeUnits)}`}assemble(){const e=[];for(const t of F(this.timeUnits)){const{rectBandPosition:n}=t,i=Ui(t.timeUnit);if(xc(t)){const{field:r,as:o}=t,{unit:a,utc:s,...l}=i,c=[o,`${o}_end`];e.push({field:E(r),type:\"timeunit\",...a?{units:Ti(a)}:{},...s?{timezone:\"utc\"}:{},...l,as:c}),e.push(...Fc(c,n,i))}else if(t){const{field:r}=t,o=r.replaceAll(\"\\\\.\",\".\"),a=Dc({timeUnit:i,field:o}),s=$c(o);e.push({type:\"formula\",expr:a,as:s}),e.push(...Fc([o,s],n,i))}}return e}}const kc=\"offsetted_rect_start\",Sc=\"offsetted_rect_end\";function Dc(e){let{timeUnit:t,field:n,reverse:i}=e;const{unit:r,utc:o}=t,a=Ei(r),{part:s,step:l}=Bi(a,t.step);return`${o?\"utcOffset\":\"timeOffset\"}('${s}', datum['${n}'], ${i?-l:l})`}function Fc(e,t,n){let[i,r]=e;if(void 0!==t&&.5!==t){const e=`datum['${i}']`,o=`datum['${r}']`;return[{type:\"formula\",expr:zc([Dc({timeUnit:n,field:i,reverse:!0}),e],t+.5),as:`${i}_${kc}`},{type:\"formula\",expr:zc([e,o],t+.5),as:`${i}_${Sc}`}]}return[]}function zc(e,t){let[n,i]=e;return`${1-t} * ${n} + ${t} * ${i}`}const Oc=\"_tuple_fields\";class _c{constructor(){qn(this,\"hasChannel\",void 0),qn(this,\"hasField\",void 0),qn(this,\"hasSelectionId\",void 0),qn(this,\"timeUnit\",void 0),qn(this,\"items\",void 0);for(var e=arguments.length,t=new Array(e),n=0;n!0,parse:(e,n,i)=>{const r=n.name,o=n.project??=new _c,a={},s={},l=new Set,c=(e,t)=>{const n=\"visual\"===t?e.channel:e.field;let i=_(`${r}_${n}`);for(let e=1;l.has(i);e++)i=_(`${r}_${n}_${e}`);return l.add(i),{[t]:i}},u=n.type,f=e.config.selection[u],m=void 0!==i.value?t.array(i.value):null;let{fields:p,encodings:g}=t.isObject(i.select)?i.select:{};if(!p&&!g&&m)for(const e of m)if(t.isObject(e))for(const t of D(e))Je[t]?(g||(g=[])).push(t):\"interval\"===u?($i('Interval selections should be initialized using \"x\", \"y\", \"longitude\", or \"latitude\" keys.'),g=f.encodings):(p??=[]).push(t);p||g||(g=f.encodings,\"fields\"in f&&(p=f.fields));for(const t of g??[]){const n=e.fieldDef(t);if(n){let i=n.field;if(n.aggregate){$i(Qn(t,n.aggregate));continue}if(!i){$i(Xn(t));continue}if(n.timeUnit&&!Ci(n.timeUnit)){i=e.vgField(t);const r={timeUnit:n.timeUnit,as:i,field:n.field};s[d(r)]=r}if(!a[i]){const r={field:i,channel:t,type:\"interval\"===u&&Ht(t)&&$r(e.getScaleComponent(t).get(\"type\"))?\"R\":n.bin?\"R-RE\":\"E\",index:o.items.length};r.signals={...c(r,\"data\"),...c(r,\"visual\")},o.items.push(a[i]=r),o.hasField[i]=a[i],o.hasSelectionId=o.hasSelectionId||i===xs,Ee(t)?(r.geoChannel=t,r.channel=Te(t),o.hasChannel[r.channel]=a[i]):o.hasChannel[t]=a[i]}}else $i(Xn(t))}for(const e of p??[]){if(o.hasField[e])continue;const t={type:\"E\",field:e,index:o.items.length};t.signals={...c(t,\"data\")},o.items.push(t),o.hasField[e]=t,o.hasSelectionId=o.hasSelectionId||e===xs}m&&(n.init=m.map((e=>o.items.map((n=>t.isObject(e)?void 0!==e[n.geoChannel||n.channel]?e[n.geoChannel||n.channel]:e[n.field]:e))))),S(s)||(o.timeUnit=new wc(null,s))},signals:(e,t,n)=>{const i=t.name+Oc;return n.filter((e=>e.name===i)).length>0||t.project.hasSelectionId?n:n.concat({name:i,value:t.project.items.map(dc)})}},Cc={defined:e=>\"interval\"===e.type&&\"global\"===e.resolve&&e.bind&&\"scales\"===e.bind,parse:(e,t)=>{const n=t.scales=[];for(const i of t.project.items){const r=i.channel;if(!Ht(r))continue;const o=e.getScaleComponent(r),a=o?o.get(\"type\"):void 0;o&&$r(a)?(o.set(\"selectionExtent\",{param:t.name,field:i.field},!0),n.push(i)):$i(\"Scale bindings are currently only supported for scales with unbinned, continuous domains.\")}},topLevelSignals:(e,n,i)=>{const r=n.scales.filter((e=>0===i.filter((t=>t.name===e.signals.data)).length));if(!e.parent||Ac(e)||0===r.length)return i;const o=i.filter((e=>e.name===n.name))[0];let a=o.update;if(a.indexOf(Tu)>=0)o.update=`{${r.map((e=>`${t.stringValue(E(e.field))}: ${e.signals.data}`)).join(\", \")}}`;else{for(const e of r){const n=`${t.stringValue(E(e.field))}: ${e.signals.data}`;a.includes(n)||(a=`${a.substring(0,a.length-1)}, ${n}}`)}o.update=a}return i.concat(r.map((e=>({name:e.signals.data}))))},signals:(e,t,n)=>{if(e.parent&&!Ac(e))for(const e of t.scales){const t=n.filter((t=>t.name===e.signals.data))[0];t.push=\"outer\",delete t.value,delete t.update}return n}};function Pc(e,n){return`domain(${t.stringValue(e.scaleName(n))})`}function Ac(e){return e.parent&&km(e.parent)&&(!e.parent.parent??Ac(e.parent.parent))}const jc=\"_brush\",Tc=\"_scale_trigger\",Ec=\"geo_interval_init_tick\",Mc=\"_init\",Lc={defined:e=>\"interval\"===e.type,parse:(e,n,i)=>{if(e.hasProjection){const e={...t.isObject(i.select)?i.select:{}};e.fields=[xs],e.encodings||(e.encodings=i.value?D(i.value):[ue,ce]),i.select={type:\"interval\",...e}}if(n.translate&&!Cc.defined(n)){const e=`!event.item || event.item.mark.name !== ${t.stringValue(n.name+jc)}`;for(const i of n.events){if(!i.between){$i(`${i} is not an ordered event stream for interval selections.`);continue}const n=t.array(i.between[0].filter??=[]);n.indexOf(e)<0&&n.push(e)}}},signals:(e,n,i)=>{const r=n.name,o=r+Au,a=F(n.project.hasChannel).filter((e=>e.channel===Z||e.channel===ee)),s=n.init?n.init[0]:null;if(i.push(...a.reduce(((i,r)=>i.concat(function(e,n,i,r){const o=!e.hasProjection,a=i.channel,s=i.signals.visual,l=t.stringValue(o?e.scaleName(a):e.projectionName()),c=e=>`scale(${l}, ${e})`,u=e.getSizeSignalRef(a===Z?\"width\":\"height\").signal,f=`${a}(unit)`,d=n.events.reduce(((e,t)=>[...e,{events:t.between[0],update:`[${f}, ${f}]`},{events:t,update:`[${s}[0], clamp(${f}, 0, ${u})]`}]),[]);if(o){const t=i.signals.data,o=Cc.defined(n),u=e.getScaleComponent(a),f=u?u.get(\"type\"):void 0,m=r?{init:mc(r,!0,c)}:{value:[]};return d.push({events:{signal:n.name+Tc},update:$r(f)?`[${c(`${t}[0]`)}, ${c(`${t}[1]`)}]`:\"[0, 0]\"}),o?[{name:t,on:[]}]:[{name:s,...m,on:d},{name:t,...r?{init:mc(r)}:{},on:[{events:{signal:s},update:`${s}[0] === ${s}[1] ? null : invert(${l}, ${s})`}]}]}{const e=a===Z?0:1,t=n.name+Mc;return[{name:s,...r?{init:`[${t}[0][${e}], ${t}[1][${e}]]`}:{value:[]},on:d}]}}(e,n,r,s&&s[r.index]))),[])),e.hasProjection){const l=t.stringValue(e.projectionName()),c=e.projectionName()+\"_center\",{x:u,y:f}=n.project.hasChannel,d=u&&u.signals.visual,m=f&&f.signals.visual,p=u?s&&s[u.index]:`${c}[0]`,g=f?s&&s[f.index]:`${c}[1]`,h=t=>e.getSizeSignalRef(t).signal,y=`[[${d?d+\"[0]\":\"0\"}, ${m?m+\"[0]\":\"0\"}],[${d?d+\"[1]\":h(\"width\")}, ${m?m+\"[1]\":h(\"height\")}]]`;if(s&&(i.unshift({name:r+Mc,init:`[scale(${l}, [${u?p[0]:p}, ${f?g[0]:g}]), scale(${l}, [${u?p[1]:p}, ${f?g[1]:g}])]`}),!u||!f)){i.find((e=>e.name===c))||i.unshift({name:c,update:`invert(${l}, [${h(\"width\")}/2, ${h(\"height\")}/2])`})}const v=`vlSelectionTuples(${`intersect(${y}, {markname: ${t.stringValue(e.getName(\"marks\"))}}, unit.mark)`}, ${`{unit: ${Mu(e)}}`})`,b=a.map((e=>e.signals.visual));return i.concat({name:o,on:[{events:[...b.length?[{signal:b.join(\" || \")}]:[],...s?[{signal:Ec}]:[]],update:v}]})}{if(!Cc.defined(n)){const n=r+Tc,o=a.map((n=>{const i=n.channel,{data:r,visual:o}=n.signals,a=t.stringValue(e.scaleName(i)),s=$r(e.getScaleComponent(i).get(\"type\"))?\"+\":\"\";return`(!isArray(${r}) || (${s}invert(${a}, ${o})[0] === ${s}${r}[0] && ${s}invert(${a}, ${o})[1] === ${s}${r}[1]))`}));o.length&&i.push({name:n,value:{},on:[{events:a.map((t=>({scale:e.scaleName(t.channel)}))),update:o.join(\" && \")+` ? ${n} : {}`}]})}const l=a.map((e=>e.signals.data)),c=`unit: ${Mu(e)}, fields: ${r+Oc}, values`;return i.concat({name:o,...s?{init:`{${c}: ${mc(s)}}`}:{},...l.length?{on:[{events:[{signal:l.join(\" || \")}],update:`${l.join(\" && \")} ? {${c}: [${l}]} : null`}]}:{}})}},topLevelSignals:(e,t,n)=>{if(xm(e)&&e.hasProjection&&t.init){n.filter((e=>e.name===Ec)).length||n.unshift({name:Ec,value:null,on:[{events:\"timer{1}\",update:`${Ec} === null ? {} : ${Ec}`}]})}return n},marks:(e,n,i)=>{const r=n.name,{x:o,y:a}=n.project.hasChannel,s=o?.signals.visual,l=a?.signals.visual,c=`data(${t.stringValue(n.name+Pu)})`;if(Cc.defined(n)||!o&&!a)return i;const u={x:void 0!==o?{signal:`${s}[0]`}:{value:0},y:void 0!==a?{signal:`${l}[0]`}:{value:0},x2:void 0!==o?{signal:`${s}[1]`}:{field:{group:\"width\"}},y2:void 0!==a?{signal:`${l}[1]`}:{field:{group:\"height\"}}};if(\"global\"===n.resolve)for(const t of D(u))u[t]=[{test:`${c}.length && ${c}[0].unit === ${Mu(e)}`,...u[t]},{value:0}];const{fill:f,fillOpacity:d,cursor:m,...p}=n.mark,g=D(p).reduce(((e,t)=>(e[t]=[{test:[void 0!==o&&`${s}[0] !== ${s}[1]`,void 0!==a&&`${l}[0] !== ${l}[1]`].filter((e=>e)).join(\" && \"),value:p[t]},{value:null}],e)),{});return[{name:`${r+jc}_bg`,type:\"rect\",clip:!0,encode:{enter:{fill:{value:f},fillOpacity:{value:d}},update:u}},...i,{name:r+jc,type:\"rect\",clip:!0,encode:{enter:{...m?{cursor:{value:m}}:{},fill:{value:\"transparent\"}},update:{...u,...g}}}]}};const qc={defined:e=>\"point\"===e.type,signals:(e,n,i)=>{const r=n.name,o=r+Oc,a=n.project,s=\"(item().isVoronoi ? datum.datum : datum)\",l=F(e.component.selection??{}).reduce(((e,t)=>\"interval\"===t.type?e.concat(t.name+jc):e),[]).map((e=>`indexof(item().mark.name, '${e}') < 0`)).join(\" && \"),c=\"datum && item().mark.marktype !== 'group' && indexof(item().mark.role, 'legend') < 0\"+(l?` && ${l}`:\"\");let u=`unit: ${Mu(e)}, `;if(n.project.hasSelectionId)u+=`${xs}: ${s}[${t.stringValue(xs)}]`;else{u+=`fields: ${o}, values: [${a.items.map((n=>{const i=e.fieldDef(n.channel);return i?.bin?`[${s}[${t.stringValue(e.vgField(n.channel,{}))}], ${s}[${t.stringValue(e.vgField(n.channel,{binSuffix:\"end\"}))}]]`:`${s}[${t.stringValue(n.field)}]`})).join(\", \")}]`}const f=n.events;return i.concat([{name:r+Au,on:f?[{events:f,update:`${c} ? {${u}} : null`,force:!0}]:[]}])}};function Uc(e,n,i,r){const o=Wo(n)&&n.condition,a=r(n);if(o){return{[i]:[...t.array(o).map((t=>{const n=r(t);if(function(e){return e.param}(t)){const{param:i,empty:r}=t;return{test:Iu(e,{param:i,empty:r}),...n}}return{test:Vu(e,t.test),...n}})),...void 0!==a?[a]:[]]}}return void 0!==a?{[i]:a}:{}}function Rc(e){let t=arguments.length>1&&void 0!==arguments[1]?arguments[1]:\"text\";const n=e.encoding[t];return Uc(e,n,t,(t=>Wc(t,e.config)))}function Wc(e,t){let n=arguments.length>2&&void 0!==arguments[2]?arguments[2]:\"datum\";if(e){if(Zo(e))return Fn(e.value);if(Jo(e)){const{format:i,formatType:r}=ma(e);return vo({fieldOrDatumDef:e,format:i,formatType:r,expr:n,config:t})}}}function Bc(e){let n=arguments.length>1&&void 0!==arguments[1]?arguments[1]:{};const{encoding:i,markDef:r,config:o,stack:a}=e,s=i.tooltip;if(t.isArray(s))return{tooltip:Hc({tooltip:s},a,o,n)};{const l=n.reactiveGeom?\"datum.datum\":\"datum\";return Uc(e,s,\"tooltip\",(e=>{const s=Wc(e,o,l);if(s)return s;if(null===e)return;let c=Cn(\"tooltip\",r,o);return!0===c&&(c={content:\"encoding\"}),t.isString(c)?{value:c}:t.isObject(c)?yn(c)?c:\"encoding\"===c.content?Hc(i,a,o,n):{signal:l}:void 0}))}}function Ic(e,n,i){let{reactiveGeom:r}=arguments.length>3&&void 0!==arguments[3]?arguments[3]:{};const o={...i,...i.tooltipFormat},a={},s=r?\"datum.datum\":\"datum\",l=[];function c(i,r){const c=tt(r),u=Ko(i)?i:{...i,type:e[c].type},f=u.title||da(u,o),d=t.array(f).join(\", \").replaceAll(/\"/g,'\\\\\"');let m;if(zt(r)){const t=\"x\"===r?\"x2\":\"y2\",n=pa(e[t]);if(cn(u.bin)&&n){const e=oa(u,{expr:s}),i=oa(n,{expr:s}),{format:r,formatType:l}=ma(u);m=Fo(e,i,r,l,o),a[t]=!0}}if((zt(r)||r===se||r===oe)&&n&&n.fieldChannel===r&&\"normalize\"===n.offset){const{format:e,formatType:t}=ma(u);m=vo({fieldOrDatumDef:u,format:e,formatType:t,expr:s,config:o,normalizeStack:!0}).signal}m??=Wc(u,o,s).signal,l.push({channel:r,key:d,value:m})}Wa(e,((e,t)=>{Ho(e)?c(e,t):Bo(e)&&c(e.condition,t)}));const u={};for(const{channel:e,key:t,value:n}of l)a[e]||u[t]||(u[t]=n);return u}function Hc(e,t,n){let{reactiveGeom:i}=arguments.length>3&&void 0!==arguments[3]?arguments[3]:{};const r=Ic(e,t,n,{reactiveGeom:i}),o=z(r).map((e=>{let[t,n]=e;return`\"${t}\": ${n}`}));return o.length>0?{signal:`{${o.join(\", \")}}`}:void 0}function Vc(e){const{markDef:t,config:n}=e,i=Cn(\"aria\",t,n);return!1===i?{}:{...i?{aria:i}:{},...Gc(e),...Yc(e)}}function Gc(e){const{mark:t,markDef:n,config:i}=e;if(!1===i.aria)return{};const r=Cn(\"ariaRoleDescription\",n,i);return null!=r?{ariaRoleDescription:{value:r}}:t in $n?{}:{ariaRoleDescription:{value:t}}}function Yc(e){const{encoding:t,markDef:n,config:i,stack:r}=e,o=t.description;if(o)return Uc(e,o,\"description\",(t=>Wc(t,e.config)));const a=Cn(\"description\",n,i);if(null!=a)return{description:Fn(a)};if(!1===i.aria)return{};const s=Ic(t,r,i);return S(s)?void 0:{description:{signal:z(s).map(((e,t)=>{let[n,i]=e;return`\"${t>0?\"; \":\"\"}${n}: \" + (${i})`})).join(\" + \")}}}function Xc(e,t){let n=arguments.length>2&&void 0!==arguments[2]?arguments[2]:{};const{markDef:i,encoding:r,config:o}=t,{vgChannel:a}=n;let{defaultRef:s,defaultValue:l}=n;void 0===s&&(l??=Cn(e,i,o,{vgChannel:a,ignoreVgConfig:!0}),void 0!==l&&(s=Fn(l)));const c=r[e];return Uc(t,c,a??e,(n=>mo({channel:e,channelDef:n,markDef:i,config:o,scaleName:t.scaleName(e),scale:t.getScaleComponent(e),stack:null,defaultRef:s})))}function Qc(e){let t=arguments.length>1&&void 0!==arguments[1]?arguments[1]:{filled:void 0};const{markDef:n,encoding:i,config:r}=e,{type:o}=n,a=t.filled??Cn(\"filled\",n,r),s=p([\"bar\",\"point\",\"circle\",\"square\",\"geoshape\"],o)?\"transparent\":void 0,l=Cn(!0===a?\"color\":void 0,n,r,{vgChannel:\"fill\"})??r.mark[!0===a&&\"color\"]??s,c=Cn(!1===a?\"color\":void 0,n,r,{vgChannel:\"stroke\"})??r.mark[!1===a&&\"color\"],u=a?\"fill\":\"stroke\",f={...l?{fill:Fn(l)}:{},...c?{stroke:Fn(c)}:{}};return n.color&&(a?n.fill:n.stroke)&&$i(ri(\"property\",{fill:\"fill\"in n,stroke:\"stroke\"in n})),{...f,...Xc(\"color\",e,{vgChannel:u,defaultValue:a?l:c}),...Xc(\"fill\",e,{defaultValue:i.fill?l:void 0}),...Xc(\"stroke\",e,{defaultValue:i.stroke?c:void 0})}}function Jc(e){const{encoding:t,mark:n}=e,i=t.order;return!Qr(n)&&Zo(i)?Uc(e,i,\"zindex\",(e=>Fn(e.value))):{}}function Kc(e){let{channel:t,markDef:n,encoding:i={},model:r,bandPosition:o}=e;const a=`${t}Offset`,s=n[a],l=i[a];if((\"xOffset\"===a||\"yOffset\"===a)&&l){return{offsetType:\"encoding\",offset:mo({channel:a,channelDef:l,markDef:n,config:r?.config,scaleName:r.scaleName(a),scale:r.getScaleComponent(a),stack:null,defaultRef:Fn(s),bandPosition:o})}}const c=n[a];return c?{offsetType:\"visual\",offset:c}:{}}function Zc(e,t,n){let{defaultPos:i,vgChannel:r}=n;const{encoding:o,markDef:a,config:s,stack:l}=t,c=o[e],u=o[it(e)],f=t.scaleName(e),d=t.getScaleComponent(e),{offset:m,offsetType:p}=Kc({channel:e,markDef:a,encoding:o,model:t,bandPosition:.5}),g=eu({model:t,defaultPos:i,channel:e,scaleName:f,scale:d}),h=!c&&zt(e)&&(o.latitude||o.longitude)?{field:t.getName(e)}:function(e){const{channel:t,channelDef:n,scaleName:i,stack:r,offset:o,markDef:a}=e;if(Jo(n)&&r&&t===r.fieldChannel){if(Ho(n)){let e=n.bandPosition;if(void 0!==e||\"text\"!==a.type||\"radius\"!==t&&\"theta\"!==t||(e=.5),void 0!==e)return fo({scaleName:i,fieldOrDatumDef:n,startSuffix:\"start\",bandPosition:e,offset:o})}return uo(n,i,{suffix:\"end\"},{offset:o})}return so(e)}({channel:e,channelDef:c,channel2Def:u,markDef:a,config:s,scaleName:f,scale:d,stack:l,offset:m,defaultRef:g,bandPosition:\"encoding\"===p?0:void 0});return h?{[r||e]:h}:void 0}function eu(e){let{model:t,defaultPos:n,channel:i,scaleName:r,scale:o}=e;const{markDef:a,config:s}=t;return()=>{const e=tt(i),l=nt(i),c=Cn(i,a,s,{vgChannel:l});if(void 0!==c)return po(i,c);switch(n){case\"zeroOrMin\":case\"zeroOrMax\":if(r){const e=o.get(\"type\");if(p([cr.LOG,cr.TIME,cr.UTC],e));else if(o.domainDefinitelyIncludesZero())return{scale:r,value:0}}if(\"zeroOrMin\"===n)return\"y\"===e?{field:{group:\"height\"}}:{value:0};switch(e){case\"radius\":return{signal:`min(${t.width.signal},${t.height.signal})/2`};case\"theta\":return{signal:\"2*PI\"};case\"x\":return{field:{group:\"width\"}};case\"y\":return{value:0}}break;case\"mid\":return{...t[rt(i)],mult:.5}}}}const tu={left:\"x\",center:\"xc\",right:\"x2\"},nu={top:\"y\",middle:\"yc\",bottom:\"y2\"};function iu(e,t,n){let i=arguments.length>3&&void 0!==arguments[3]?arguments[3]:\"middle\";if(\"radius\"===e||\"theta\"===e)return nt(e);const r=\"x\"===e?\"align\":\"baseline\",o=Cn(r,t,n);let a;return yn(o)?($i(function(e){return`The ${e} for range marks cannot be an expression`}(r)),a=void 0):a=o,\"x\"===e?tu[a||(\"top\"===i?\"left\":\"center\")]:nu[a||i]}function ru(e,t,n){let{defaultPos:i,defaultPos2:r,range:o}=n;return o?ou(e,t,{defaultPos:i,defaultPos2:r}):Zc(e,t,{defaultPos:i})}function ou(e,t,n){let{defaultPos:i,defaultPos2:r}=n;const{markDef:o,config:a}=t,s=it(e),l=rt(e),c=function(e,t,n){const{encoding:i,mark:r,markDef:o,stack:a,config:s}=e,l=tt(n),c=rt(n),u=nt(n),f=i[l],d=e.scaleName(l),m=e.getScaleComponent(l),{offset:p}=Kc(n in i||n in o?{channel:n,markDef:o,encoding:i,model:e}:{channel:l,markDef:o,encoding:i,model:e});if(!f&&(\"x2\"===n||\"y2\"===n)&&(i.latitude||i.longitude)){const t=rt(n),i=e.markDef[t];return null!=i?{[t]:{value:i}}:{[u]:{field:e.getName(n)}}}const g=function(e){let{channel:t,channelDef:n,channel2Def:i,markDef:r,config:o,scaleName:a,scale:s,stack:l,offset:c,defaultRef:u}=e;if(Jo(n)&&l&&t.charAt(0)===l.fieldChannel.charAt(0))return uo(n,a,{suffix:\"start\"},{offset:c});return so({channel:t,channelDef:i,scaleName:a,scale:s,stack:l,markDef:r,config:o,offset:c,defaultRef:u})}({channel:n,channelDef:f,channel2Def:i[n],markDef:o,config:s,scaleName:d,scale:m,stack:a,offset:p,defaultRef:void 0});if(void 0!==g)return{[u]:g};return au(n,o)||au(n,{[n]:An(n,o,s.style),[c]:An(c,o,s.style)})||au(n,s[r])||au(n,s.mark)||{[u]:eu({model:e,defaultPos:t,channel:n,scaleName:d,scale:m})()}}(t,r,s);return{...Zc(e,t,{defaultPos:i,vgChannel:c[l]?iu(e,o,a):nt(e)}),...c}}function au(e,t){const n=rt(e),i=nt(e);if(void 0!==t[i])return{[i]:po(e,t[i])};if(void 0!==t[e])return{[i]:po(e,t[e])};if(t[n]){const i=t[n];if(!io(i))return{[n]:po(e,i)};$i(function(e){return`Position range does not support relative band size for ${e}.`}(n))}}function su(e,n){const{config:i,encoding:r,markDef:o}=e,a=o.type,s=it(n),l=rt(n),c=r[n],u=r[s],f=e.getScaleComponent(n),d=f?f.get(\"type\"):void 0,m=o.orient,p=r[l]??r.size??Cn(\"size\",o,i,{vgChannel:l}),g=ot(n),h=\"bar\"===a&&(\"x\"===n?\"vertical\"===m:\"horizontal\"===m);return!Ho(c)||!(ln(c.bin)||cn(c.bin)||c.timeUnit&&!u)||p&&!io(p)||r[g]||xr(d)?(Jo(c)&&xr(d)||h)&&!u?function(e,n,i){const{markDef:r,encoding:o,config:a,stack:s}=i,l=r.orient,c=i.scaleName(n),u=i.getScaleComponent(n),f=rt(n),d=it(n),m=ot(n),p=i.scaleName(m),g=i.getScaleComponent(at(n)),h=\"horizontal\"===l&&\"y\"===n||\"vertical\"===l&&\"x\"===n;let y;(o.size||r.size)&&(h?y=Xc(\"size\",i,{vgChannel:f,defaultRef:Fn(r.size)}):$i(function(e){return`Cannot apply size to non-oriented mark \"${e}\".`}(r.type)));const v=!!y,b=qo({channel:n,fieldDef:e,markDef:r,config:a,scaleType:(u||g)?.get(\"type\"),useVlSizeChannel:h});y=y||{[f]:lu(f,p||c,g||u,a,b,!!e,r.type)};const x=\"band\"===(u||g)?.get(\"type\")&&io(b)&&!v?\"top\":\"middle\",$=iu(n,r,a,x),w=\"xc\"===$||\"yc\"===$,{offset:k,offsetType:S}=Kc({channel:n,markDef:r,encoding:o,model:i,bandPosition:w?.5:0}),D=so({channel:n,channelDef:e,markDef:r,config:a,scaleName:c,scale:u,stack:s,offset:k,defaultRef:eu({model:i,defaultPos:\"mid\",channel:n,scaleName:c,scale:u}),bandPosition:w?\"encoding\"===S?0:.5:yn(b)?{signal:`(1-${b})/2`}:io(b)?(1-b.band)/2:0});if(f)return{[$]:D,...y};{const e=nt(d),n=y[f],i=k?{...n,offset:k}:n;return{[$]:D,[e]:t.isArray(D)?[D[0],{...D[1],offset:i}]:{...D,offset:i}}}}(c,n,e):ou(n,e,{defaultPos:\"zeroOrMax\",defaultPos2:\"zeroOrMin\"}):function(e){let{fieldDef:t,fieldDef2:n,channel:i,model:r}=e;const{config:o,markDef:a,encoding:s}=r,l=r.getScaleComponent(i),c=r.scaleName(i),u=l?l.get(\"type\"):void 0,f=l.get(\"reverse\"),d=qo({channel:i,fieldDef:t,markDef:a,config:o,scaleType:u}),m=r.component.axes[i]?.[0],p=m?.get(\"translate\")??.5,g=zt(i)?Cn(\"binSpacing\",a,o)??0:0,h=it(i),y=nt(i),v=nt(h),b=Pn(\"minBandSize\",a,o),{offset:x}=Kc({channel:i,markDef:a,encoding:s,model:r,bandPosition:0}),{offset:$}=Kc({channel:h,markDef:a,encoding:s,model:r,bandPosition:0}),w=function(e){let{scaleName:t,fieldDef:n}=e;const i=oa(n,{expr:\"datum\"});return`abs(scale(\"${t}\", ${oa(n,{expr:\"datum\",suffix:\"end\"})}) - scale(\"${t}\", ${i}))`}({fieldDef:t,scaleName:c}),k=cu(i,g,f,p,x,b,w),S=cu(h,g,f,p,$??x,b,w),D=yn(d)?{signal:`(1-${d.signal})/2`}:io(d)?(1-d.band)/2:.5,F=Lo({fieldDef:t,fieldDef2:n,markDef:a,config:o});if(ln(t.bin)||t.timeUnit){const e=t.timeUnit&&.5!==F;return{[v]:uu({fieldDef:t,scaleName:c,bandPosition:D,offset:S,useRectOffsetField:e}),[y]:uu({fieldDef:t,scaleName:c,bandPosition:yn(D)?{signal:`1-${D.signal}`}:1-D,offset:k,useRectOffsetField:e})}}if(cn(t.bin)){const e=uo(t,c,{},{offset:S});if(Ho(n))return{[v]:e,[y]:uo(n,c,{},{offset:k})};if(un(t.bin)&&t.bin.step)return{[v]:e,[y]:{signal:`scale(\"${c}\", ${oa(t,{expr:\"datum\"})} + ${t.bin.step})`,offset:k}}}return void $i(vi(h))}({fieldDef:c,fieldDef2:u,channel:n,model:e})}function lu(e,n,i,r,o,a,s){if(io(o)){if(!i)return{mult:o.band,field:{group:e}};{const e=i.get(\"type\");if(\"band\"===e){let e=`bandwidth('${n}')`;1!==o.band&&(e=`${o.band} * ${e}`);const t=Pn(\"minBandSize\",{type:s},r);return{signal:t?`max(${On(t)}, ${e})`:e}}1!==o.band&&($i(function(e){return`Cannot use the relative band size with ${e} scale.`}(e)),o=void 0)}}else{if(yn(o))return o;if(o)return{value:o}}if(i){const e=i.get(\"range\");if(vn(e)&&t.isNumber(e.step))return{value:e.step-2}}if(!a){const{bandPaddingInner:n,barBandPaddingInner:i,rectBandPaddingInner:o}=r.scale,a=U(n,\"bar\"===s?i:o);if(yn(a))return{signal:`(1 - (${a.signal})) * ${e}`};if(t.isNumber(a))return{signal:`${1-a} * ${e}`}}return{value:js(r.view,e)-2}}function cu(e,t,n,i,r,o,a){if(Ae(e))return 0;const s=\"x\"===e||\"y2\"===e,l=s?-t/2:t/2;if(yn(n)||yn(r)||yn(i)||o){const e=On(n),t=On(r),c=On(i),u=On(o),f=o?`(${a} < ${u} ? ${s?\"\":\"-\"}0.5 * (${u} - (${a})) : ${l})`:l;return{signal:(c?`${c} + `:\"\")+(e?`(${e} ? -1 : 1) * `:\"\")+(t?`(${t} + ${f})`:f)}}return r=r||0,i+(n?-r-l:+r+l)}function uu(e){let{fieldDef:t,scaleName:n,bandPosition:i,offset:r,useRectOffsetField:o}=e;return fo({scaleName:n,fieldOrDatumDef:t,bandPosition:i,offset:r,...o?{startSuffix:kc,endSuffix:Sc}:{}})}const fu=new Set([\"aria\",\"width\",\"height\"]);function du(e,t){const{fill:n,stroke:i}=\"include\"===t.color?Qc(e):{};return{...pu(e.markDef,t),...mu(e,\"fill\",n),...mu(e,\"stroke\",i),...Xc(\"opacity\",e),...Xc(\"fillOpacity\",e),...Xc(\"strokeOpacity\",e),...Xc(\"strokeWidth\",e),...Xc(\"strokeDash\",e),...Jc(e),...Bc(e),...Rc(e,\"href\"),...Vc(e)}}function mu(e,n,i){const{config:r,mark:o,markDef:a}=e;if(\"hide\"===Cn(\"invalid\",a,r)&&i&&!Qr(o)){const r=function(e,t){let{invalid:n=!1,channels:i}=t;const r=i.reduce(((t,n)=>{const i=e.getScaleComponent(n);if(i){const r=i.get(\"type\"),o=e.vgField(n,{expr:\"datum\"});o&&$r(r)&&(t[o]=!0)}return t}),{}),o=D(r);if(o.length>0){const e=n?\"||\":\"&&\";return o.map((e=>co(e,n))).join(` ${e} `)}return}(e,{invalid:!0,channels:It});if(r)return{[n]:[{test:r,value:null},...t.array(i)]}}return i?{[n]:i}:{}}function pu(e,t){return xn.reduce(((n,i)=>(fu.has(i)||void 0===e[i]||\"ignore\"===t[i]||(n[i]=Fn(e[i])),n)),{})}function gu(e){const{config:t,markDef:n}=e;if(Cn(\"invalid\",n,t)){const t=function(e,t){let{invalid:n=!1,channels:i}=t;const r=i.reduce(((t,n)=>{const i=e.getScaleComponent(n);if(i){const r=i.get(\"type\"),o=e.vgField(n,{expr:\"datum\",binSuffix:e.stack?.impute?\"mid\":void 0});o&&$r(r)&&(t[o]=!0)}return t}),{}),o=D(r);if(o.length>0){const e=n?\"||\":\"&&\";return o.map((e=>co(e,n))).join(` ${e} `)}return}(e,{channels:Ft});if(t)return{defined:{signal:t}}}return{}}function hu(e,t){if(void 0!==t)return{[e]:Fn(t)}}const yu=\"voronoi\",vu={defined:e=>\"point\"===e.type&&e.nearest,parse:(e,t)=>{if(t.events)for(const n of t.events)n.markname=e.getName(yu)},marks:(e,t,n)=>{const{x:i,y:r}=t.project.hasChannel,o=e.mark;if(Qr(o))return $i(`The \"nearest\" transform is not supported for ${o} marks.`),n;const a={name:e.getName(yu),type:\"path\",interactive:!0,from:{data:e.getName(\"marks\")},encode:{update:{fill:{value:\"transparent\"},strokeWidth:{value:.35},stroke:{value:\"transparent\"},isVoronoi:{value:!0},...Bc(e,{reactiveGeom:!0})}},transform:[{type:\"voronoi\",x:{expr:i||!r?\"datum.datum.x || 0\":\"0\"},y:{expr:r||!i?\"datum.datum.y || 0\":\"0\"},size:[e.getSizeSignalRef(\"width\"),e.getSizeSignalRef(\"height\")]}]};let s=0,l=!1;return n.forEach(((t,n)=>{const i=t.name??\"\";i===e.component.mark[0].name?s=n:i.indexOf(yu)>=0&&(l=!0)})),l||n.splice(s+1,0,a),n}},bu={defined:e=>\"point\"===e.type&&\"global\"===e.resolve&&e.bind&&\"scales\"!==e.bind&&!ws(e.bind),parse:(e,t,n)=>qu(t,n),topLevelSignals:(e,n,i)=>{const r=n.name,o=n.project,a=n.bind,s=n.init&&n.init[0],l=vu.defined(n)?\"(item().isVoronoi ? datum.datum : datum)\":\"datum\";return o.items.forEach(((e,o)=>{const c=_(`${r}_${e.field}`);i.filter((e=>e.name===c)).length||i.unshift({name:c,...s?{init:mc(s[o])}:{value:null},on:n.events?[{events:n.events,update:`datum && item().mark.marktype !== 'group' ? ${l}[${t.stringValue(e.field)}] : null`}]:[],bind:a[e.field]??a[e.channel]??a})})),i},signals:(e,t,n)=>{const i=t.name,r=t.project,o=n.filter((e=>e.name===i+Au))[0],a=i+Oc,s=r.items.map((e=>_(`${i}_${e.field}`))),l=s.map((e=>`${e} !== null`)).join(\" && \");return s.length&&(o.update=`${l} ? {fields: ${a}, values: [${s.join(\", \")}]} : null`),delete o.value,delete o.on,n}},xu=\"_toggle\",$u={defined:e=>\"point\"===e.type&&!!e.toggle,signals:(e,t,n)=>n.concat({name:t.name+xu,value:!1,on:[{events:t.events,update:t.toggle}]}),modifyExpr:(e,t)=>{const n=t.name+Au,i=t.name+xu;return`${i} ? null : ${n}, `+(\"global\"===t.resolve?`${i} ? null : true, `:`${i} ? null : {unit: ${Mu(e)}}, `)+`${i} ? ${n} : null`}},wu={defined:e=>void 0!==e.clear&&!1!==e.clear,parse:(e,n)=>{n.clear&&(n.clear=t.isString(n.clear)?t.parseSelector(n.clear,\"view\"):n.clear)},topLevelSignals:(e,t,n)=>{if(bu.defined(t))for(const e of t.project.items){const i=n.findIndex((n=>n.name===_(`${t.name}_${e.field}`)));-1!==i&&n[i].on.push({events:t.clear,update:\"null\"})}return n},signals:(e,t,n)=>{function i(e,i){-1!==e&&n[e].on&&n[e].on.push({events:t.clear,update:i})}if(\"interval\"===t.type)for(const e of t.project.items){const t=n.findIndex((t=>t.name===e.signals.visual));if(i(t,\"[0, 0]\"),-1===t){i(n.findIndex((t=>t.name===e.signals.data)),\"null\")}}else{let e=n.findIndex((e=>e.name===t.name+Au));i(e,\"null\"),$u.defined(t)&&(e=n.findIndex((e=>e.name===t.name+xu)),i(e,\"false\"))}return n}},ku={defined:e=>{const t=\"global\"===e.resolve&&e.bind&&ws(e.bind),n=1===e.project.items.length&&e.project.items[0].field!==xs;return t&&!n&&$i(\"Legend bindings are only supported for selections over an individual field or encoding channel.\"),t&&n},parse:(e,n,i)=>{const r=l(i);if(r.select=t.isString(r.select)?{type:r.select,toggle:n.toggle}:{...r.select,toggle:n.toggle},qu(n,r),t.isObject(i.select)&&(i.select.on||i.select.clear)){const e='event.item && indexof(event.item.mark.role, \"legend\") < 0';for(const i of n.events)i.filter=t.array(i.filter??[]),i.filter.includes(e)||i.filter.push(e)}const o=ks(n.bind)?n.bind.legend:\"click\",a=t.isString(o)?t.parseSelector(o,\"view\"):t.array(o);n.bind={legend:{merge:a}}},topLevelSignals:(e,t,n)=>{const i=t.name,r=ks(t.bind)&&t.bind.legend,o=e=>t=>{const n=l(t);return n.markname=e,n};for(const e of t.project.items){if(!e.hasLegend)continue;const a=`${_(e.field)}_legend`,s=`${i}_${a}`;if(0===n.filter((e=>e.name===s)).length){const e=r.merge.map(o(`${a}_symbols`)).concat(r.merge.map(o(`${a}_labels`))).concat(r.merge.map(o(`${a}_entries`)));n.unshift({name:s,...t.init?{}:{value:null},on:[{events:e,update:\"isDefined(datum.value) ? datum.value : item().items[0].items[0].datum.value\",force:!0},{events:r.merge,update:`!event.item || !datum ? null : ${s}`,force:!0}]})}}return n},signals:(e,t,n)=>{const i=t.name,r=t.project,o=n.find((e=>e.name===i+Au)),a=i+Oc,s=r.items.filter((e=>e.hasLegend)).map((e=>_(`${i}_${_(e.field)}_legend`))),l=`${s.map((e=>`${e} !== null`)).join(\" && \")} ? {fields: ${a}, values: [${s.join(\", \")}]} : null`;t.events&&s.length>0?o.on.push({events:s.map((e=>({signal:e}))),update:l}):s.length>0&&(o.update=l,delete o.value,delete o.on);const c=n.find((e=>e.name===i+xu)),u=ks(t.bind)&&t.bind.legend;return c&&(t.events?c.on.push({...c.on[0],events:u}):c.on[0].events=u),n}};const Su=\"_translate_anchor\",Du=\"_translate_delta\",Fu={defined:e=>\"interval\"===e.type&&e.translate,signals:(e,n,i)=>{const r=n.name,o=Cc.defined(n),a=r+Su,{x:s,y:l}=n.project.hasChannel;let c=t.parseSelector(n.translate,\"scope\");return o||(c=c.map((e=>(e.between[0].markname=r+jc,e)))),i.push({name:a,value:{},on:[{events:c.map((e=>e.between[0])),update:\"{x: x(unit), y: y(unit)\"+(void 0!==s?`, extent_x: ${o?Pc(e,Z):`slice(${s.signals.visual})`}`:\"\")+(void 0!==l?`, extent_y: ${o?Pc(e,ee):`slice(${l.signals.visual})`}`:\"\")+\"}\"}]},{name:r+Du,value:{},on:[{events:c,update:`{x: ${a}.x - x(unit), y: ${a}.y - y(unit)}`}]}),void 0!==s&&zu(e,n,s,\"width\",i),void 0!==l&&zu(e,n,l,\"height\",i),i}};function zu(e,t,n,i,r){const o=t.name,a=o+Su,s=o+Du,l=n.channel,c=Cc.defined(t),u=r.filter((e=>e.name===n.signals[c?\"data\":\"visual\"]))[0],f=e.getSizeSignalRef(i).signal,d=e.getScaleComponent(l),m=d&&d.get(\"type\"),p=d&&d.get(\"reverse\"),g=`${a}.extent_${l}`,h=`${c&&d?\"log\"===m?\"panLog\":\"symlog\"===m?\"panSymlog\":\"pow\"===m?\"panPow\":\"panLinear\":\"panLinear\"}(${g}, ${`${c?l===Z?p?\"\":\"-\":p?\"-\":\"\":\"\"}${s}.${l} / ${c?`${f}`:`span(${g})`}`}${c?\"pow\"===m?`, ${d.get(\"exponent\")??1}`:\"symlog\"===m?`, ${d.get(\"constant\")??1}`:\"\":\"\"})`;u.on.push({events:{signal:s},update:c?h:`clampRange(${h}, 0, ${f})`})}const Ou=\"_zoom_anchor\",_u=\"_zoom_delta\",Nu={defined:e=>\"interval\"===e.type&&e.zoom,signals:(e,n,i)=>{const r=n.name,o=Cc.defined(n),a=r+_u,{x:s,y:l}=n.project.hasChannel,c=t.stringValue(e.scaleName(Z)),u=t.stringValue(e.scaleName(ee));let f=t.parseSelector(n.zoom,\"scope\");return o||(f=f.map((e=>(e.markname=r+jc,e)))),i.push({name:r+Ou,on:[{events:f,update:o?\"{\"+[c?`x: invert(${c}, x(unit))`:\"\",u?`y: invert(${u}, y(unit))`:\"\"].filter((e=>e)).join(\", \")+\"}\":\"{x: x(unit), y: y(unit)}\"}]},{name:a,on:[{events:f,force:!0,update:\"pow(1.001, event.deltaY * pow(16, event.deltaMode))\"}]}),void 0!==s&&Cu(e,n,s,\"width\",i),void 0!==l&&Cu(e,n,l,\"height\",i),i}};function Cu(e,t,n,i,r){const o=t.name,a=n.channel,s=Cc.defined(t),l=r.filter((e=>e.name===n.signals[s?\"data\":\"visual\"]))[0],c=e.getSizeSignalRef(i).signal,u=e.getScaleComponent(a),f=u&&u.get(\"type\"),d=s?Pc(e,a):l.name,m=o+_u,p=`${s&&u?\"log\"===f?\"zoomLog\":\"symlog\"===f?\"zoomSymlog\":\"pow\"===f?\"zoomPow\":\"zoomLinear\":\"zoomLinear\"}(${d}, ${`${o}${Ou}.${a}`}, ${m}${s?\"pow\"===f?`, ${u.get(\"exponent\")??1}`:\"symlog\"===f?`, ${u.get(\"constant\")??1}`:\"\":\"\"})`;l.on.push({events:{signal:m},update:s?p:`clampRange(${p}, 0, ${c})`})}const Pu=\"_store\",Au=\"_tuple\",ju=\"_modify\",Tu=\"vlSelectionResolve\",Eu=[qc,Lc,Nc,$u,bu,Cc,ku,wu,Fu,Nu,vu];function Mu(e){let{escape:n}=arguments.length>1&&void 0!==arguments[1]?arguments[1]:{escape:!0},i=n?t.stringValue(e.name):e.name;const r=function(e){let t=e.parent;for(;t&&!$m(t);)t=t.parent;return t}(e);if(r){const{facet:e}=r;for(const n of Re)e[n]&&(i+=` + '__facet_${n}_' + (facet[${t.stringValue(r.vgField(n))}])`)}return i}function Lu(e){return F(e.component.selection??{}).reduce(((e,t)=>e||t.project.hasSelectionId),!1)}function qu(e,n){!t.isString(n.select)&&n.select.on||delete e.events,!t.isString(n.select)&&n.select.clear||delete e.clear,!t.isString(n.select)&&n.select.toggle||delete e.toggle}function Uu(e){const t=[];return\"Identifier\"===e.type?[e.name]:\"Literal\"===e.type?[e.value]:(\"MemberExpression\"===e.type&&(t.push(...Uu(e.object)),t.push(...Uu(e.property))),t)}function Ru(e){return\"MemberExpression\"===e.object.type?Ru(e.object):\"datum\"===e.object.name}function Wu(e){const n=t.parseExpression(e),i=new Set;return n.visit((e=>{\"MemberExpression\"===e.type&&Ru(e)&&i.add(Uu(e).slice(1).join(\".\"))})),i}class Bu extends vc{clone(){return new Bu(null,this.model,l(this.filter))}constructor(e,t,n){super(e),this.model=t,this.filter=n,qn(this,\"expr\",void 0),qn(this,\"_dependentFields\",void 0),this.expr=Vu(this.model,this.filter,this),this._dependentFields=Wu(this.expr)}dependentFields(){return this._dependentFields}producedFields(){return new Set}assemble(){return{type:\"filter\",expr:this.expr}}hash(){return`Filter ${this.expr}`}}function Iu(e,n,i){let r=arguments.length>3&&void 0!==arguments[3]?arguments[3]:\"datum\";const o=t.isString(n)?n:n.param,a=_(o),s=t.stringValue(a+Pu);let l;try{l=e.getSelectionComponent(a,o)}catch(e){return`!!${a}`}if(l.project.timeUnit){const t=i??e.component.data.raw,n=l.project.timeUnit.clone();t.parent?n.insertAsParentOf(t):t.parent=n}const c=`${l.project.hasSelectionId?\"vlSelectionIdTest(\":\"vlSelectionTest(\"}${s}, ${r}${\"global\"===l.resolve?\")\":`, ${t.stringValue(l.resolve)})`}`,u=`length(data(${s}))`;return!1===n.empty?`${u} && ${c}`:`!${u} || ${c}`}function Hu(e,n,i){const r=_(n),o=i.encoding;let a,s=i.field;try{a=e.getSelectionComponent(r,n)}catch(e){return r}if(o||s){if(o&&!s){const e=a.project.items.filter((e=>e.channel===o));!e.length||e.length>1?(s=a.project.items[0].field,$i((e.length?\"Multiple \":\"No \")+`matching ${t.stringValue(o)} encoding found for selection ${t.stringValue(i.param)}. `+`Using \"field\": ${t.stringValue(s)}.`)):s=e[0].field}}else s=a.project.items[0].field,a.project.items.length>1&&$i(`A \"field\" or \"encoding\" must be specified when using a selection as a scale domain. Using \"field\": ${t.stringValue(s)}.`);return`${a.name}[${t.stringValue(E(s))}]`}function Vu(e,n,i){return N(n,(n=>t.isString(n)?n:function(e){return e?.param}(n)?Iu(e,n,i):Zi(n)))}function Gu(e,t,n,i){e.encode??={},e.encode[t]??={},e.encode[t].update??={},e.encode[t].update[n]=i}function Yu(e,n,i){let r=arguments.length>3&&void 0!==arguments[3]?arguments[3]:{header:!1};const{disable:o,orient:a,scale:s,labelExpr:l,title:c,zindex:u,...f}=e.combine();if(!o){for(const e in f){const i=Oa[e],r=f[e];if(i&&i!==n&&\"both\"!==i)delete f[e];else if(Fa(r)){const{condition:n,...i}=r,o=t.array(n),a=Da[e];if(a){const{vgProp:t,part:n}=a;Gu(f,n,t,[...o.map((e=>{const{test:t,...n}=e;return{test:Vu(null,t),...n}})),i]),delete f[e]}else if(null===a){const t={signal:o.map((e=>{const{test:t,...n}=e;return`${Vu(null,t)} ? ${zn(n)} : `})).join(\"\")+zn(i)};f[e]=t}}else if(yn(r)){const t=Da[e];if(t){const{vgProp:n,part:i}=t;Gu(f,i,n,r),delete f[e]}}p([\"labelAlign\",\"labelBaseline\"],e)&&null===f[e]&&delete f[e]}if(\"grid\"===n){if(!f.grid)return;if(f.encode){const{grid:e}=f.encode;f.encode={...e?{grid:e}:{}},S(f.encode)&&delete f.encode}return{scale:s,orient:a,...f,domain:!1,labels:!1,aria:!1,maxExtent:0,minExtent:0,ticks:!1,zindex:U(u,0)}}{if(!r.header&&e.mainExtracted)return;if(void 0!==l){let e=l;f.encode?.labels?.update&&yn(f.encode.labels.update.text)&&(e=M(l,\"datum.label\",f.encode.labels.update.text.signal)),Gu(f,\"labels\",\"text\",{signal:e})}if(null===f.labelAlign&&delete f.labelAlign,f.encode){for(const t of za)e.hasAxisPart(t)||delete f.encode[t];S(f.encode)&&delete f.encode}const n=function(e,n){if(e)return t.isArray(e)&&!hn(e)?e.map((e=>da(e,n))).join(\", \"):e}(c,i);return{scale:s,orient:a,grid:!1,...n?{title:n}:{},...f,...!1===i.aria?{aria:!1}:{},zindex:U(u,0)}}}}function Xu(e){const{axes:t}=e.component,n=[];for(const i of Ft)if(t[i])for(const r of t[i])if(!r.get(\"disable\")&&!r.get(\"gridScale\")){const t=\"x\"===i?\"height\":\"width\",r=e.getSizeSignalRef(t).signal;t!==r&&n.push({name:t,update:r})}return n}function Qu(e,t,n,i){return Object.assign.apply(null,[{},...e.map((e=>{if(\"axisOrient\"===e){const e=\"x\"===n?\"bottom\":\"left\",r=t[\"x\"===n?\"axisBottom\":\"axisLeft\"]||{},o=t[\"x\"===n?\"axisTop\":\"axisRight\"]||{},a=new Set([...D(r),...D(o)]),s={};for(const t of a.values())s[t]={signal:`${i.signal} === \"${e}\" ? ${On(r[t])} : ${On(o[t])}`};return s}return t[e]}))])}function Ju(e,n){const i=[{}];for(const r of e){let e=n[r]?.style;if(e){e=t.array(e);for(const t of e)i.push(n.style[t])}}return Object.assign.apply(null,i)}function Ku(e,t,n){let i=arguments.length>3&&void 0!==arguments[3]?arguments[3]:{};const r=jn(e,n,t);if(void 0!==r)return{configFrom:\"style\",configValue:r};for(const t of[\"vlOnlyAxisConfig\",\"vgAxisConfig\",\"axisConfigStyle\"])if(void 0!==i[t]?.[e])return{configFrom:t,configValue:i[t][e]};return{}}const Zu={scale:e=>{let{model:t,channel:n}=e;return t.scaleName(n)},format:e=>{let{format:t}=e;return t},formatType:e=>{let{formatType:t}=e;return t},grid:e=>{let{fieldOrDatumDef:t,axis:n,scaleType:i}=e;return n.grid??function(e,t){return!xr(e)&&Ho(t)&&!ln(t?.bin)&&!cn(t?.bin)}(i,t)},gridScale:e=>{let{model:t,channel:n}=e;return function(e,t){const n=\"x\"===t?\"y\":\"x\";if(e.getScaleComponent(n))return e.scaleName(n);return}(t,n)},labelAlign:e=>{let{axis:t,labelAngle:n,orient:i,channel:r}=e;return t.labelAlign||nf(n,i,r)},labelAngle:e=>{let{labelAngle:t}=e;return t},labelBaseline:e=>{let{axis:t,labelAngle:n,orient:i,channel:r}=e;return t.labelBaseline||tf(n,i,r)},labelFlush:e=>{let{axis:t,fieldOrDatumDef:n,channel:i}=e;return t.labelFlush??function(e,t){if(\"x\"===t&&p([\"quantitative\",\"temporal\"],e))return!0;return}(n.type,i)},labelOverlap:e=>{let{axis:n,fieldOrDatumDef:i,scaleType:r}=e;return n.labelOverlap??function(e,n,i,r){if(i&&!t.isObject(r)||\"nominal\"!==e&&\"ordinal\"!==e)return\"log\"!==n&&\"symlog\"!==n||\"greedy\";return}(i.type,r,Ho(i)&&!!i.timeUnit,Ho(i)?i.sort:void 0)},orient:e=>{let{orient:t}=e;return t},tickCount:e=>{let{channel:t,model:n,axis:i,fieldOrDatumDef:r,scaleType:o}=e;const a=\"x\"===t?\"width\":\"y\"===t?\"height\":void 0,s=a?n.getSizeSignalRef(a):void 0;return i.tickCount??function(e){let{fieldOrDatumDef:t,scaleType:n,size:i,values:r}=e;if(!r&&!xr(n)&&\"log\"!==n){if(Ho(t)){if(ln(t.bin))return{signal:`ceil(${i.signal}/10)`};if(t.timeUnit&&p([\"month\",\"hours\",\"day\",\"quarter\"],Ui(t.timeUnit)?.unit))return}return{signal:`ceil(${i.signal}/40)`}}return}({fieldOrDatumDef:r,scaleType:o,size:s,values:i.values})},tickMinStep:function(e){let{format:t,fieldOrDatumDef:n}=e;if(\"d\"===t)return 1;if(Ho(n)){const{timeUnit:e}=n;if(e){const t=Ri(e);if(t)return{signal:t}}}return},title:e=>{let{axis:t,model:n,channel:i}=e;if(void 0!==t.title)return t.title;const r=rf(n,i);if(void 0!==r)return r;const o=n.typedFieldDef(i),a=\"x\"===i?\"x2\":\"y2\",s=n.fieldDef(a);return En(o?[Eo(o)]:[],Ho(s)?[Eo(s)]:[])},values:e=>{let{axis:n,fieldOrDatumDef:i}=e;return function(e,n){const i=e.values;if(t.isArray(i))return ka(n,i);if(yn(i))return i;return}(n,i)},zindex:e=>{let{axis:t,fieldOrDatumDef:n,mark:i}=e;return t.zindex??function(e,t){if(\"rect\"===e&&aa(t))return 1;return 0}(i,n)}};function ef(e){return`(((${e.signal} % 360) + 360) % 360)`}function tf(e,t,n,i){if(void 0!==e){if(\"x\"===n){if(yn(e)){const n=ef(e);return{signal:`(45 < ${n} && ${n} < 135) || (225 < ${n} && ${n} < 315) ? \"middle\" :(${n} <= 45 || 315 <= ${n}) === ${yn(t)?`(${t.signal} === \"top\")`:\"top\"===t} ? \"bottom\" : \"top\"`}}if(45{if(ea(t)&&Po(t.sort)){const{field:i,timeUnit:r}=t,o=t.sort,a=o.map(((e,t)=>`${Zi({field:i,timeUnit:r,equal:e})} ? ${t} : `)).join(\"\")+o.length;e=new of(e,{calculate:a,as:af(t,n,{forAs:!0})})}})),e}producedFields(){return new Set([this.transform.as])}dependentFields(){return this._dependentFields}assemble(){return{type:\"formula\",expr:this.transform.calculate,as:this.transform.as}}hash(){return`Calculate ${d(this.transform)}`}}function af(e,t,n){return oa(e,{prefix:t,suffix:\"sort_index\",...n??{}})}function sf(e,t){return p([\"top\",\"bottom\"],t)?\"column\":p([\"left\",\"right\"],t)||\"row\"===e?\"row\":\"column\"}function lf(e,t,n,i){const r=\"row\"===i?n.headerRow:\"column\"===i?n.headerColumn:n.headerFacet;return U((t||{})[e],r[e],n.header[e])}function cf(e,t,n,i){const r={};for(const o of e){const e=lf(o,t||{},n,i);void 0!==e&&(r[o]=e)}return r}const uf=[\"row\",\"column\"],ff=[\"header\",\"footer\"];function df(e,t){const n=e.component.layoutHeaders[t].title,i=e.config?e.config:void 0,r=e.component.layoutHeaders[t].facetFieldDef?e.component.layoutHeaders[t].facetFieldDef:void 0,{titleAnchor:o,titleAngle:a,titleOrient:s}=cf([\"titleAnchor\",\"titleAngle\",\"titleOrient\"],r.header,i,t),l=sf(t,s),c=H(a);return{name:`${t}-title`,type:\"group\",role:`${l}-title`,title:{text:n,...\"row\"===t?{orient:\"left\"}:{},style:\"guide-title\",...pf(c,l),...mf(l,c,o),...$f(i,r,t,hs,ps)}}}function mf(e,t){switch(arguments.length>2&&void 0!==arguments[2]?arguments[2]:\"middle\"){case\"start\":return{align:\"left\"};case\"end\":return{align:\"right\"}}const n=nf(t,\"row\"===e?\"left\":\"top\",\"row\"===e?\"y\":\"x\");return n?{align:n}:{}}function pf(e,t){const n=tf(e,\"row\"===t?\"left\":\"top\",\"row\"===t?\"y\":\"x\",!0);return n?{baseline:n}:{}}function gf(e,t){const n=e.component.layoutHeaders[t],i=[];for(const r of ff)if(n[r])for(const o of n[r]){const a=vf(e,t,r,n,o);null!=a&&i.push(a)}return i}function hf(e,n){const{sort:i}=e;return Co(i)?{field:oa(i,{expr:\"datum\"}),order:i.order??\"ascending\"}:t.isArray(i)?{field:af(e,n,{expr:\"datum\"}),order:\"ascending\"}:{field:oa(e,{expr:\"datum\"}),order:i??\"ascending\"}}function yf(e,t,n){const{format:i,formatType:r,labelAngle:o,labelAnchor:a,labelOrient:s,labelExpr:l}=cf([\"format\",\"formatType\",\"labelAngle\",\"labelAnchor\",\"labelOrient\",\"labelExpr\"],e.header,n,t),c=vo({fieldOrDatumDef:e,format:i,formatType:r,expr:\"parent\",config:n}).signal,u=sf(t,s);return{text:{signal:l?M(M(l,\"datum.label\",c),\"datum.value\",oa(e,{expr:\"parent\"})):c},...\"row\"===t?{orient:\"left\"}:{},style:\"guide-label\",frame:\"group\",...pf(o,u),...mf(u,o,a),...$f(n,e,t,ys,gs)}}function vf(e,t,n,i,r){if(r){let o=null;const{facetFieldDef:a}=i,s=e.config?e.config:void 0;if(a&&r.labels){const{labelOrient:e}=cf([\"labelOrient\"],a.header,s,t);(\"row\"===t&&!p([\"top\",\"bottom\"],e)||\"column\"===t&&!p([\"left\",\"right\"],e))&&(o=yf(a,t,s))}const l=$m(e)&&!Ao(e.facet),c=r.axes,u=c?.length>0;if(o||u){const s=\"row\"===t?\"height\":\"width\";return{name:e.getName(`${t}_${n}`),type:\"group\",role:`${t}-${n}`,...i.facetFieldDef?{from:{data:e.getName(`${t}_domain`)},sort:hf(a,t)}:{},...u&&l?{from:{data:e.getName(`facet_domain_${t}`)}}:{},...o?{title:o}:{},...r.sizeSignal?{encode:{update:{[s]:r.sizeSignal}}}:{},...u?{axes:c}:{}}}}return null}const bf={column:{start:0,end:1},row:{start:1,end:0}};function xf(e,t){return bf[t][e]}function $f(e,t,n,i,r){const o={};for(const a of i){if(!r[a])continue;const i=lf(a,t?.header,e,n);void 0!==i&&(o[r[a]]=i)}return o}function wf(e){return[...kf(e,\"width\"),...kf(e,\"height\"),...kf(e,\"childWidth\"),...kf(e,\"childHeight\")]}function kf(e,t){const n=\"width\"===t?\"x\":\"y\",i=e.component.layoutSize.get(t);if(!i||\"merged\"===i)return[];const r=e.getSizeSignalRef(t).signal;if(\"step\"===i){const t=e.getScaleComponent(n);if(t){const i=t.get(\"type\"),o=t.get(\"range\");if(xr(i)&&vn(o)){const i=e.scaleName(n);if($m(e.parent)){if(\"independent\"===e.parent.component.resolve.scale[n])return[Sf(i,o)]}return[Sf(i,o),{name:r,update:Df(i,t,`domain('${i}').length`)}]}}throw new Error(\"layout size is step although width/height is not step.\")}if(\"container\"==i){const t=r.endsWith(\"width\"),n=t?\"containerSize()[0]\":\"containerSize()[1]\",i=`isFinite(${n}) ? ${n} : ${As(e.config.view,t?\"width\":\"height\")}`;return[{name:r,init:i,on:[{update:i,events:\"window:resize\"}]}]}return[{name:r,value:i}]}function Sf(e,t){const n=`${e}_step`;return yn(t.step)?{name:n,update:t.step.signal}:{name:n,value:t.step}}function Df(e,t,n){const i=t.get(\"type\"),r=t.get(\"padding\"),o=U(t.get(\"paddingOuter\"),r);let a=t.get(\"paddingInner\");return a=\"band\"===i?void 0!==a?a:r:1,`bandspace(${n}, ${On(a)}, ${On(o)}) * ${e}_step`}function Ff(e){return\"childWidth\"===e?\"width\":\"childHeight\"===e?\"height\":e}function zf(e,t){return D(e).reduce(((n,i)=>{const r=e[i];return{...n,...Uc(t,r,i,(e=>Fn(e.value)))}}),{})}function Of(e,t){if($m(t))return\"theta\"===e?\"independent\":\"shared\";if(km(t))return\"shared\";if(wm(t))return zt(e)||\"theta\"===e||\"radius\"===e?\"independent\":\"shared\";throw new Error(\"invalid model type for resolve\")}function _f(e,t){const n=e.scale[t],i=zt(t)?\"axis\":\"legend\";return\"independent\"===n?(\"shared\"===e[i][t]&&$i(function(e){return`Setting the scale to be independent for \"${e}\" means we also have to set the guide (axis or legend) to be independent.`}(t)),\"independent\"):e[i][t]||\"shared\"}const Nf=D({aria:1,clipHeight:1,columnPadding:1,columns:1,cornerRadius:1,description:1,direction:1,fillColor:1,format:1,formatType:1,gradientLength:1,gradientOpacity:1,gradientStrokeColor:1,gradientStrokeWidth:1,gradientThickness:1,gridAlign:1,labelAlign:1,labelBaseline:1,labelColor:1,labelFont:1,labelFontSize:1,labelFontStyle:1,labelFontWeight:1,labelLimit:1,labelOffset:1,labelOpacity:1,labelOverlap:1,labelPadding:1,labelSeparation:1,legendX:1,legendY:1,offset:1,orient:1,padding:1,rowPadding:1,strokeColor:1,symbolDash:1,symbolDashOffset:1,symbolFillColor:1,symbolLimit:1,symbolOffset:1,symbolOpacity:1,symbolSize:1,symbolStrokeColor:1,symbolStrokeWidth:1,symbolType:1,tickCount:1,tickMinStep:1,title:1,titleAlign:1,titleAnchor:1,titleBaseline:1,titleColor:1,titleFont:1,titleFontSize:1,titleFontStyle:1,titleFontWeight:1,titleLimit:1,titleLineHeight:1,titleOpacity:1,titleOrient:1,titlePadding:1,type:1,values:1,zindex:1,disable:1,labelExpr:1,selections:1,opacity:1,shape:1,stroke:1,fill:1,size:1,strokeWidth:1,strokeDash:1,encode:1});class Cf extends Jl{}const Pf={symbols:function(e,n){let{fieldOrDatumDef:i,model:r,channel:o,legendCmpt:a,legendType:s}=n;if(\"symbol\"!==s)return;const{markDef:l,encoding:c,config:u,mark:f}=r,d=l.filled&&\"trail\"!==f;let m={..._n({},r,eo),...Qc(r,{filled:d})};const p=a.get(\"symbolOpacity\")??u.legend.symbolOpacity,g=a.get(\"symbolFillColor\")??u.legend.symbolFillColor,h=a.get(\"symbolStrokeColor\")??u.legend.symbolStrokeColor,y=void 0===p?Af(c.opacity)??l.opacity:void 0;if(m.fill)if(\"fill\"===o||d&&o===me)delete m.fill;else if(m.fill.field)g?delete m.fill:(m.fill=Fn(u.legend.symbolBaseFillColor??\"black\"),m.fillOpacity=Fn(y??1));else if(t.isArray(m.fill)){const e=jf(c.fill??c.color)??l.fill??(d&&l.color);e&&(m.fill=Fn(e))}if(m.stroke)if(\"stroke\"===o||!d&&o===me)delete m.stroke;else if(m.stroke.field||h)delete m.stroke;else if(t.isArray(m.stroke)){const e=U(jf(c.stroke||c.color),l.stroke,d?l.color:void 0);e&&(m.stroke={value:e})}if(o!==be){const e=Ho(i)&&Ef(r,a,i);e?m.opacity=[{test:e,...Fn(y??1)},Fn(u.legend.unselectedOpacity)]:y&&(m.opacity=Fn(y))}return m={...m,...e},S(m)?void 0:m},gradient:function(e,t){let{model:n,legendType:i,legendCmpt:r}=t;if(\"gradient\"!==i)return;const{config:o,markDef:a,encoding:s}=n;let l={};const c=void 0===(r.get(\"gradientOpacity\")??o.legend.gradientOpacity)?Af(s.opacity)||a.opacity:void 0;c&&(l.opacity=Fn(c));return l={...l,...e},S(l)?void 0:l},labels:function(e,t){let{fieldOrDatumDef:n,model:i,channel:r,legendCmpt:o}=t;const a=i.legend(r)||{},s=i.config,l=Ho(n)?Ef(i,o,n):void 0,c=l?[{test:l,value:1},{value:s.legend.unselectedOpacity}]:void 0,{format:u,formatType:f}=a;let d;go(f)?d=xo({fieldOrDatumDef:n,field:\"datum.value\",format:u,formatType:f,config:s}):void 0===u&&void 0===f&&s.customFormatTypes&&(\"quantitative\"===n.type&&s.numberFormatType?d=xo({fieldOrDatumDef:n,field:\"datum.value\",format:s.numberFormat,formatType:s.numberFormatType,config:s}):\"temporal\"===n.type&&s.timeFormatType&&Ho(n)&&void 0===n.timeUnit&&(d=xo({fieldOrDatumDef:n,field:\"datum.value\",format:s.timeFormat,formatType:s.timeFormatType,config:s})));const m={...c?{opacity:c}:{},...d?{text:d}:{},...e};return S(m)?void 0:m},entries:function(e,t){let{legendCmpt:n}=t;const i=n.get(\"selections\");return i?.length?{...e,fill:{value:\"transparent\"}}:e}};function Af(e){return Tf(e,((e,t)=>Math.max(e,t.value)))}function jf(e){return Tf(e,((e,t)=>U(e,t.value)))}function Tf(e,n){return function(e){const n=e?.condition;return!!n&&(t.isArray(n)||Zo(n))}(e)?t.array(e.condition).reduce(n,e.value):Zo(e)?e.value:void 0}function Ef(e,n,i){const r=n.get(\"selections\");if(!r?.length)return;const o=t.stringValue(i.field);return r.map((e=>`(!length(data(${t.stringValue(_(e)+Pu)})) || (${e}[${o}] && indexof(${e}[${o}], datum.value) >= 0))`)).join(\" || \")}const Mf={direction:e=>{let{direction:t}=e;return t},format:e=>{let{fieldOrDatumDef:t,legend:n,config:i}=e;const{format:r,formatType:o}=n;return $o(t,t.type,r,o,i,!1)},formatType:e=>{let{legend:t,fieldOrDatumDef:n,scaleType:i}=e;const{formatType:r}=t;return wo(r,n,i)},gradientLength:e=>{const{legend:t,legendConfig:n}=e;return t.gradientLength??n.gradientLength??function(e){let{legendConfig:t,model:n,direction:i,orient:r,scaleType:o}=e;const{gradientHorizontalMaxLength:a,gradientHorizontalMinLength:s,gradientVerticalMaxLength:l,gradientVerticalMinLength:c}=t;if(wr(o))return\"horizontal\"===i?\"top\"===r||\"bottom\"===r?Uf(n,\"width\",s,a):s:Uf(n,\"height\",c,l);return}(e)},labelOverlap:e=>{let{legend:t,legendConfig:n,scaleType:i}=e;return t.labelOverlap??n.labelOverlap??function(e){if(p([\"quantile\",\"threshold\",\"log\",\"symlog\"],e))return\"greedy\";return}(i)},symbolType:e=>{let{legend:t,markDef:n,channel:i,encoding:r}=e;return t.symbolType??function(e,t,n,i){if(\"shape\"!==t){const e=jf(n)??i;if(e)return e}switch(e){case\"bar\":case\"rect\":case\"image\":case\"square\":return\"square\";case\"line\":case\"trail\":case\"rule\":return\"stroke\";case\"arc\":case\"point\":case\"circle\":case\"tick\":case\"geoshape\":case\"area\":case\"text\":return\"circle\"}}(n.type,i,r.shape,n.shape)},title:e=>{let{fieldOrDatumDef:t,config:n}=e;return ua(t,n,{allowDisabling:!0})},type:e=>{let{legendType:t,scaleType:n,channel:i}=e;if(qe(i)&&wr(n)){if(\"gradient\"===t)return}else if(\"symbol\"===t)return;return t},values:e=>{let{fieldOrDatumDef:n,legend:i}=e;return function(e,n){const i=e.values;if(t.isArray(i))return ka(n,i);if(yn(i))return i;return}(i,n)}};function Lf(e){const{legend:t}=e;return U(t.type,function(e){let{channel:t,timeUnit:n,scaleType:i}=e;if(qe(t)){if(p([\"quarter\",\"month\",\"day\"],n))return\"symbol\";if(wr(i))return\"gradient\"}return\"symbol\"}(e))}function qf(e){let{legendConfig:t,legendType:n,orient:i,legend:r}=e;return r.direction??t[n?\"gradientDirection\":\"symbolDirection\"]??function(e,t){switch(e){case\"top\":case\"bottom\":return\"horizontal\";case\"left\":case\"right\":case\"none\":case void 0:return;default:return\"gradient\"===t?\"horizontal\":void 0}}(i,n)}function Uf(e,t,n,i){return{signal:`clamp(${e.getSizeSignalRef(t).signal}, ${n}, ${i})`}}function Rf(e){const t=xm(e)?function(e){const{encoding:t}=e,n={};for(const i of[me,...bs]){const r=ga(t[i]);r&&e.getScaleComponent(i)&&(i===he&&Ho(r)&&r.type===lr||(n[i]=Bf(e,i)))}return n}(e):function(e){const{legends:t,resolve:n}=e.component;for(const i of e.children){Rf(i);for(const r of D(i.component.legends))n.legend[r]=_f(e.component.resolve,r),\"shared\"===n.legend[r]&&(t[r]=If(t[r],i.component.legends[r]),t[r]||(n.legend[r]=\"independent\",delete t[r]))}for(const i of D(t))for(const t of e.children)t.component.legends[i]&&\"shared\"===n.legend[i]&&delete t.component.legends[i];return t}(e);return e.component.legends=t,t}function Wf(e,t,n,i){switch(t){case\"disable\":return void 0!==n;case\"values\":return!!n?.values;case\"title\":if(\"title\"===t&&e===i?.title)return!0}return e===(n||{})[t]}function Bf(e,t){let n=e.legend(t);const{markDef:i,encoding:r,config:o}=e,a=o.legend,s=new Cf({},function(e,t){const n=e.scaleName(t);if(\"trail\"===e.mark){if(\"color\"===t)return{stroke:n};if(\"size\"===t)return{strokeWidth:n}}return\"color\"===t?e.markDef.filled?{fill:n}:{stroke:n}:{[t]:n}}(e,t));!function(e,t,n){const i=e.fieldDef(t)?.field;for(const r of F(e.component.selection??{})){const e=r.project.hasField[i]??r.project.hasChannel[t];if(e&&ku.defined(r)){const t=n.get(\"selections\")??[];t.push(r.name),n.set(\"selections\",t,!1),e.hasLegend=!0}}}(e,t,s);const l=void 0!==n?!n:a.disable;if(s.set(\"disable\",l,void 0!==n),l)return s;n=n||{};const c=e.getScaleComponent(t).get(\"type\"),u=ga(r[t]),f=Ho(u)?Ui(u.timeUnit)?.unit:void 0,d=n.orient||o.legend.orient||\"right\",m=Lf({legend:n,channel:t,timeUnit:f,scaleType:c}),p={legend:n,channel:t,model:e,markDef:i,encoding:r,fieldOrDatumDef:u,legendConfig:a,config:o,scaleType:c,orient:d,legendType:m,direction:qf({legend:n,legendType:m,orient:d,legendConfig:a})};for(const i of Nf){if(\"gradient\"===m&&i.startsWith(\"symbol\")||\"symbol\"===m&&i.startsWith(\"gradient\"))continue;const r=i in Mf?Mf[i](p):n[i];if(void 0!==r){const a=Wf(r,i,n,e.fieldDef(t));(a||void 0===o.legend[i])&&s.set(i,r,a)}}const g=n?.encoding??{},h=s.get(\"selections\"),y={},v={fieldOrDatumDef:u,model:e,channel:t,legendCmpt:s,legendType:m};for(const t of[\"labels\",\"legend\",\"title\",\"symbols\",\"gradient\",\"entries\"]){const n=zf(g[t]??{},e),i=t in Pf?Pf[t](n,v):n;void 0===i||S(i)||(y[t]={...h?.length&&Ho(u)?{name:`${_(u.field)}_legend_${t}`}:{},...h?.length?{interactive:!!h}:{},update:i})}return S(y)||s.set(\"encode\",y,!!n?.encoding),s}function If(e,t){if(!e)return t.clone();const n=e.getWithExplicit(\"orient\"),i=t.getWithExplicit(\"orient\");if(n.explicit&&i.explicit&&n.value!==i.value)return;let r=!1;for(const n of Nf){const i=nc(e.getWithExplicit(n),t.getWithExplicit(n),n,\"legend\",((e,t)=>{switch(n){case\"symbolType\":return Hf(e,t);case\"title\":return Ln(e,t);case\"type\":return r=!0,Zl(\"symbol\")}return tc(e,t,n,\"legend\")}));e.setWithExplicit(n,i)}return r&&(e.implicit?.encode?.gradient&&C(e.implicit,[\"encode\",\"gradient\"]),e.explicit?.encode?.gradient&&C(e.explicit,[\"encode\",\"gradient\"])),e}function Hf(e,t){return\"circle\"===t.value?t:e}function Vf(e){const t=e.component.legends,n={};for(const i of D(t)){const r=X(e.getScaleComponent(i).get(\"domains\"));if(n[r])for(const e of n[r]){If(e,t[i])||n[r].push(t[i])}else n[r]=[t[i].clone()]}return F(n).flat().map((t=>function(e,t){const{disable:n,labelExpr:i,selections:r,...o}=e.combine();if(n)return;!1===t.aria&&null==o.aria&&(o.aria=!1);if(o.encode?.symbols){const e=o.encode.symbols.update;!e.fill||\"transparent\"===e.fill.value||e.stroke||o.stroke||(e.stroke={value:\"transparent\"});for(const t of bs)o[t]&&delete e[t]}o.title||delete o.title;if(void 0!==i){let e=i;o.encode?.labels?.update&&yn(o.encode.labels.update.text)&&(e=M(i,\"datum.label\",o.encode.labels.update.text.signal)),function(e,t,n,i){e.encode??={},e.encode[t]??={},e.encode[t].update??={},e.encode[t].update[n]=i}(o,\"labels\",\"text\",{signal:e})}return o}(t,e.config))).filter((e=>void 0!==e))}function Gf(e){return km(e)||wm(e)?function(e){return e.children.reduce(((e,t)=>e.concat(t.assembleProjections())),Yf(e))}(e):Yf(e)}function Yf(e){const t=e.component.projection;if(!t||t.merged)return[];const n=t.combine(),{name:i}=n;if(t.data){const r={signal:`[${t.size.map((e=>e.signal)).join(\", \")}]`},o=t.data.reduce(((t,n)=>{const i=yn(n)?n.signal:`data('${e.lookupDataSource(n)}')`;return p(t,i)||t.push(i),t}),[]);if(o.length<=0)throw new Error(\"Projection's fit didn't find any data sources\");return[{name:i,size:r,fit:{signal:o.length>1?`[${o.join(\", \")}]`:o[0]},...n}]}return[{name:i,translate:{signal:\"[width / 2, height / 2]\"},...n}]}const Xf=[\"type\",\"clipAngle\",\"clipExtent\",\"center\",\"rotate\",\"precision\",\"reflectX\",\"reflectY\",\"coefficient\",\"distance\",\"fraction\",\"lobes\",\"parallel\",\"radius\",\"ratio\",\"spacing\",\"tilt\"];class Qf extends Jl{constructor(e,t,n,i){super({...t},{name:e}),this.specifiedProjection=t,this.size=n,this.data=i,qn(this,\"merged\",!1)}get isFit(){return!!this.data}}function Jf(e){e.component.projection=xm(e)?function(e){if(e.hasProjection){const t=pn(e.specifiedProjection),n=!(t&&(null!=t.scale||null!=t.translate)),i=n?[e.getSizeSignalRef(\"width\"),e.getSizeSignalRef(\"height\")]:void 0,r=n?function(e){const t=[],{encoding:n}=e;for(const i of[[ue,ce],[de,fe]])(ga(n[i[0]])||ga(n[i[1]]))&&t.push({signal:e.getName(`geojson_${t.length}`)});e.channelHasField(he)&&e.typedFieldDef(he).type===lr&&t.push({signal:e.getName(`geojson_${t.length}`)});0===t.length&&t.push(e.requestDataName(fc.Main));return t}(e):void 0,o=new Qf(e.projectionName(!0),{...pn(e.config.projection)??{},...t??{}},i,r);return o.get(\"type\")||o.set(\"type\",\"equalEarth\",!1),o}return}(e):function(e){if(0===e.children.length)return;let n;for(const t of e.children)Jf(t);const i=h(e.children,(e=>{const i=e.component.projection;if(i){if(n){const e=function(e,n){const i=h(Xf,(i=>!t.hasOwnProperty(e.explicit,i)&&!t.hasOwnProperty(n.explicit,i)||!!(t.hasOwnProperty(e.explicit,i)&&t.hasOwnProperty(n.explicit,i)&&Y(e.get(i),n.get(i)))));if(Y(e.size,n.size)){if(i)return e;if(Y(e.explicit,{}))return n;if(Y(n.explicit,{}))return e}return null}(n,i);return e&&(n=e),!!e}return n=i,!0}return!0}));if(n&&i){const t=e.projectionName(!0),i=new Qf(t,n.specifiedProjection,n.size,l(n.data));for(const n of e.children){const e=n.component.projection;e&&(e.isFit&&i.data.push(...n.component.projection.data),n.renameProjection(e.get(\"name\"),t),e.merged=!0)}return i}return}(e)}function Kf(e,t,n,i){if(Sa(t,n)){const r=xm(e)?e.axis(n)??e.legend(n)??{}:{},o=oa(t,{expr:\"datum\"}),a=oa(t,{expr:\"datum\",binSuffix:\"end\"});return{formulaAs:oa(t,{binSuffix:\"range\",forAs:!0}),formula:Fo(o,a,r.format,r.formatType,i)}}return{}}function Zf(e,t){return`${sn(e)}_${t}`}function ed(e,t,n){const i=Zf(ba(n,void 0)??{},t);return e.getName(`${i}_bins`)}function td(e,n,i){let r,o;r=function(e){return\"as\"in e}(e)?t.isString(e.as)?[e.as,`${e.as}_end`]:[e.as[0],e.as[1]]:[oa(e,{forAs:!0}),oa(e,{binSuffix:\"end\",forAs:!0})];const a={...ba(n,void 0)},s=Zf(a,e.field),{signal:l,extentSignal:c}=function(e,t){return{signal:e.getName(`${t}_bins`),extentSignal:e.getName(`${t}_extent`)}}(i,s);if(fn(a.extent)){const e=a.extent;o=Hu(i,e.param,e),delete a.extent}return{key:s,binComponent:{bin:a,field:e.field,as:[r],...l?{signal:l}:{},...c?{extentSignal:c}:{},...o?{span:o}:{}}}}class nd extends vc{clone(){return new nd(null,l(this.bins))}constructor(e,t){super(e),this.bins=t}static makeFromEncoding(e,t){const n=t.reduceFieldDef(((e,n,i)=>{if(Ko(n)&&ln(n.bin)){const{key:r,binComponent:o}=td(n,n.bin,t);e[r]={...o,...e[r],...Kf(t,n,i,t.config)}}return e}),{});return S(n)?null:new nd(e,n)}static makeFromTransform(e,t,n){const{key:i,binComponent:r}=td(t,t.bin,n);return new nd(e,{[i]:r})}merge(e,t){for(const n of D(e.bins))n in this.bins?(t(e.bins[n].signal,this.bins[n].signal),this.bins[n].as=b([...this.bins[n].as,...e.bins[n].as],d)):this.bins[n]=e.bins[n];for(const t of e.children)e.removeChild(t),t.parent=this;e.remove()}producedFields(){return new Set(F(this.bins).map((e=>e.as)).flat(2))}dependentFields(){return new Set(F(this.bins).map((e=>e.field)))}hash(){return`Bin ${d(this.bins)}`}assemble(){return F(this.bins).flatMap((e=>{const t=[],[n,...i]=e.as,{extent:r,...o}=e.bin,a={type:\"bin\",field:E(e.field),as:n,signal:e.signal,...fn(r)?{extent:null}:{extent:r},...e.span?{span:{signal:`span(${e.span})`}}:{},...o};!r&&e.extentSignal&&(t.push({type:\"extent\",field:E(e.field),signal:e.extentSignal}),a.extent={signal:e.extentSignal}),t.push(a);for(const e of i)for(let i=0;i<2;i++)t.push({type:\"formula\",expr:oa({field:n[i]},{expr:\"datum\"}),as:e[i]});return e.formula&&t.push({type:\"formula\",expr:e.formula,as:e.formulaAs}),t}))}}function id(e,n,i,r){const o=xm(r)?r.encoding[it(n)]:void 0;if(Ko(i)&&xm(r)&&Uo(i,o,r.markDef,r.config)){e.add(oa(i,{})),e.add(oa(i,{suffix:\"end\"}));const{mark:t,markDef:o,config:a}=r,s=Lo({fieldDef:i,markDef:o,config:a});Jr(t)&&.5!==s&&zt(n)&&(e.add(oa(i,{suffix:kc})),e.add(oa(i,{suffix:Sc}))),i.bin&&Sa(i,n)&&e.add(oa(i,{binSuffix:\"range\"}))}else if(Ee(n)){const t=Te(n);e.add(r.getName(t))}else e.add(oa(i));return ea(i)&&function(e){return t.isObject(e)&&\"field\"in e}(i.scale?.range)&&e.add(i.scale.range.field),e}class rd extends vc{clone(){return new rd(null,new Set(this.dimensions),l(this.measures))}constructor(e,t,n){super(e),this.dimensions=t,this.measures=n}get groupBy(){return this.dimensions}static makeFromEncoding(e,t){let n=!1;t.forEachFieldDef((e=>{e.aggregate&&(n=!0)}));const i={},r=new Set;return n?(t.forEachFieldDef(((e,n)=>{const{aggregate:o,field:a}=e;if(o)if(\"count\"===o)i[\"*\"]??={},i[\"*\"].count=new Set([oa(e,{forAs:!0})]);else{if(Zt(o)||en(o)){const e=Zt(o)?\"argmin\":\"argmax\",t=o[e];i[t]??={},i[t][e]=new Set([oa({op:e,field:t},{forAs:!0})])}else i[a]??={},i[a][o]=new Set([oa(e,{forAs:!0})]);Ht(n)&&\"unaggregated\"===t.scaleDomain(n)&&(i[a]??={},i[a].min=new Set([oa({field:a,aggregate:\"min\"},{forAs:!0})]),i[a].max=new Set([oa({field:a,aggregate:\"max\"},{forAs:!0})]))}else id(r,n,e,t)})),r.size+D(i).length===0?null:new rd(e,r,i)):null}static makeFromTransform(e,t){const n=new Set,i={};for(const e of t.aggregate){const{op:t,field:n,as:r}=e;t&&(\"count\"===t?(i[\"*\"]??={},i[\"*\"].count=new Set([r||oa(e,{forAs:!0})])):(i[n]??={},i[n][t]=new Set([r||oa(e,{forAs:!0})])))}for(const e of t.groupby??[])n.add(e);return n.size+D(i).length===0?null:new rd(e,n,i)}merge(e){return x(this.dimensions,e.dimensions)?(function(e,t){for(const n of D(t)){const i=t[n];for(const t of D(i))n in e?e[n][t]=new Set([...e[n][t]??[],...i[t]]):e[n]={[t]:i[t]}}}(this.measures,e.measures),!0):(function(){xi.debug(...arguments)}(\"different dimensions, cannot merge\"),!1)}addDimensions(e){e.forEach(this.dimensions.add,this.dimensions)}dependentFields(){return new Set([...this.dimensions,...D(this.measures)])}producedFields(){const e=new Set;for(const t of D(this.measures))for(const n of D(this.measures[t])){const i=this.measures[t][n];0===i.size?e.add(`${n}_${t}`):i.forEach(e.add,e)}return e}hash(){return`Aggregate ${d({dimensions:this.dimensions,measures:this.measures})}`}assemble(){const e=[],t=[],n=[];for(const i of D(this.measures))for(const r of D(this.measures[i]))for(const o of this.measures[i][r])n.push(o),e.push(r),t.push(\"*\"===i?null:E(i));return{type:\"aggregate\",groupby:[...this.dimensions].map(E),ops:e,fields:t,as:n}}}class od extends vc{constructor(e,n,i,r){super(e),this.model=n,this.name=i,this.data=r,qn(this,\"column\",void 0),qn(this,\"row\",void 0),qn(this,\"facet\",void 0),qn(this,\"childModel\",void 0);for(const e of Re){const i=n.facet[e];if(i){const{bin:r,sort:o}=i;this[e]={name:n.getName(`${e}_domain`),fields:[oa(i),...ln(r)?[oa(i,{binSuffix:\"end\"})]:[]],...Co(o)?{sortField:o}:t.isArray(o)?{sortIndexField:af(i,e)}:{}}}}this.childModel=n.child}hash(){let e=\"Facet\";for(const t of Re)this[t]&&(e+=` ${t.charAt(0)}:${d(this[t])}`);return e}get fields(){const e=[];for(const t of Re)this[t]?.fields&&e.push(...this[t].fields);return e}dependentFields(){const e=new Set(this.fields);for(const t of Re)this[t]&&(this[t].sortField&&e.add(this[t].sortField.field),this[t].sortIndexField&&e.add(this[t].sortIndexField));return e}producedFields(){return new Set}getSource(){return this.name}getChildIndependentFieldsWithStep(){const e={};for(const t of Ft){const n=this.childModel.component.scales[t];if(n&&!n.merged){const i=n.get(\"type\"),r=n.get(\"range\");if(xr(i)&&vn(r)){const n=Qd(Jd(this.childModel,t));n?e[t]=n:$i(Yn(t))}}}return e}assembleRowColumnHeaderData(e,t,n){const i={row:\"y\",column:\"x\",facet:void 0}[e],r=[],o=[],a=[];i&&n&&n[i]&&(t?(r.push(`distinct_${n[i]}`),o.push(\"max\")):(r.push(n[i]),o.push(\"distinct\")),a.push(`distinct_${n[i]}`));const{sortField:s,sortIndexField:l}=this[e];if(s){const{op:e=zo,field:t}=s;r.push(t),o.push(e),a.push(oa(s,{forAs:!0}))}else l&&(r.push(l),o.push(\"max\"),a.push(l));return{name:this[e].name,source:t??this.data,transform:[{type:\"aggregate\",groupby:this[e].fields,...r.length?{fields:r,ops:o,as:a}:{}}]}}assembleFacetHeaderData(e){const{columns:t}=this.model.layout,{layoutHeaders:n}=this.model.component,i=[],r={};for(const e of uf){for(const t of ff){const i=(n[e]&&n[e][t])??[];for(const t of i)if(t.axes?.length>0){r[e]=!0;break}}if(r[e]){const n=`length(data(\"${this.facet.name}\"))`,r=\"row\"===e?t?{signal:`ceil(${n} / ${t})`}:1:t?{signal:`min(${n}, ${t})`}:{signal:n};i.push({name:`${this.facet.name}_${e}`,transform:[{type:\"sequence\",start:0,stop:r}]})}}const{row:o,column:a}=r;return(o||a)&&i.unshift(this.assembleRowColumnHeaderData(\"facet\",null,e)),i}assemble(){const e=[];let t=null;const n=this.getChildIndependentFieldsWithStep(),{column:i,row:r,facet:o}=this;if(i&&r&&(n.x||n.y)){t=`cross_${this.column.name}_${this.row.name}`;const i=[].concat(n.x??[],n.y??[]),r=i.map((()=>\"distinct\"));e.push({name:t,source:this.data,transform:[{type:\"aggregate\",groupby:this.fields,fields:i,ops:r}]})}for(const i of[J,Q])this[i]&&e.push(this.assembleRowColumnHeaderData(i,t,n));if(o){const t=this.assembleFacetHeaderData(n);t&&e.push(...t)}return e}}function ad(e){return e.startsWith(\"'\")&&e.endsWith(\"'\")||e.startsWith('\"')&&e.endsWith('\"')?e.slice(1,-1):e}function sd(e){const n={};return a(e.filter,(e=>{if(Ji(e)){let i=null;Ii(e)?i=Sn(e.equal):Vi(e)?i=Sn(e.lte):Hi(e)?i=Sn(e.lt):Gi(e)?i=Sn(e.gt):Yi(e)?i=Sn(e.gte):Xi(e)?i=e.range[0]:Qi(e)&&(i=(e.oneOf??e.in)[0]),i&&(wi(i)?n[e.field]=\"date\":t.isNumber(i)?n[e.field]=\"number\":t.isString(i)&&(n[e.field]=\"string\")),e.timeUnit&&(n[e.field]=\"date\")}})),n}function ld(e){const n={};function i(e){var i;$a(e)?n[e.field]=\"date\":\"quantitative\"===e.type&&(i=e.aggregate,t.isString(i)&&p([\"min\",\"max\"],i))?n[e.field]=\"number\":q(e.field)>1?e.field in n||(n[e.field]=\"flatten\"):ea(e)&&Co(e.sort)&&q(e.sort.field)>1&&(e.sort.field in n||(n[e.sort.field]=\"flatten\"))}if((xm(e)||$m(e))&&e.forEachFieldDef(((t,n)=>{if(Ko(t))i(t);else{const r=tt(n),o=e.fieldDef(r);i({...t,type:o.type})}})),xm(e)){const{mark:t,markDef:i,encoding:r}=e;if(Qr(t)&&!e.encoding.order){const e=r[\"horizontal\"===i.orient?\"y\":\"x\"];Ho(e)&&\"quantitative\"===e.type&&!(e.field in n)&&(n[e.field]=\"number\")}}return n}class cd extends vc{clone(){return new cd(null,l(this._parse))}constructor(e,t){super(e),qn(this,\"_parse\",void 0),this._parse=t}hash(){return`Parse ${d(this._parse)}`}static makeExplicit(e,t,n){let i={};const r=t.data;return!sc(r)&&r?.format?.parse&&(i=r.format.parse),this.makeWithAncestors(e,i,{},n)}static makeWithAncestors(e,t,n,i){for(const e of D(n)){const t=i.getWithExplicit(e);void 0!==t.value&&(t.explicit||t.value===n[e]||\"derived\"===t.value||\"flatten\"===n[e]?delete n[e]:$i(ei(e,n[e],t.value)))}for(const e of D(t)){const n=i.get(e);void 0!==n&&(n===t[e]?delete t[e]:$i(ei(e,t[e],n)))}const r=new Jl(t,n);i.copyAll(r);const o={};for(const e of D(r.combine())){const t=r.get(e);null!==t&&(o[e]=t)}return 0===D(o).length||i.parseNothing?null:new cd(e,o)}get parse(){return this._parse}merge(e){this._parse={...this._parse,...e.parse},e.remove()}assembleFormatParse(){const e={};for(const t of D(this._parse)){const n=this._parse[t];1===q(t)&&(e[t]=n)}return e}producedFields(){return new Set(D(this._parse))}dependentFields(){return new Set(D(this._parse))}assembleTransforms(){let e=arguments.length>0&&void 0!==arguments[0]&&arguments[0];return D(this._parse).filter((t=>!e||q(t)>1)).map((e=>{const t=function(e,t){const n=A(e);if(\"number\"===t)return`toNumber(${n})`;if(\"boolean\"===t)return`toBoolean(${n})`;if(\"string\"===t)return`toString(${n})`;if(\"date\"===t)return`toDate(${n})`;if(\"flatten\"===t)return n;if(t.startsWith(\"date:\"))return`timeParse(${n},'${ad(t.slice(5,t.length))}')`;if(t.startsWith(\"utc:\"))return`utcParse(${n},'${ad(t.slice(4,t.length))}')`;return $i(`Unrecognized parse \"${t}\".`),null}(e,this._parse[e]);if(!t)return null;return{type:\"formula\",expr:t,as:L(e)}})).filter((e=>null!==e))}}class ud extends vc{clone(){return new ud(null)}constructor(e){super(e)}dependentFields(){return new Set}producedFields(){return new Set([xs])}hash(){return\"Identifier\"}assemble(){return{type:\"identifier\",as:xs}}}class fd extends vc{clone(){return new fd(null,this.params)}constructor(e,t){super(e),this.params=t}dependentFields(){return new Set}producedFields(){}hash(){return`Graticule ${d(this.params)}`}assemble(){return{type:\"graticule\",...!0===this.params?{}:this.params}}}class dd extends vc{clone(){return new dd(null,this.params)}constructor(e,t){super(e),this.params=t}dependentFields(){return new Set}producedFields(){return new Set([this.params.as??\"data\"])}hash(){return`Hash ${d(this.params)}`}assemble(){return{type:\"sequence\",...this.params}}}class md extends vc{constructor(e){let t;if(super(null),qn(this,\"_data\",void 0),qn(this,\"_name\",void 0),qn(this,\"_generator\",void 0),e??={name:\"source\"},sc(e)||(t=e.format?{...f(e.format,[\"parse\"])}:{}),oc(e))this._data={values:e.values};else if(rc(e)){if(this._data={url:e.url},!t.type){let n=/(?:\\.([^.]+))?$/.exec(e.url)[1];p([\"json\",\"csv\",\"tsv\",\"dsv\",\"topojson\"],n)||(n=\"json\"),t.type=n}}else cc(e)?this._data={values:[{type:\"Sphere\"}]}:(ac(e)||sc(e))&&(this._data={});this._generator=sc(e),e.name&&(this._name=e.name),t&&!S(t)&&(this._data.format=t)}dependentFields(){return new Set}producedFields(){}get data(){return this._data}hasName(){return!!this._name}get isGenerator(){return this._generator}get dataName(){return this._name}set dataName(e){this._name=e}set parent(e){throw new Error(\"Source nodes have to be roots.\")}remove(){throw new Error(\"Source nodes are roots and cannot be removed.\")}hash(){throw new Error(\"Cannot hash sources\")}assemble(){return{name:this._name,...this._data,transform:[]}}}function pd(e){return e instanceof md||e instanceof fd||e instanceof dd}var gd=new WeakMap;class hd{constructor(){Wn(this,gd,{writable:!0,value:void 0}),Un(this,gd,!1)}setModified(){Un(this,gd,!0)}get modifiedFlag(){return function(e,t){return t.get?t.get.call(e):t.value}(e=this,Rn(e,gd,\"get\"));var e}}class yd extends hd{getNodeDepths(e,t,n){n.set(e,t);for(const i of e.children)this.getNodeDepths(i,t+1,n);return n}optimize(e){const t=[...this.getNodeDepths(e,0,new Map).entries()].sort(((e,t)=>t[1]-e[1]));for(const e of t)this.run(e[0]);return this.modifiedFlag}}class vd extends hd{optimize(e){this.run(e);for(const t of e.children)this.optimize(t);return this.modifiedFlag}}class bd extends vd{mergeNodes(e,t){const n=t.shift();for(const i of t)e.removeChild(i),i.parent=n,i.remove()}run(e){const t=e.children.map((e=>e.hash())),n={};for(let i=0;i1&&(this.setModified(),this.mergeNodes(e,n[t]))}}class xd extends vd{constructor(e){super(),qn(this,\"requiresSelectionId\",void 0),this.requiresSelectionId=e&&Lu(e)}run(e){e instanceof ud&&(this.requiresSelectionId&&(pd(e.parent)||e.parent instanceof rd||e.parent instanceof cd)||(this.setModified(),e.remove()))}}class $d extends hd{optimize(e){return this.run(e,new Set),this.modifiedFlag}run(e,t){let n=new Set;e instanceof wc&&(n=e.producedFields(),$(n,t)&&(this.setModified(),e.removeFormulas(t),0===e.producedFields.length&&e.remove()));for(const i of e.children)this.run(i,new Set([...t,...n]))}}class wd extends vd{constructor(){super()}run(e){e instanceof bc&&!e.isRequired()&&(this.setModified(),e.remove())}}class kd extends yd{run(e){if(!(pd(e)||e.numChildren()>1))for(const t of e.children)if(t instanceof cd)if(e instanceof cd)this.setModified(),e.merge(t);else{if(k(e.producedFields(),t.dependentFields()))continue;this.setModified(),t.swapWithParent()}}}class Sd extends yd{run(e){const t=[...e.children],n=e.children.filter((e=>e instanceof cd));if(e.numChildren()>1&&n.length>=1){const i={},r=new Set;for(const e of n){const t=e.parse;for(const e of D(t))e in i?i[e]!==t[e]&&r.add(e):i[e]=t[e]}for(const e of r)delete i[e];if(!S(i)){this.setModified();const n=new cd(e,i);for(const r of t){if(r instanceof cd)for(const e of D(i))delete r.parse[e];e.removeChild(r),r.parent=n,r instanceof cd&&0===D(r.parse).length&&r.remove()}}}}}class Dd extends yd{run(e){e instanceof bc||e.numChildren()>0||e instanceof od||e instanceof md||(this.setModified(),e.remove())}}class Fd extends yd{run(e){const t=e.children.filter((e=>e instanceof wc)),n=t.pop();for(const e of t)this.setModified(),n.merge(e)}}class zd extends yd{run(e){const t=e.children.filter((e=>e instanceof rd)),n={};for(const e of t){const t=d(e.groupBy);t in n||(n[t]=[]),n[t].push(e)}for(const t of D(n)){const i=n[t];if(i.length>1){const t=i.pop();for(const n of i)t.merge(n)&&(e.removeChild(n),n.parent=t,n.remove(),this.setModified())}}}}class Od extends yd{constructor(e){super(),this.model=e}run(e){const t=!(pd(e)||e instanceof Bu||e instanceof cd||e instanceof ud),n=[],i=[];for(const r of e.children)r instanceof nd&&(t&&!k(e.producedFields(),r.dependentFields())?n.push(r):i.push(r));if(n.length>0){const t=n.pop();for(const e of n)t.merge(e,this.model.renameSignal.bind(this.model));this.setModified(),e instanceof nd?e.merge(t,this.model.renameSignal.bind(this.model)):t.swapWithParent()}if(i.length>1){const e=i.pop();for(const t of i)e.merge(t,this.model.renameSignal.bind(this.model));this.setModified()}}}class _d extends yd{run(e){const t=[...e.children];if(!g(t,(e=>e instanceof bc))||e.numChildren()<=1)return;const n=[];let i;for(const r of t)if(r instanceof bc){let t=r;for(;1===t.numChildren();){const[e]=t.children;if(!(e instanceof bc))break;t=e}n.push(...t.children),i?(e.removeChild(r),r.parent=i.parent,i.parent.removeChild(i),i.parent=t,this.setModified()):i=t}else n.push(r);if(n.length){this.setModified();for(const e of n)e.parent.removeChild(e),e.parent=i}}}class Nd extends vc{clone(){return new Nd(null,l(this.transform))}constructor(e,t){super(e),this.transform=t}addDimensions(e){this.transform.groupby=b(this.transform.groupby.concat(e),(e=>e))}dependentFields(){const e=new Set;return this.transform.groupby&&this.transform.groupby.forEach(e.add,e),this.transform.joinaggregate.map((e=>e.field)).filter((e=>void 0!==e)).forEach(e.add,e),e}producedFields(){return new Set(this.transform.joinaggregate.map(this.getDefaultName))}getDefaultName(e){return e.as??oa(e)}hash(){return`JoinAggregateTransform ${d(this.transform)}`}assemble(){const e=[],t=[],n=[];for(const i of this.transform.joinaggregate)t.push(i.op),n.push(this.getDefaultName(i)),e.push(void 0===i.field?null:i.field);const i=this.transform.groupby;return{type:\"joinaggregate\",as:n,ops:t,fields:e,...void 0!==i?{groupby:i}:{}}}}class Cd extends vc{clone(){return new Cd(null,l(this._stack))}constructor(e,t){super(e),qn(this,\"_stack\",void 0),this._stack=t}static makeFromTransform(e,n){const{stack:i,groupby:r,as:o,offset:a=\"zero\"}=n,s=[],l=[];if(void 0!==n.sort)for(const e of n.sort)s.push(e.field),l.push(U(e.order,\"ascending\"));const c={field:s,order:l};let u;return u=function(e){return t.isArray(e)&&e.every((e=>t.isString(e)))&&e.length>1}(o)?o:t.isString(o)?[o,`${o}_end`]:[`${n.stack}_start`,`${n.stack}_end`],new Cd(e,{dimensionFieldDefs:[],stackField:i,groupby:r,offset:a,sort:c,facetby:[],as:u})}static makeFromEncoding(e,n){const i=n.stack,{encoding:r}=n;if(!i)return null;const{groupbyChannels:o,fieldChannel:a,offset:s,impute:l}=i,c=o.map((e=>pa(r[e]))).filter((e=>!!e)),u=function(e){return e.stack.stackBy.reduce(((e,t)=>{const n=oa(t.fieldDef);return n&&e.push(n),e}),[])}(n),f=n.encoding.order;let d;if(t.isArray(f)||Ho(f))d=Tn(f);else{const e=Ro(f)?f.sort:\"y\"===a?\"descending\":\"ascending\";d=u.reduce(((t,n)=>(t.field.push(n),t.order.push(e),t)),{field:[],order:[]})}return new Cd(e,{dimensionFieldDefs:c,stackField:n.vgField(a),facetby:[],stackby:u,sort:d,offset:s,impute:l,as:[n.vgField(a,{suffix:\"start\",forAs:!0}),n.vgField(a,{suffix:\"end\",forAs:!0})]})}get stack(){return this._stack}addDimensions(e){this._stack.facetby.push(...e)}dependentFields(){const e=new Set;return e.add(this._stack.stackField),this.getGroupbyFields().forEach(e.add,e),this._stack.facetby.forEach(e.add,e),this._stack.sort.field.forEach(e.add,e),e}producedFields(){return new Set(this._stack.as)}hash(){return`Stack ${d(this._stack)}`}getGroupbyFields(){const{dimensionFieldDefs:e,impute:t,groupby:n}=this._stack;return e.length>0?e.map((e=>e.bin?t?[oa(e,{binSuffix:\"mid\"})]:[oa(e,{}),oa(e,{binSuffix:\"end\"})]:[oa(e)])).flat():n??[]}assemble(){const e=[],{facetby:t,dimensionFieldDefs:n,stackField:i,stackby:r,sort:o,offset:a,impute:s,as:l}=this._stack;if(s)for(const o of n){const{bandPosition:n=.5,bin:a}=o;if(a){const t=oa(o,{expr:\"datum\"}),i=oa(o,{expr:\"datum\",binSuffix:\"end\"});e.push({type:\"formula\",expr:`${n}*${t}+${1-n}*${i}`,as:oa(o,{binSuffix:\"mid\",forAs:!0})})}e.push({type:\"impute\",field:i,groupby:[...r,...t],key:oa(o,{binSuffix:\"mid\"}),method:\"value\",value:0})}return e.push({type:\"stack\",groupby:[...this.getGroupbyFields(),...t],field:i,sort:o,as:l,offset:a}),e}}class Pd extends vc{clone(){return new Pd(null,l(this.transform))}constructor(e,t){super(e),this.transform=t}addDimensions(e){this.transform.groupby=b(this.transform.groupby.concat(e),(e=>e))}dependentFields(){const e=new Set;return(this.transform.groupby??[]).forEach(e.add,e),(this.transform.sort??[]).forEach((t=>e.add(t.field))),this.transform.window.map((e=>e.field)).filter((e=>void 0!==e)).forEach(e.add,e),e}producedFields(){return new Set(this.transform.window.map(this.getDefaultName))}getDefaultName(e){return e.as??oa(e)}hash(){return`WindowTransform ${d(this.transform)}`}assemble(){const e=[],t=[],n=[],i=[];for(const r of this.transform.window)t.push(r.op),n.push(this.getDefaultName(r)),i.push(void 0===r.param?null:r.param),e.push(void 0===r.field?null:r.field);const r=this.transform.frame,o=this.transform.groupby;if(r&&null===r[0]&&null===r[1]&&t.every((e=>tn(e))))return{type:\"joinaggregate\",as:n,ops:t,fields:e,...void 0!==o?{groupby:o}:{}};const a=[],s=[];if(void 0!==this.transform.sort)for(const e of this.transform.sort)a.push(e.field),s.push(e.order??\"ascending\");const l={field:a,order:s},c=this.transform.ignorePeers;return{type:\"window\",params:i,as:n,ops:t,fields:e,sort:l,...void 0!==c?{ignorePeers:c}:{},...void 0!==o?{groupby:o}:{},...void 0!==r?{frame:r}:{}}}}function Ad(e){if(e instanceof od)if(1!==e.numChildren()||e.children[0]instanceof bc){const n=e.model.component.data.main;jd(n);const i=(t=e,function e(n){if(!(n instanceof od)){const i=n.clone();if(i instanceof bc){const e=Td+i.getSource();i.setSource(e),t.model.component.data.outputNodes[e]=i}else(i instanceof rd||i instanceof Cd||i instanceof Pd||i instanceof Nd)&&i.addDimensions(t.fields);for(const t of n.children.flatMap(e))t.parent=i;return[i]}return n.children.flatMap(e)}),r=e.children.map(i).flat();for(const e of r)e.parent=n}else{const t=e.children[0];(t instanceof rd||t instanceof Cd||t instanceof Pd||t instanceof Nd)&&t.addDimensions(e.fields),t.swapWithParent(),Ad(e)}else e.children.map(Ad);var t}function jd(e){if(e instanceof bc&&e.type===fc.Main&&1===e.numChildren()){const t=e.children[0];t instanceof od||(t.swapWithParent(),jd(e))}}const Td=\"scale_\",Ed=5;function Md(e){for(const t of e){for(const e of t.children)if(e.parent!==t)return!1;if(!Md(t.children))return!1}return!0}function Ld(e,t){let n=!1;for(const i of t)n=e.optimize(i)||n;return n}function qd(e,t,n){let i=e.sources,r=!1;return r=Ld(new wd,i)||r,r=Ld(new xd(t),i)||r,i=i.filter((e=>e.numChildren()>0)),r=Ld(new Dd,i)||r,i=i.filter((e=>e.numChildren()>0)),n||(r=Ld(new kd,i)||r,r=Ld(new Od(t),i)||r,r=Ld(new $d,i)||r,r=Ld(new Sd,i)||r,r=Ld(new zd,i)||r,r=Ld(new Fd,i)||r,r=Ld(new bd,i)||r,r=Ld(new _d,i)||r),e.sources=i,r}class Ud{constructor(e){qn(this,\"signal\",void 0),Object.defineProperty(this,\"signal\",{enumerable:!0,get:e})}static fromName(e,t){return new Ud((()=>e(t)))}}function Rd(e){xm(e)?function(e){const t=e.component.scales;for(const n of D(t)){const i=Wd(e,n);if(t[n].setWithExplicit(\"domains\",i),Vd(e,n),e.component.data.isFaceted){let t=e;for(;!$m(t)&&t.parent;)t=t.parent;if(\"shared\"===t.component.resolve.scale[n])for(const e of i.value)bn(e)&&(e.data=Td+e.data.replace(Td,\"\"))}}}(e):function(e){for(const t of e.children)Rd(t);const t=e.component.scales;for(const n of D(t)){let i,r=null;for(const t of e.children){const e=t.component.scales[n];if(e){i=void 0===i?e.getWithExplicit(\"domains\"):nc(i,e.getWithExplicit(\"domains\"),\"domains\",\"scale\",Yd);const t=e.get(\"selectionExtent\");r&&t&&r.param!==t.param&&$i(Kn),r=t}}t[n].setWithExplicit(\"domains\",i),r&&t[n].set(\"selectionExtent\",r,!0)}}(e)}function Wd(e,t){const n=e.getScaleComponent(t).get(\"type\"),{encoding:i}=e,r=function(e,t,n,i){if(\"unaggregated\"===e){const{valid:e,reason:i}=Gd(t,n);if(!e)return void $i(i)}else if(void 0===e&&i.useUnaggregatedDomain){const{valid:e}=Gd(t,n);if(e)return\"unaggregated\"}return e}(e.scaleDomain(t),e.typedFieldDef(t),n,e.config.scale);return r!==e.scaleDomain(t)&&(e.specifiedScales[t]={...e.specifiedScales[t],domain:r}),\"x\"===t&&ga(i.x2)?ga(i.x)?nc(Id(n,r,e,\"x\"),Id(n,r,e,\"x2\"),\"domain\",\"scale\",Yd):Id(n,r,e,\"x2\"):\"y\"===t&&ga(i.y2)?ga(i.y)?nc(Id(n,r,e,\"y\"),Id(n,r,e,\"y2\"),\"domain\",\"scale\",Yd):Id(n,r,e,\"y2\"):Id(n,r,e,t)}function Bd(e,t,n){const i=Ui(n)?.unit;return\"temporal\"===t||i?function(e,t,n){return e.map((e=>({signal:`{data: ${wa(e,{timeUnit:n,type:t})}}`})))}(e,t,i):[e]}function Id(e,n,i,r){const{encoding:o,markDef:a,mark:s,config:l,stack:c}=i,u=ga(o[r]),{type:f}=u,d=u.timeUnit;if(function(e){return e?.unionWith}(n)){const t=Id(e,void 0,i,r);return Kl([...Bd(n.unionWith,f,d),...t.value])}if(yn(n))return Kl([n]);if(n&&\"unaggregated\"!==n&&!Sr(n))return Kl(Bd(n,f,d));if(c&&r===c.fieldChannel){if(\"normalize\"===c.offset)return Zl([[0,1]]);const e=i.requestDataName(fc.Main);return Zl([{data:e,field:i.vgField(r,{suffix:\"start\"})},{data:e,field:i.vgField(r,{suffix:\"end\"})}])}const m=Ht(r)&&Ho(u)?function(e,t,n){if(!xr(n))return;const i=e.fieldDef(t),r=i.sort;if(Po(r))return{op:\"min\",field:af(i,t),order:\"ascending\"};const{stack:o}=e,a=o?new Set([...o.groupbyFields,...o.stackBy.map((e=>e.fieldDef.field))]):void 0;if(Co(r)){return Hd(r,o&&!a.has(r.field))}if(No(r)){const{encoding:t,order:n}=r,i=e.fieldDef(t),{aggregate:s,field:l}=i,c=o&&!a.has(l);if(Zt(s)||en(s))return Hd({field:oa(i),order:n},c);if(tn(s)||!s)return Hd({op:s,field:l,order:n},c)}else{if(\"descending\"===r)return{op:\"min\",field:e.vgField(t),order:\"descending\"};if(p([\"ascending\",void 0],r))return!0}return}(i,r,e):void 0;if(Go(u)){return Zl(Bd([u.datum],f,d))}const g=u;if(\"unaggregated\"===n){const e=i.requestDataName(fc.Main),{field:t}=u;return Zl([{data:e,field:oa({field:t,aggregate:\"min\"})},{data:e,field:oa({field:t,aggregate:\"max\"})}])}if(ln(g.bin)){if(xr(e))return Zl(\"bin-ordinal\"===e?[]:[{data:O(m)?i.requestDataName(fc.Main):i.requestDataName(fc.Raw),field:i.vgField(r,Sa(g,r)?{binSuffix:\"range\"}:{}),sort:!0!==m&&t.isObject(m)?m:{field:i.vgField(r,{}),op:\"min\"}}]);{const{bin:e}=g;if(ln(e)){const t=ed(i,g.field,e);return Zl([new Ud((()=>{const e=i.getSignalName(t);return`[${e}.start, ${e}.stop]`}))])}return Zl([{data:i.requestDataName(fc.Main),field:i.vgField(r,{})}])}}if(g.timeUnit&&p([\"time\",\"utc\"],e)){const e=o[it(r)];if(Uo(g,e,a,l)){const t=i.requestDataName(fc.Main),n=Lo({fieldDef:g,fieldDef2:e,markDef:a,config:l}),o=Jr(s)&&.5!==n&&zt(r);return Zl([{data:t,field:i.vgField(r,o?{suffix:kc}:{})},{data:t,field:i.vgField(r,{suffix:o?Sc:\"end\"})}])}}return Zl(m?[{data:O(m)?i.requestDataName(fc.Main):i.requestDataName(fc.Raw),field:i.vgField(r),sort:m}]:[{data:i.requestDataName(fc.Main),field:i.vgField(r)}])}function Hd(e,t){const{op:n,field:i,order:r}=e;return{op:n??(t?\"sum\":zo),...i?{field:E(i)}:{},...r?{order:r}:{}}}function Vd(e,t){const n=e.component.scales[t],i=e.specifiedScales[t].domain,r=e.fieldDef(t)?.bin,o=Sr(i)&&i,a=un(r)&&fn(r.extent)&&r.extent;(o||a)&&n.set(\"selectionExtent\",o??a,!0)}function Gd(e,n){const{aggregate:i,type:r}=e;return i?t.isString(i)&&!an.has(i)?{valid:!1,reason:fi(i)}:\"quantitative\"===r&&\"log\"===n?{valid:!1,reason:di(e)}:{valid:!0}:{valid:!1,reason:ui(e)}}function Yd(e,t,n,i){return e.explicit&&t.explicit&&$i(function(e,t,n,i){return`Conflicting ${t.toString()} property \"${e.toString()}\" (${X(n)} and ${X(i)}). Using the union of the two domains.`}(n,i,e.value,t.value)),{explicit:e.explicit,value:[...e.value,...t.value]}}function Xd(e){const n=b(e.map((e=>{if(bn(e)){const{sort:t,...n}=e;return n}return e})),d),i=b(e.map((e=>{if(bn(e)){const t=e.sort;return void 0===t||O(t)||(\"op\"in t&&\"count\"===t.op&&delete t.field,\"ascending\"===t.order&&delete t.order),t}})).filter((e=>void 0!==e)),d);if(0===n.length)return;if(1===n.length){const n=e[0];if(bn(n)&&i.length>0){let e=i[0];if(i.length>1){$i(gi);const n=i.filter((e=>t.isObject(e)&&\"op\"in e&&\"min\"!==e.op));e=!i.every((e=>t.isObject(e)&&\"op\"in e))||1!==n.length||n[0]}else if(t.isObject(e)&&\"field\"in e){const t=e.field;n.field===t&&(e=!e.order||{order:e.order})}return{...n,sort:e}}return n}const r=b(i.map((e=>O(e)||!(\"op\"in e)||t.isString(e.op)&&e.op in Kt?e:($i(function(e){return`Dropping sort property ${X(e)} as unioned domains only support boolean or op \"count\", \"min\", and \"max\".`}(e)),!0))),d);let o;1===r.length?o=r[0]:r.length>1&&($i(gi),o=!0);const a=b(e.map((e=>bn(e)?e.data:null)),(e=>e));if(1===a.length&&null!==a[0]){return{data:a[0],fields:n.map((e=>e.field)),...o?{sort:o}:{}}}return{fields:n,...o?{sort:o}:{}}}function Qd(e){if(bn(e)&&t.isString(e.field))return e.field;if(function(e){return!t.isArray(e)&&\"fields\"in e&&!(\"data\"in e)}(e)){let n;for(const i of e.fields)if(bn(i)&&t.isString(i.field))if(n){if(n!==i.field)return $i(\"Detected faceted independent scales that union domain of multiple fields from different data sources. We will use the first field. The result view size may be incorrect.\"),n}else n=i.field;return $i(\"Detected faceted independent scales that union domain of the same fields from different source. We will assume that this is the same field from a different fork of the same data source. However, if this is not the case, the result view size may be incorrect.\"),n}if(function(e){return!t.isArray(e)&&\"fields\"in e&&\"data\"in e}(e)){$i(\"Detected faceted independent scales that union domain of multiple fields from the same data source. We will use the first field. The result view size may be incorrect.\");const n=e.fields[0];return t.isString(n)?n:void 0}}function Jd(e,t){const n=e.component.scales[t].get(\"domains\").map((t=>(bn(t)&&(t.data=e.lookupDataSource(t.data)),t)));return Xd(n)}function Kd(e){return km(e)||wm(e)?e.children.reduce(((e,t)=>e.concat(Kd(t))),Zd(e)):Zd(e)}function Zd(e){return D(e.component.scales).reduce(((n,i)=>{const r=e.component.scales[i];if(r.merged)return n;const o=r.combine(),{name:a,type:s,selectionExtent:l,domains:c,range:u,reverse:f,...d}=o,m=function(e,n,i,r){if(zt(i)){if(vn(e))return{step:{signal:`${n}_step`}}}else if(t.isObject(e)&&bn(e))return{...e,data:r.lookupDataSource(e.data)};return e}(o.range,a,i,e),p=Jd(e,i),g=l?function(e,n,i,r){const o=Hu(e,n.param,n);return{signal:$r(i.get(\"type\"))&&t.isArray(r)&&r[0]>r[1]?`isValid(${o}) && reverse(${o})`:o}}(e,l,r,p):null;return n.push({name:a,type:s,...p?{domain:p}:{},...g?{domainRaw:g}:{},range:m,...void 0!==f?{reverse:f}:{},...d}),n}),[])}class em extends Jl{constructor(e,t){super({},{name:e}),qn(this,\"merged\",!1),this.setWithExplicit(\"type\",t)}domainDefinitelyIncludesZero(){return!1!==this.get(\"zero\")||g(this.get(\"domains\"),(e=>t.isArray(e)&&2===e.length&&t.isNumber(e[0])&&e[0]<=0&&t.isNumber(e[1])&&e[1]>=0))}}const tm=[\"range\",\"scheme\"];function nm(e,n){const i=e.fieldDef(n);if(i?.bin){const{bin:r,field:o}=i,a=rt(n),s=e.getName(a);if(t.isObject(r)&&r.binned&&void 0!==r.step)return new Ud((()=>{const t=e.scaleName(n),i=`(domain(\"${t}\")[1] - domain(\"${t}\")[0]) / ${r.step}`;return`${e.getSignalName(s)} / (${i})`}));if(ln(r)){const t=ed(e,o,r);return new Ud((()=>{const n=e.getSignalName(t),i=`(${n}.stop - ${n}.start) / ${n}.step`;return`${e.getSignalName(s)} / (${i})`}))}}}function im(e,n){const i=n.specifiedScales[e],{size:r}=n,o=n.getScaleComponent(e).get(\"type\");for(const r of tm)if(void 0!==i[r]){const a=Ar(o,r),s=jr(e,r);if(a)if(s)$i(s);else switch(r){case\"range\":{const r=i.range;if(t.isArray(r)){if(zt(e))return Kl(r.map((e=>{if(\"width\"===e||\"height\"===e){const t=n.getName(e),i=n.getSignalName.bind(n);return Ud.fromName(i,t)}return e})))}else if(t.isObject(r))return Kl({data:n.requestDataName(fc.Main),field:r.field,sort:{op:\"min\",field:n.vgField(e)}});return Kl(r)}case\"scheme\":return Kl(rm(i[r]))}else $i(mi(o,r,e))}const a=e===Z||\"xOffset\"===e?\"width\":\"height\",s=r[a];if(Ns(s))if(zt(e))if(xr(o)){const t=am(s,n,e);if(t)return Kl({step:t})}else $i(pi(a));else if(Pt(e)){const t=e===ie?\"x\":\"y\";if(\"band\"===n.getScaleComponent(t).get(\"type\")){const e=sm(s,o);if(e)return Kl(e)}}const{rangeMin:l,rangeMax:u}=i,f=function(e,n){const{size:i,config:r,mark:o,encoding:a}=n,{type:s}=ga(a[e]),l=n.getScaleComponent(e),u=l.get(\"type\"),{domain:f,domainMid:d}=n.specifiedScales[e];switch(e){case Z:case ee:if(p([\"point\",\"band\"],u)){const t=lm(e,i,r.view);if(Ns(t)){return{step:am(t,n,e)}}}return om(e,n,u);case ie:case re:return function(e,t,n){const i=e===ie?\"x\":\"y\",r=t.getScaleComponent(i);if(!r)return om(i,t,n,{center:!0});const o=r.get(\"type\"),a=t.scaleName(i),{markDef:s,config:l}=t;if(\"band\"===o){const e=lm(i,t.size,t.config.view);if(Ns(e)){const t=sm(e,n);if(t)return t}return[0,{signal:`bandwidth('${a}')`}]}{const n=t.encoding[i];if(Ho(n)&&n.timeUnit){const e=Ri(n.timeUnit,(e=>`scale('${a}', ${e})`)),i=t.config.scale.bandWithNestedOffsetPaddingInner,r=Lo({fieldDef:n,markDef:s,config:l})-.5,o=0!==r?` + ${r}`:\"\";if(i){return[{signal:`${yn(i)?`${i.signal}/2`+o:`${i/2+r}`} * (${e})`},{signal:`${yn(i)?`(1 - ${i.signal}/2)`+o:`${1-i/2+r}`} * (${e})`}]}return[0,{signal:e}]}return c(`Cannot use ${e} scale if ${i} scale is not discrete.`)}}(e,n,u);case ye:{const a=cm(o,n.component.scales[e].get(\"zero\"),r),s=function(e,n,i,r){const o={x:nm(i,\"x\"),y:nm(i,\"y\")};switch(e){case\"bar\":case\"tick\":{if(void 0!==r.scale.maxBandSize)return r.scale.maxBandSize;const e=fm(n,o,r.view);return t.isNumber(e)?e-1:new Ud((()=>`${e.signal} - 1`))}case\"line\":case\"trail\":case\"rule\":return r.scale.maxStrokeWidth;case\"text\":return r.scale.maxFontSize;case\"point\":case\"square\":case\"circle\":{if(r.scale.maxSize)return r.scale.maxSize;const e=fm(n,o,r.view);return t.isNumber(e)?Math.pow(um*e,2):new Ud((()=>`pow(${um} * ${e.signal}, 2)`))}}throw new Error(ai(\"size\",e))}(o,i,n,r);return kr(u)?function(e,t,n){const i=()=>{const i=On(t),r=On(e),o=`(${i} - ${r}) / (${n} - 1)`;return`sequence(${r}, ${i} + ${o}, ${o})`};return yn(t)?new Ud(i):{signal:i()}}(a,s,function(e,n,i,r){switch(e){case\"quantile\":return n.scale.quantileCount;case\"quantize\":return n.scale.quantizeCount;case\"threshold\":return void 0!==i&&t.isArray(i)?i.length+1:($i(function(e){return`Domain for ${e} is required for threshold scale.`}(r)),3)}}(u,r,f,e)):[a,s]}case se:return[0,2*Math.PI];case ve:return[0,360];case oe:return[0,new Ud((()=>`min(${n.getSignalName(\"width\")},${n.getSignalName(\"height\")})/2`))];case we:return[r.scale.minStrokeWidth,r.scale.maxStrokeWidth];case ke:return[[1,0],[4,2],[2,1],[1,1],[1,2,4,2]];case he:return\"symbol\";case me:case pe:case ge:return\"ordinal\"===u?\"nominal\"===s?\"category\":\"ordinal\":void 0!==d?\"diverging\":\"rect\"===o||\"geoshape\"===o?\"heatmap\":\"ramp\";case be:case xe:case $e:return[r.scale.minOpacity,r.scale.maxOpacity]}}(e,n);return(void 0!==l||void 0!==u)&&Ar(o,\"rangeMin\")&&t.isArray(f)&&2===f.length?Kl([l??f[0],u??f[1]]):Zl(f)}function rm(e){return function(e){return!t.isString(e)&&!!e.name}(e)?{scheme:e.name,...f(e,[\"name\"])}:{scheme:e}}function om(e,t,n){let{center:i}=arguments.length>3&&void 0!==arguments[3]?arguments[3]:{};const r=rt(e),o=t.getName(r),a=t.getSignalName.bind(t);return e===ee&&$r(n)?i?[Ud.fromName((e=>`${a(e)}/2`),o),Ud.fromName((e=>`-${a(e)}/2`),o)]:[Ud.fromName(a,o),0]:i?[Ud.fromName((e=>`-${a(e)}/2`),o),Ud.fromName((e=>`${a(e)}/2`),o)]:[0,Ud.fromName(a,o)]}function am(e,n,i){const{encoding:r}=n,o=n.getScaleComponent(i),a=at(i),s=r[a];if(\"offset\"===_s({step:e,offsetIsDiscrete:Jo(s)&&ir(s.type)})&&Ea(r,a)){const i=n.getScaleComponent(a);let r=`domain('${n.scaleName(a)}').length`;if(\"band\"===i.get(\"type\")){r=`bandspace(${r}, ${i.get(\"paddingInner\")??i.get(\"padding\")??0}, ${i.get(\"paddingOuter\")??i.get(\"padding\")??0})`}const s=o.get(\"paddingInner\")??o.get(\"padding\");return{signal:`${e.step} * ${r} / (1-${l=s,yn(l)?l.signal:t.stringValue(l)})`}}return e.step;var l}function sm(e,t){if(\"offset\"===_s({step:e,offsetIsDiscrete:xr(t)}))return{step:e.step}}function lm(e,t,n){const i=e===Z?\"width\":\"height\",r=t[i];return r||Ts(n,i)}function cm(e,t,n){if(t)return yn(t)?{signal:`${t.signal} ? 0 : ${cm(e,!1,n)}`}:0;switch(e){case\"bar\":case\"tick\":return n.scale.minBandSize;case\"line\":case\"trail\":case\"rule\":return n.scale.minStrokeWidth;case\"text\":return n.scale.minFontSize;case\"point\":case\"square\":case\"circle\":return n.scale.minSize}throw new Error(ai(\"size\",e))}const um=.95;function fm(e,t,n){const i=Ns(e.width)?e.width.step:js(n,\"width\"),r=Ns(e.height)?e.height.step:js(n,\"height\");return t.x||t.y?new Ud((()=>`min(${[t.x?t.x.signal:i,t.y?t.y.signal:r].join(\", \")})`)):Math.min(i,r)}function dm(e,t){xm(e)?function(e,t){const n=e.component.scales,{config:i,encoding:r,markDef:o,specifiedScales:a}=e;for(const s of D(n)){const l=a[s],c=n[s],u=e.getScaleComponent(s),f=ga(r[s]),d=l[t],m=u.get(\"type\"),p=u.get(\"padding\"),g=u.get(\"paddingInner\"),h=Ar(m,t),y=jr(s,t);if(void 0!==d&&(h?y&&$i(y):$i(mi(m,t,s))),h&&void 0===y)if(void 0!==d){const e=f.timeUnit,n=f.type;switch(t){case\"domainMax\":case\"domainMin\":wi(l[t])||\"temporal\"===n||e?c.set(t,{signal:wa(l[t],{type:n,timeUnit:e})},!0):c.set(t,l[t],!0);break;default:c.copyKeyFromObject(t,l)}}else{const n=t in mm?mm[t]({model:e,channel:s,fieldOrDatumDef:f,scaleType:m,scalePadding:p,scalePaddingInner:g,domain:l.domain,domainMin:l.domainMin,domainMax:l.domainMax,markDef:o,config:i,hasNestedOffsetScale:Ma(r,s),hasSecondaryRangeChannel:!!r[it(s)]}):i.scale[t];void 0!==n&&c.set(t,n,!1)}}}(e,t):gm(e,t)}const mm={bins:e=>{let{model:t,fieldOrDatumDef:n}=e;return Ho(n)?function(e,t){const n=t.bin;if(ln(n)){const i=ed(e,t.field,n);return new Ud((()=>e.getSignalName(i)))}if(cn(n)&&un(n)&&void 0!==n.step)return{step:n.step};return}(t,n):void 0},interpolate:e=>{let{channel:t,fieldOrDatumDef:n}=e;return function(e,t){if(p([me,pe,ge],e)&&\"nominal\"!==t)return\"hcl\";return}(t,n.type)},nice:e=>{let{scaleType:n,channel:i,domain:r,domainMin:o,domainMax:a,fieldOrDatumDef:s}=e;return function(e,n,i,r,o,a){if(pa(a)?.bin||t.isArray(i)||null!=o||null!=r||p([cr.TIME,cr.UTC],e))return;return!!zt(n)||void 0}(n,i,r,o,a,s)},padding:e=>{let{channel:t,scaleType:n,fieldOrDatumDef:i,markDef:r,config:o}=e;return function(e,t,n,i,r,o){if(zt(e)){if(wr(t)){if(void 0!==n.continuousPadding)return n.continuousPadding;const{type:t,orient:a}=r;if(\"bar\"===t&&(!Ho(i)||!i.bin&&!i.timeUnit)&&(\"vertical\"===a&&\"x\"===e||\"horizontal\"===a&&\"y\"===e))return o.continuousBandSize}if(t===cr.POINT)return n.pointPadding}return}(t,n,o.scale,i,r,o.bar)},paddingInner:e=>{let{scalePadding:t,channel:n,markDef:i,scaleType:r,config:o,hasNestedOffsetScale:a}=e;return function(e,t,n,i,r){let o=arguments.length>5&&void 0!==arguments[5]&&arguments[5];if(void 0!==e)return;if(zt(t)){const{bandPaddingInner:e,barBandPaddingInner:t,rectBandPaddingInner:i,bandWithNestedOffsetPaddingInner:a}=r;return o?a:U(e,\"bar\"===n?t:i)}if(Pt(t)&&i===cr.BAND)return r.offsetBandPaddingInner;return}(t,n,i.type,r,o.scale,a)},paddingOuter:e=>{let{scalePadding:t,channel:n,scaleType:i,scalePaddingInner:r,config:o,hasNestedOffsetScale:a}=e;return function(e,t,n,i,r){let o=arguments.length>5&&void 0!==arguments[5]&&arguments[5];if(void 0!==e)return;if(zt(t)){const{bandPaddingOuter:e,bandWithNestedOffsetPaddingOuter:t}=r;if(o)return t;if(n===cr.BAND)return U(e,yn(i)?{signal:`${i.signal}/2`}:i/2)}else if(Pt(t)){if(n===cr.POINT)return.5;if(n===cr.BAND)return r.offsetBandPaddingOuter}return}(t,n,i,r,o.scale,a)},reverse:e=>{let{fieldOrDatumDef:t,scaleType:n,channel:i,config:r}=e;return function(e,t,n,i){if(\"x\"===n&&void 0!==i.xReverse)return $r(e)&&\"descending\"===t?yn(i.xReverse)?{signal:`!${i.xReverse.signal}`}:!i.xReverse:i.xReverse;if($r(e)&&\"descending\"===t)return!0;return}(n,Ho(t)?t.sort:void 0,i,r.scale)},zero:e=>{let{channel:n,fieldOrDatumDef:i,domain:r,markDef:o,scaleType:a,config:s,hasSecondaryRangeChannel:l}=e;return function(e,n,i,r,o,a,s){if(i&&\"unaggregated\"!==i&&$r(o)){if(t.isArray(i)){const e=i[0],n=i[i.length-1];if(t.isNumber(e)&&e<=0&&t.isNumber(n)&&n>=0)return!0}return!1}if(\"size\"===e&&\"quantitative\"===n.type&&!kr(o))return!0;if((!Ho(n)||!n.bin)&&p([...Ft,..._t],e)){const{orient:t,type:n}=r;return(!p([\"bar\",\"area\",\"line\",\"trail\"],n)||!(\"horizontal\"===t&&\"y\"===e||\"vertical\"===t&&\"x\"===e))&&(!(!p([\"bar\",\"area\"],n)||s)||a?.zero)}return!1}(n,i,r,o,a,s.scale,l)}};function pm(e){xm(e)?function(e){const t=e.component.scales;for(const n of It){const i=t[n];if(!i)continue;const r=im(n,e);i.setWithExplicit(\"range\",r)}}(e):gm(e,\"range\")}function gm(e,t){const n=e.component.scales;for(const n of e.children)\"range\"===t?pm(n):dm(n,t);for(const i of D(n)){let r;for(const n of e.children){const e=n.component.scales[i];if(e){r=nc(r,e.getWithExplicit(t),t,\"scale\",ec(((e,n)=>\"range\"===t&&e.step&&n.step?e.step-n.step:0)))}}n[i].setWithExplicit(t,r)}}function hm(e,t,n,i){const r=function(e,t,n,i){switch(t.type){case\"nominal\":case\"ordinal\":if(qe(e)||\"discrete\"===Qt(e))return\"shape\"===e&&\"ordinal\"===t.type&&$i(ci(e,\"ordinal\")),\"ordinal\";if(zt(e)||Pt(e)){if(p([\"rect\",\"bar\",\"image\",\"rule\"],n.type))return\"band\";if(i)return\"band\"}else if(\"arc\"===n.type&&e in Ot)return\"band\";return io(n[rt(e)])||ta(t)&&t.axis?.tickBand?\"band\":\"point\";case\"temporal\":return qe(e)?\"time\":\"discrete\"===Qt(e)?($i(ci(e,\"temporal\")),\"ordinal\"):Ho(t)&&t.timeUnit&&Ui(t.timeUnit).utc?\"utc\":\"time\";case\"quantitative\":return qe(e)?Ho(t)&&ln(t.bin)?\"bin-ordinal\":\"linear\":\"discrete\"===Qt(e)?($i(ci(e,\"quantitative\")),\"ordinal\"):\"linear\";case\"geojson\":return}throw new Error(ii(t.type))}(t,n,i,arguments.length>4&&void 0!==arguments[4]&&arguments[4]),{type:o}=e;return Ht(t)?void 0!==o?function(e,t){let n=arguments.length>2&&void 0!==arguments[2]&&arguments[2];if(!Ht(e))return!1;switch(e){case Z:case ee:case ie:case re:case se:case oe:return!!wr(t)||\"band\"===t||\"point\"===t&&!n;case ye:case we:case be:case xe:case $e:case ve:return wr(t)||kr(t)||p([\"band\",\"point\",\"ordinal\"],t);case me:case pe:case ge:return\"band\"!==t;case ke:case he:return\"ordinal\"===t||kr(t)}}(t,o)?Ho(n)&&(a=o,s=n.type,!(p([or,sr],s)?void 0===a||xr(a):s===ar?p([cr.TIME,cr.UTC,void 0],a):s!==rr||hr(a)||kr(a)||void 0===a))?($i(function(e,t){return`FieldDef does not work with \"${e}\" scale. We are using \"${t}\" scale instead.`}(o,r)),r):o:($i(function(e,t,n){return`Channel \"${e}\" does not work with \"${t}\" scale. We are using \"${n}\" scale instead.`}(t,o,r)),r):r:null;var a,s}function ym(e){xm(e)?e.component.scales=function(e){const{encoding:t,mark:n,markDef:i}=e,r={};for(const o of It){const a=ga(t[o]);if(a&&n===Xr&&o===he&&a.type===lr)continue;let s=a&&a.scale;if(a&&null!==s&&!1!==s){s??={};const n=hm(s,o,a,i,Ma(t,o));r[o]=new em(e.scaleName(`${o}`,!0),{value:n,explicit:s.type===n})}}return r}(e):e.component.scales=function(e){const t=e.component.scales={},n={},i=e.component.resolve;for(const t of e.children){ym(t);for(const r of D(t.component.scales))if(i.scale[r]??=Of(r,e),\"shared\"===i.scale[r]){const e=n[r],o=t.component.scales[r].getWithExplicit(\"type\");e?fr(e.value,o.value)?n[r]=nc(e,o,\"type\",\"scale\",vm):(i.scale[r]=\"independent\",delete n[r]):n[r]=o}}for(const i of D(n)){const r=e.scaleName(i,!0),o=n[i];t[i]=new em(r,o);for(const t of e.children){const e=t.component.scales[i];e&&(t.renameScale(e.get(\"name\"),r),e.merged=!0)}}return t}(e)}const vm=ec(((e,t)=>mr(e)-mr(t)));class bm{constructor(){qn(this,\"nameMap\",void 0),this.nameMap={}}rename(e,t){this.nameMap[e]=t}has(e){return void 0!==this.nameMap[e]}get(e){for(;this.nameMap[e]&&e!==this.nameMap[e];)e=this.nameMap[e];return e}}function xm(e){return\"unit\"===e?.type}function $m(e){return\"facet\"===e?.type}function wm(e){return\"concat\"===e?.type}function km(e){return\"layer\"===e?.type}class Sm{constructor(e,n,i,r,o,a,c){this.type=n,this.parent=i,this.config=o,qn(this,\"name\",void 0),qn(this,\"size\",void 0),qn(this,\"title\",void 0),qn(this,\"description\",void 0),qn(this,\"data\",void 0),qn(this,\"transforms\",void 0),qn(this,\"layout\",void 0),qn(this,\"scaleNameMap\",void 0),qn(this,\"projectionNameMap\",void 0),qn(this,\"signalNameMap\",void 0),qn(this,\"component\",void 0),qn(this,\"view\",void 0),qn(this,\"children\",void 0),qn(this,\"correctDataNames\",(e=>(e.from?.data&&(e.from.data=this.lookupDataSource(e.from.data)),e.from?.facet?.data&&(e.from.facet.data=this.lookupDataSource(e.from.facet.data)),e))),this.parent=i,this.config=o,this.view=pn(c),this.name=e.name??r,this.title=hn(e.title)?{text:e.title}:e.title?pn(e.title):void 0,this.scaleNameMap=i?i.scaleNameMap:new bm,this.projectionNameMap=i?i.projectionNameMap:new bm,this.signalNameMap=i?i.signalNameMap:new bm,this.data=e.data,this.description=e.description,this.transforms=(e.transform??[]).map((e=>bl(e)?{filter:s(e.filter,tr)}:e)),this.layout=\"layer\"===n||\"unit\"===n?{}:function(e,n,i){const r=i[n],o={},{spacing:a,columns:s}=r;void 0!==a&&(o.spacing=a),void 0!==s&&(To(e)&&!Ao(e.facet)||Fs(e))&&(o.columns=s),zs(e)&&(o.columns=1);for(const n of Ps)if(void 0!==e[n])if(\"spacing\"===n){const i=e[n];o[n]=t.isNumber(i)?i:{row:i.row??a,column:i.column??a}}else o[n]=e[n];return o}(e,n,o),this.component={data:{sources:i?i.component.data.sources:[],outputNodes:i?i.component.data.outputNodes:{},outputNodeRefCounts:i?i.component.data.outputNodeRefCounts:{},isFaceted:To(e)||i?.component.data.isFaceted&&void 0===e.data},layoutSize:new Jl,layoutHeaders:{row:{},column:{},facet:{}},mark:null,resolve:{scale:{},axis:{},legend:{},...a?l(a):{}},selection:null,scales:null,projection:null,axes:{},legends:{}}}get width(){return this.getSizeSignalRef(\"width\")}get height(){return this.getSizeSignalRef(\"height\")}parse(){this.parseScale(),this.parseLayoutSize(),this.renameTopLevelLayoutSizeSignal(),this.parseSelections(),this.parseProjection(),this.parseData(),this.parseAxesAndHeaders(),this.parseLegends(),this.parseMarkGroup()}parseScale(){!function(e){let{ignoreRange:t}=arguments.length>1&&void 0!==arguments[1]?arguments[1]:{};ym(e),Rd(e);for(const t of Pr)dm(e,t);t||pm(e)}(this)}parseProjection(){Jf(this)}renameTopLevelLayoutSizeSignal(){\"width\"!==this.getName(\"width\")&&this.renameSignal(this.getName(\"width\"),\"width\"),\"height\"!==this.getName(\"height\")&&this.renameSignal(this.getName(\"height\"),\"height\")}parseLegends(){Rf(this)}assembleEncodeFromView(e){const{style:t,...n}=e,i={};for(const e of D(n)){const t=n[e];void 0!==t&&(i[e]=Fn(t))}return i}assembleGroupEncodeEntry(e){let t={};return this.view&&(t=this.assembleEncodeFromView(this.view)),e||(this.description&&(t.description=Fn(this.description)),\"unit\"!==this.type&&\"layer\"!==this.type)?S(t)?void 0:t:{width:this.getSizeSignalRef(\"width\"),height:this.getSizeSignalRef(\"height\"),...t??{}}}assembleLayout(){if(!this.layout)return;const{spacing:e,...t}=this.layout,{component:n,config:i}=this,r=function(e,t){const n={};for(const i of Re){const r=e[i];if(r?.facetFieldDef){const{titleAnchor:e,titleOrient:o}=cf([\"titleAnchor\",\"titleOrient\"],r.facetFieldDef.header,t,i),a=sf(i,o),s=xf(e,a);void 0!==s&&(n[a]=s)}}return S(n)?void 0:n}(n.layoutHeaders,i);return{padding:e,...this.assembleDefaultLayout(),...t,...r?{titleBand:r}:{}}}assembleDefaultLayout(){return{}}assembleHeaderMarks(){const{layoutHeaders:e}=this.component;let t=[];for(const n of Re)e[n].title&&t.push(df(this,n));for(const e of uf)t=t.concat(gf(this,e));return t}assembleAxes(){return function(e,t){const{x:n=[],y:i=[]}=e;return[...n.map((e=>Yu(e,\"grid\",t))),...i.map((e=>Yu(e,\"grid\",t))),...n.map((e=>Yu(e,\"main\",t))),...i.map((e=>Yu(e,\"main\",t)))].filter((e=>e))}(this.component.axes,this.config)}assembleLegends(){return Vf(this)}assembleProjections(){return Gf(this)}assembleTitle(){const{encoding:e,...t}=this.title??{},n={...gn(this.config.title).nonMarkTitleProperties,...t,...e?{encode:{update:e}}:{}};if(n.text)return p([\"unit\",\"layer\"],this.type)?p([\"middle\",void 0],n.anchor)&&(n.frame??=\"group\"):n.anchor??=\"start\",S(n)?void 0:n}assembleGroup(){let e=arguments.length>0&&void 0!==arguments[0]?arguments[0]:[];const t={};e=e.concat(this.assembleSignals()),e.length>0&&(t.signals=e);const n=this.assembleLayout();n&&(t.layout=n),t.marks=[].concat(this.assembleHeaderMarks(),this.assembleMarks());const i=!this.parent||$m(this.parent)?Kd(this):[];i.length>0&&(t.scales=i);const r=this.assembleAxes();r.length>0&&(t.axes=r);const o=this.assembleLegends();return o.length>0&&(t.legends=o),t}getName(e){return _((this.name?`${this.name}_`:\"\")+e)}getDataName(e){return this.getName(fc[e].toLowerCase())}requestDataName(e){const t=this.getDataName(e),n=this.component.data.outputNodeRefCounts;return n[t]=(n[t]||0)+1,t}getSizeSignalRef(e){if($m(this.parent)){const t=Nt(Ff(e)),n=this.component.scales[t];if(n&&!n.merged){const e=n.get(\"type\"),i=n.get(\"range\");if(xr(e)&&vn(i)){const e=n.get(\"name\"),i=Qd(Jd(this,t));if(i){return{signal:Df(e,n,oa({aggregate:\"distinct\",field:i},{expr:\"datum\"}))}}return $i(Yn(t)),null}}}return{signal:this.signalNameMap.get(this.getName(e))}}lookupDataSource(e){const t=this.component.data.outputNodes[e];return t?t.getSource():e}getSignalName(e){return this.signalNameMap.get(e)}renameSignal(e,t){this.signalNameMap.rename(e,t)}renameScale(e,t){this.scaleNameMap.rename(e,t)}renameProjection(e,t){this.projectionNameMap.rename(e,t)}scaleName(e,t){return t?this.getName(e):Ke(e)&&Ht(e)&&this.component.scales[e]||this.scaleNameMap.has(this.getName(e))?this.scaleNameMap.get(this.getName(e)):void 0}projectionName(e){return e?this.getName(\"projection\"):this.component.projection&&!this.component.projection.merged||this.projectionNameMap.has(this.getName(\"projection\"))?this.projectionNameMap.get(this.getName(\"projection\")):void 0}getScaleComponent(e){if(!this.component.scales)throw new Error(\"getScaleComponent cannot be called before parseScale(). Make sure you have called parseScale or use parseUnitModelWithScale().\");const t=this.component.scales[e];return t&&!t.merged?t:this.parent?this.parent.getScaleComponent(e):void 0}getSelectionComponent(e,t){let n=this.component.selection[e];if(!n&&this.parent&&(n=this.parent.getSelectionComponent(e,t)),!n)throw new Error(function(e){return`Cannot find a selection named \"${e}\".`}(t));return n}hasAxisOrientSignalRef(){return this.component.axes.x?.some((e=>e.hasOrientSignalRef()))||this.component.axes.y?.some((e=>e.hasOrientSignalRef()))}}class Dm extends Sm{vgField(e){let t=arguments.length>1&&void 0!==arguments[1]?arguments[1]:{};const n=this.fieldDef(e);if(n)return oa(n,t)}reduceFieldDef(e,n){return function(e,n,i,r){return e?D(e).reduce(((i,o)=>{const a=e[o];return t.isArray(a)?a.reduce(((e,t)=>n.call(r,e,t,o)),i):n.call(r,i,a,o)}),i):i}(this.getMapping(),((t,n,i)=>{const r=pa(n);return r?e(t,r,i):t}),n)}forEachFieldDef(e,t){Wa(this.getMapping(),((t,n)=>{const i=pa(t);i&&e(i,n)}),t)}}class Fm extends vc{clone(){return new Fm(null,l(this.transform))}constructor(e,t){super(e),this.transform=t,this.transform=l(t);const n=this.transform.as??[void 0,void 0];this.transform.as=[n[0]??\"value\",n[1]??\"density\"]}dependentFields(){return new Set([this.transform.density,...this.transform.groupby??[]])}producedFields(){return new Set(this.transform.as)}hash(){return`DensityTransform ${d(this.transform)}`}assemble(){const{density:e,...t}=this.transform,n={type:\"kde\",field:e,...t};return this.transform.groupby&&(n.resolve=\"shared\"),n}}class zm extends vc{clone(){return new zm(null,l(this.transform))}constructor(e,t){super(e),this.transform=t,this.transform=l(t)}dependentFields(){return new Set([this.transform.extent])}producedFields(){return new Set([])}hash(){return`ExtentTransform ${d(this.transform)}`}assemble(){const{extent:e,param:t}=this.transform;return{type:\"extent\",field:e,signal:t}}}class Om extends vc{clone(){return new Om(null,{...this.filter})}constructor(e,t){super(e),this.filter=t}static make(e,t){const{config:n,mark:i,markDef:r}=t;if(\"filter\"!==Cn(\"invalid\",r,n))return null;const o=t.reduceFieldDef(((e,n,r)=>{const o=Ht(r)&&t.getScaleComponent(r);if(o){$r(o.get(\"type\"))&&\"count\"!==n.aggregate&&!Qr(i)&&(e[n.field]=n)}return e}),{});return D(o).length?new Om(e,o):null}dependentFields(){return new Set(D(this.filter))}producedFields(){return new Set}hash(){return`FilterInvalid ${d(this.filter)}`}assemble(){const e=D(this.filter).reduce(((e,t)=>{const n=this.filter[t],i=oa(n,{expr:\"datum\"});return null!==n&&(\"temporal\"===n.type?e.push(`(isDate(${i}) || (isValid(${i}) && isFinite(+${i})))`):\"quantitative\"===n.type&&(e.push(`isValid(${i})`),e.push(`isFinite(+${i})`))),e}),[]);return e.length>0?{type:\"filter\",expr:e.join(\" && \")}:null}}class _m extends vc{clone(){return new _m(this.parent,l(this.transform))}constructor(e,t){super(e),this.transform=t,this.transform=l(t);const{flatten:n,as:i=[]}=this.transform;this.transform.as=n.map(((e,t)=>i[t]??e))}dependentFields(){return new Set(this.transform.flatten)}producedFields(){return new Set(this.transform.as)}hash(){return`FlattenTransform ${d(this.transform)}`}assemble(){const{flatten:e,as:t}=this.transform;return{type:\"flatten\",fields:e,as:t}}}class Nm extends vc{clone(){return new Nm(null,l(this.transform))}constructor(e,t){super(e),this.transform=t,this.transform=l(t);const n=this.transform.as??[void 0,void 0];this.transform.as=[n[0]??\"key\",n[1]??\"value\"]}dependentFields(){return new Set(this.transform.fold)}producedFields(){return new Set(this.transform.as)}hash(){return`FoldTransform ${d(this.transform)}`}assemble(){const{fold:e,as:t}=this.transform;return{type:\"fold\",fields:e,as:t}}}class Cm extends vc{clone(){return new Cm(null,l(this.fields),this.geojson,this.signal)}static parseAll(e,t){if(t.component.projection&&!t.component.projection.isFit)return e;let n=0;for(const i of[[ue,ce],[de,fe]]){const r=i.map((e=>{const n=ga(t.encoding[e]);return Ho(n)?n.field:Go(n)?{expr:`${n.datum}`}:Zo(n)?{expr:`${n.value}`}:void 0}));(r[0]||r[1])&&(e=new Cm(e,r,null,t.getName(\"geojson_\"+n++)))}if(t.channelHasField(he)){const i=t.typedFieldDef(he);i.type===lr&&(e=new Cm(e,null,i.field,t.getName(\"geojson_\"+n++)))}return e}constructor(e,t,n,i){super(e),this.fields=t,this.geojson=n,this.signal=i}dependentFields(){const e=(this.fields??[]).filter(t.isString);return new Set([...this.geojson?[this.geojson]:[],...e])}producedFields(){return new Set}hash(){return`GeoJSON ${this.geojson} ${this.signal} ${d(this.fields)}`}assemble(){return[...this.geojson?[{type:\"filter\",expr:`isValid(datum[\"${this.geojson}\"])`}]:[],{type:\"geojson\",...this.fields?{fields:this.fields}:{},...this.geojson?{geojson:this.geojson}:{},signal:this.signal}]}}class Pm extends vc{clone(){return new Pm(null,this.projection,l(this.fields),l(this.as))}constructor(e,t,n,i){super(e),this.projection=t,this.fields=n,this.as=i}static parseAll(e,t){if(!t.projectionName())return e;for(const n of[[ue,ce],[de,fe]]){const i=n.map((e=>{const n=ga(t.encoding[e]);return Ho(n)?n.field:Go(n)?{expr:`${n.datum}`}:Zo(n)?{expr:`${n.value}`}:void 0})),r=n[0]===de?\"2\":\"\";(i[0]||i[1])&&(e=new Pm(e,t.projectionName(),i,[t.getName(`x${r}`),t.getName(`y${r}`)]))}return e}dependentFields(){return new Set(this.fields.filter(t.isString))}producedFields(){return new Set(this.as)}hash(){return`Geopoint ${this.projection} ${d(this.fields)} ${d(this.as)}`}assemble(){return{type:\"geopoint\",projection:this.projection,fields:this.fields,as:this.as}}}class Am extends vc{clone(){return new Am(null,l(this.transform))}constructor(e,t){super(e),this.transform=t}dependentFields(){return new Set([this.transform.impute,this.transform.key,...this.transform.groupby??[]])}producedFields(){return new Set([this.transform.impute])}processSequence(e){const{start:t=0,stop:n,step:i}=e;return{signal:`sequence(${[t,n,...i?[i]:[]].join(\",\")})`}}static makeFromTransform(e,t){return new Am(e,t)}static makeFromEncoding(e,t){const n=t.encoding,i=n.x,r=n.y;if(Ho(i)&&Ho(r)){const o=i.impute?i:r.impute?r:void 0;if(void 0===o)return;const a=i.impute?r:r.impute?i:void 0,{method:s,value:l,frame:c,keyvals:u}=o.impute,f=Ba(t.mark,n);return new Am(e,{impute:o.field,key:a.field,...s?{method:s}:{},...void 0!==l?{value:l}:{},...c?{frame:c}:{},...void 0!==u?{keyvals:u}:{},...f.length?{groupby:f}:{}})}return null}hash(){return`Impute ${d(this.transform)}`}assemble(){const{impute:e,key:t,keyvals:n,method:i,groupby:r,value:o,frame:a=[null,null]}=this.transform,s={type:\"impute\",field:e,key:t,...n?{keyvals:(l=n,void 0!==l?.stop?this.processSequence(n):n)}:{},method:\"value\",...r?{groupby:r}:{},value:i&&\"value\"!==i?null:o};var l;if(i&&\"value\"!==i){return[s,{type:\"window\",as:[`imputed_${e}_value`],ops:[i],fields:[e],frame:a,ignorePeers:!1,...r?{groupby:r}:{}},{type:\"formula\",expr:`datum.${e} === null ? datum.imputed_${e}_value : datum.${e}`,as:e}]}return[s]}}class jm extends vc{clone(){return new jm(null,l(this.transform))}constructor(e,t){super(e),this.transform=t,this.transform=l(t);const n=this.transform.as??[void 0,void 0];this.transform.as=[n[0]??t.on,n[1]??t.loess]}dependentFields(){return new Set([this.transform.loess,this.transform.on,...this.transform.groupby??[]])}producedFields(){return new Set(this.transform.as)}hash(){return`LoessTransform ${d(this.transform)}`}assemble(){const{loess:e,on:t,...n}=this.transform;return{type:\"loess\",x:t,y:e,...n}}}class Tm extends vc{clone(){return new Tm(null,l(this.transform),this.secondary)}constructor(e,t,n){super(e),this.transform=t,this.secondary=n}static make(e,t,n,i){const r=t.component.data.sources,{from:o}=n;let a=null;if(function(e){return\"data\"in e}(o)){let e=Qm(o.data,r);e||(e=new md(o.data),r.push(e));const n=t.getName(`lookup_${i}`);a=new bc(e,n,fc.Lookup,t.component.data.outputNodeRefCounts),t.component.data.outputNodes[n]=a}else if(function(e){return\"param\"in e}(o)){const e=o.param;let i;n={as:e,...n};try{i=t.getSelectionComponent(_(e),e)}catch(t){throw new Error(function(e){return`Lookups can only be performed on selection parameters. \"${e}\" is a variable parameter.`}(e))}if(a=i.materialized,!a)throw new Error(function(e){return`Cannot define and lookup the \"${e}\" selection in the same view. Try moving the lookup into a second, layered view?`}(e))}return new Tm(e,n,a.getSource())}dependentFields(){return new Set([this.transform.lookup])}producedFields(){return new Set(this.transform.as?t.array(this.transform.as):this.transform.from.fields)}hash(){return`Lookup ${d({transform:this.transform,secondary:this.secondary})}`}assemble(){let e;if(this.transform.from.fields)e={values:this.transform.from.fields,...this.transform.as?{as:t.array(this.transform.as)}:{}};else{let n=this.transform.as;t.isString(n)||($i('If \"from.fields\" is not specified, \"as\" has to be a string that specifies the key to be used for the data from the secondary source.'),n=\"_lookup\"),e={as:[n]}}return{type:\"lookup\",from:this.secondary,key:this.transform.from.key,fields:[this.transform.lookup],...e,...this.transform.default?{default:this.transform.default}:{}}}}class Em extends vc{clone(){return new Em(null,l(this.transform))}constructor(e,t){super(e),this.transform=t,this.transform=l(t);const n=this.transform.as??[void 0,void 0];this.transform.as=[n[0]??\"prob\",n[1]??\"value\"]}dependentFields(){return new Set([this.transform.quantile,...this.transform.groupby??[]])}producedFields(){return new Set(this.transform.as)}hash(){return`QuantileTransform ${d(this.transform)}`}assemble(){const{quantile:e,...t}=this.transform;return{type:\"quantile\",field:e,...t}}}class Mm extends vc{clone(){return new Mm(null,l(this.transform))}constructor(e,t){super(e),this.transform=t,this.transform=l(t);const n=this.transform.as??[void 0,void 0];this.transform.as=[n[0]??t.on,n[1]??t.regression]}dependentFields(){return new Set([this.transform.regression,this.transform.on,...this.transform.groupby??[]])}producedFields(){return new Set(this.transform.as)}hash(){return`RegressionTransform ${d(this.transform)}`}assemble(){const{regression:e,on:t,...n}=this.transform;return{type:\"regression\",x:t,y:e,...n}}}class Lm extends vc{clone(){return new Lm(null,l(this.transform))}constructor(e,t){super(e),this.transform=t}addDimensions(e){this.transform.groupby=b((this.transform.groupby??[]).concat(e),(e=>e))}producedFields(){}dependentFields(){return new Set([this.transform.pivot,this.transform.value,...this.transform.groupby??[]])}hash(){return`PivotTransform ${d(this.transform)}`}assemble(){const{pivot:e,value:t,groupby:n,limit:i,op:r}=this.transform;return{type:\"pivot\",field:e,value:t,...void 0!==i?{limit:i}:{},...void 0!==r?{op:r}:{},...void 0!==n?{groupby:n}:{}}}}class qm extends vc{clone(){return new qm(null,l(this.transform))}constructor(e,t){super(e),this.transform=t}dependentFields(){return new Set}producedFields(){return new Set}hash(){return`SampleTransform ${d(this.transform)}`}assemble(){return{type:\"sample\",size:this.transform.sample}}}function Um(e){let t=0;return function n(i,r){if(i instanceof md&&!i.isGenerator&&!rc(i.data)){e.push(r);r={name:null,source:r.name,transform:[]}}if(i instanceof cd&&(i.parent instanceof md&&!r.source?(r.format={...r.format??{},parse:i.assembleFormatParse()},r.transform.push(...i.assembleTransforms(!0))):r.transform.push(...i.assembleTransforms())),i instanceof od)return r.name||(r.name=\"data_\"+t++),!r.source||r.transform.length>0?(e.push(r),i.data=r.name):i.data=r.source,void e.push(...i.assemble());if((i instanceof fd||i instanceof dd||i instanceof Om||i instanceof Bu||i instanceof of||i instanceof Pm||i instanceof rd||i instanceof Tm||i instanceof Pd||i instanceof Nd||i instanceof Nm||i instanceof _m||i instanceof Fm||i instanceof jm||i instanceof Em||i instanceof Mm||i instanceof ud||i instanceof qm||i instanceof Lm||i instanceof zm)&&r.transform.push(i.assemble()),(i instanceof nd||i instanceof wc||i instanceof Am||i instanceof Cd||i instanceof Cm)&&r.transform.push(...i.assemble()),i instanceof bc)if(r.source&&0===r.transform.length)i.setSource(r.source);else if(i.parent instanceof bc)i.setSource(r.name);else if(r.name||(r.name=\"data_\"+t++),i.setSource(r.name),1===i.numChildren()){e.push(r);r={name:null,source:r.name,transform:[]}}switch(i.numChildren()){case 0:i instanceof bc&&(!r.source||r.transform.length>0)&&e.push(r);break;case 1:n(i.children[0],r);break;default:{r.name||(r.name=\"data_\"+t++);let o=r.name;!r.source||r.transform.length>0?e.push(r):o=r.source;for(const e of i.children){n(e,{name:null,source:o,transform:[]})}break}}}}function Rm(e){return\"top\"===e||\"left\"===e||yn(e)?\"header\":\"footer\"}function Wm(e,n){const{facet:i,config:r,child:o,component:a}=e;if(e.channelHasField(n)){const s=i[n],l=lf(\"title\",null,r,n);let c=ua(s,r,{allowDisabling:!0,includeDefault:void 0===l||!!l});o.component.layoutHeaders[n].title&&(c=t.isArray(c)?c.join(\", \"):c,c+=` / ${o.component.layoutHeaders[n].title}`,o.component.layoutHeaders[n].title=null);const u=lf(\"labelOrient\",s.header,r,n),f=null!==s.header&&U(s.header?.labels,r.header.labels,!0),d=p([\"bottom\",\"right\"],u)?\"footer\":\"header\";a.layoutHeaders[n]={title:null!==s.header?c:null,facetFieldDef:s,[d]:\"facet\"===n?[]:[Bm(e,n,f)]}}}function Bm(e,t,n){const i=\"row\"===t?\"height\":\"width\";return{labels:n,sizeSignal:e.child.component.layoutSize.get(i)?e.child.getSizeSignalRef(i):void 0,axes:[]}}function Im(e,t){const{child:n}=e;if(n.component.axes[t]){const{layoutHeaders:i,resolve:r}=e.component;if(r.axis[t]=_f(r,t),\"shared\"===r.axis[t]){const r=\"x\"===t?\"column\":\"row\",o=i[r];for(const i of n.component.axes[t]){const t=Rm(i.get(\"orient\"));o[t]??=[Bm(e,r,!1)];const n=Yu(i,\"main\",e.config,{header:!0});n&&o[t][0].axes.push(n),i.mainExtracted=!0}}}}function Hm(e){for(const t of e.children)t.parseLayoutSize()}function Vm(e,t){const n=Ff(t),i=Nt(n),r=e.component.resolve,o=e.component.layoutSize;let a;for(const t of e.children){const o=t.component.layoutSize.getWithExplicit(n),s=r.scale[i]??Of(i,e);if(\"independent\"===s&&\"step\"===o.value){a=void 0;break}if(a){if(\"independent\"===s&&a.value!==o.value){a=void 0;break}a=nc(a,o,n,\"\")}else a=o}if(a){for(const i of e.children)e.renameSignal(i.getName(n),e.getName(t)),i.component.layoutSize.set(n,\"merged\",!1);o.setWithExplicit(t,a)}else o.setWithExplicit(t,{explicit:!1,value:void 0})}function Gm(e,t){const n=\"width\"===t?\"x\":\"y\",i=e.config,r=e.getScaleComponent(n);if(r){const e=r.get(\"type\"),n=r.get(\"range\");if(xr(e)){const e=Ts(i.view,t);return vn(n)||Ns(e)?\"step\":e}return As(i.view,t)}if(e.hasProjection||\"arc\"===e.mark)return As(i.view,t);{const e=Ts(i.view,t);return Ns(e)?e.step:e}}function Ym(e,t,n){return oa(t,{suffix:`by_${oa(e)}`,...n??{}})}class Xm extends Dm{constructor(e,t,n,i){super(e,\"facet\",t,n,i,e.resolve),qn(this,\"facet\",void 0),qn(this,\"child\",void 0),qn(this,\"children\",void 0),this.child=wp(e.spec,this,this.getName(\"child\"),void 0,i),this.children=[this.child],this.facet=this.initFacet(e.facet)}initFacet(e){if(!Ao(e))return{facet:this.initFacetFieldDef(e,\"facet\")};const t=D(e),n={};for(const i of t){if(![Q,J].includes(i)){$i(ai(i,\"facet\"));break}const t=e[i];if(void 0===t.field){$i(oi(t,i));break}n[i]=this.initFacetFieldDef(t,i)}return n}initFacetFieldDef(e,t){const n=va(e,t);return n.header?n.header=pn(n.header):null===n.header&&(n.header=null),n}channelHasField(e){return!!this.facet[e]}fieldDef(e){return this.facet[e]}parseData(){this.component.data=Jm(this),this.child.parseData()}parseLayoutSize(){Hm(this)}parseSelections(){this.child.parseSelections(),this.component.selection=this.child.component.selection}parseMarkGroup(){this.child.parseMarkGroup()}parseAxesAndHeaders(){this.child.parseAxesAndHeaders(),function(e){for(const t of Re)Wm(e,t);Im(e,\"x\"),Im(e,\"y\")}(this)}assembleSelectionTopLevelSignals(e){return this.child.assembleSelectionTopLevelSignals(e)}assembleSignals(){return this.child.assembleSignals(),[]}assembleSelectionData(e){return this.child.assembleSelectionData(e)}getHeaderLayoutMixins(){const e={};for(const t of Re)for(const n of ff){const i=this.component.layoutHeaders[t],r=i[n],{facetFieldDef:o}=i;if(o){const n=lf(\"titleOrient\",o.header,this.config,t);if([\"right\",\"bottom\"].includes(n)){const i=sf(t,n);e.titleAnchor??={},e.titleAnchor[i]=\"end\"}}if(r?.[0]){const r=\"row\"===t?\"height\":\"width\",o=\"header\"===n?\"headerBand\":\"footerBand\";\"facet\"===t||this.child.component.layoutSize.get(r)||(e[o]??={},e[o][t]=.5),i.title&&(e.offset??={},e.offset[\"row\"===t?\"rowTitle\":\"columnTitle\"]=10)}}return e}assembleDefaultLayout(){const{column:e,row:t}=this.facet,n=e?this.columnDistinctSignal():t?1:void 0;let i=\"all\";return(t||\"independent\"!==this.component.resolve.scale.x)&&(e||\"independent\"!==this.component.resolve.scale.y)||(i=\"none\"),{...this.getHeaderLayoutMixins(),...n?{columns:n}:{},bounds:\"full\",align:i}}assembleLayoutSignals(){return this.child.assembleLayoutSignals()}columnDistinctSignal(){if(!(this.parent&&this.parent instanceof Xm)){return{signal:`length(data('${this.getName(\"column_domain\")}'))`}}}assembleGroupStyle(){}assembleGroup(e){return this.parent&&this.parent instanceof Xm?{...this.channelHasField(\"column\")?{encode:{update:{columns:{field:oa(this.facet.column,{prefix:\"distinct\"})}}}}:{},...super.assembleGroup(e)}:super.assembleGroup(e)}getCardinalityAggregateForChild(){const e=[],t=[],n=[];if(this.child instanceof Xm){if(this.child.channelHasField(\"column\")){const i=oa(this.child.facet.column);e.push(i),t.push(\"distinct\"),n.push(`distinct_${i}`)}}else for(const i of Ft){const r=this.child.component.scales[i];if(r&&!r.merged){const o=r.get(\"type\"),a=r.get(\"range\");if(xr(o)&&vn(a)){const r=Qd(Jd(this.child,i));r?(e.push(r),t.push(\"distinct\"),n.push(`distinct_${r}`)):$i(Yn(i))}}}return{fields:e,ops:t,as:n}}assembleFacet(){const{name:e,data:n}=this.component.data.facetRoot,{row:i,column:r}=this.facet,{fields:o,ops:a,as:s}=this.getCardinalityAggregateForChild(),l=[];for(const e of Re){const n=this.facet[e];if(n){l.push(oa(n));const{bin:c,sort:u}=n;if(ln(c)&&l.push(oa(n,{binSuffix:\"end\"})),Co(u)){const{field:e,op:t=zo}=u,l=Ym(n,u);i&&r?(o.push(l),a.push(\"max\"),s.push(l)):(o.push(e),a.push(t),s.push(l))}else if(t.isArray(u)){const t=af(n,e);o.push(t),a.push(\"max\"),s.push(t)}}}const c=!!i&&!!r;return{name:e,data:n,groupby:l,...c||o.length>0?{aggregate:{...c?{cross:c}:{},...o.length?{fields:o,ops:a,as:s}:{}}}:{}}}facetSortFields(e){const{facet:n}=this,i=n[e];return i?Co(i.sort)?[Ym(i,i.sort,{expr:\"datum\"})]:t.isArray(i.sort)?[af(i,e,{expr:\"datum\"})]:[oa(i,{expr:\"datum\"})]:[]}facetSortOrder(e){const{facet:n}=this,i=n[e];if(i){const{sort:e}=i;return[(Co(e)?e.order:!t.isArray(e)&&e)||\"ascending\"]}return[]}assembleLabelTitle(){const{facet:e,config:t}=this;if(e.facet)return yf(e.facet,\"facet\",t);const n={row:[\"top\",\"bottom\"],column:[\"left\",\"right\"]};for(const i of uf)if(e[i]){const r=lf(\"labelOrient\",e[i]?.header,t,i);if(n[i].includes(r))return yf(e[i],i,t)}}assembleMarks(){const{child:e}=this,t=function(e){const t=[],n=Um(t);for(const t of e.children)n(t,{source:e.name,name:null,transform:[]});return t}(this.component.data.facetRoot),n=e.assembleGroupEncodeEntry(!1),i=this.assembleLabelTitle()||e.assembleTitle(),r=e.assembleGroupStyle();return[{name:this.getName(\"cell\"),type:\"group\",...i?{title:i}:{},...r?{style:r}:{},from:{facet:this.assembleFacet()},sort:{field:Re.map((e=>this.facetSortFields(e))).flat(),order:Re.map((e=>this.facetSortOrder(e))).flat()},...t.length>0?{data:t}:{},...n?{encode:{update:n}}:{},...e.assembleGroup(gc(this,[]))}]}getMapping(){return this.facet}}function Qm(e,t){for(const n of t){const t=n.data;if(e.name&&n.hasName()&&e.name!==n.dataName)continue;const i=e.format?.mesh,r=t.format?.feature;if(i&&r)continue;const o=e.format?.feature;if((o||r)&&o!==r)continue;const a=t.format?.mesh;if(!i&&!a||i===a)if(oc(e)&&oc(t)){if(Y(e.values,t.values))return n}else if(rc(e)&&rc(t)){if(e.url===t.url)return n}else if(ac(e)&&e.name===n.dataName)return n}return null}function Jm(e){let t=function(e,t){if(e.data||!e.parent){if(null===e.data){const e=new md({values:[]});return t.push(e),e}const n=Qm(e.data,t);if(n)return sc(e.data)||(n.data.format=y({},e.data.format,n.data.format)),!n.hasName()&&e.data.name&&(n.dataName=e.data.name),n;{const n=new md(e.data);return t.push(n),n}}return e.parent.component.data.facetRoot?e.parent.component.data.facetRoot:e.parent.component.data.main}(e,e.component.data.sources);const{outputNodes:n,outputNodeRefCounts:i}=e.component.data,r=e.data,o=!(r&&(sc(r)||rc(r)||oc(r)))&&e.parent?e.parent.component.data.ancestorParse.clone():new ic;sc(r)?(lc(r)?t=new dd(t,r.sequence):uc(r)&&(t=new fd(t,r.graticule)),o.parseNothing=!0):null===r?.format?.parse&&(o.parseNothing=!0),t=cd.makeExplicit(t,e,o)??t,t=new ud(t);const a=e.parent&&km(e.parent);(xm(e)||$m(e))&&a&&(t=nd.makeFromEncoding(t,e)??t),e.transforms.length>0&&(t=function(e,t,n){let i=0;for(const r of t.transforms){let o,a;if(Nl(r))a=e=new of(e,r),o=\"derived\";else if(bl(r)){const i=sd(r);a=e=cd.makeWithAncestors(e,{},i,n)??e,e=new Bu(e,t,r.filter)}else if(Cl(r))a=e=nd.makeFromTransform(e,r,t),o=\"number\";else if(Al(r))o=\"date\",void 0===n.getWithExplicit(r.field).value&&(e=new cd(e,{[r.field]:o}),n.set(r.field,o,!1)),a=e=wc.makeFromTransform(e,r);else if(jl(r))a=e=rd.makeFromTransform(e,r),o=\"number\",Lu(t)&&(e=new ud(e));else if(xl(r))a=e=Tm.make(e,t,r,i++),o=\"derived\";else if(zl(r))a=e=new Pd(e,r),o=\"number\";else if(Ol(r))a=e=new Nd(e,r),o=\"number\";else if(Tl(r))a=e=Cd.makeFromTransform(e,r),o=\"derived\";else if(El(r))a=e=new Nm(e,r),o=\"derived\";else if(Ml(r))a=e=new zm(e,r),o=\"derived\";else if(_l(r))a=e=new _m(e,r),o=\"derived\";else if($l(r))a=e=new Lm(e,r),o=\"derived\";else if(Fl(r))e=new qm(e,r);else if(Pl(r))a=e=Am.makeFromTransform(e,r),o=\"derived\";else if(wl(r))a=e=new Fm(e,r),o=\"derived\";else if(kl(r))a=e=new Em(e,r),o=\"derived\";else if(Sl(r))a=e=new Mm(e,r),o=\"derived\";else{if(!Dl(r)){$i(`Ignoring an invalid transform: ${X(r)}.`);continue}a=e=new jm(e,r),o=\"derived\"}if(a&&void 0!==o)for(const e of a.producedFields()??[])n.set(e,o,!1)}return e}(t,e,o));const s=function(e){const t={};if(xm(e)&&e.component.selection)for(const n of D(e.component.selection)){const i=e.component.selection[n];for(const e of i.project.items)!e.channel&&q(e.field)>1&&(t[e.field]=\"flatten\")}return t}(e),l=ld(e);t=cd.makeWithAncestors(t,{},{...s,...l},o)??t,xm(e)&&(t=Cm.parseAll(t,e),t=Pm.parseAll(t,e)),(xm(e)||$m(e))&&(a||(t=nd.makeFromEncoding(t,e)??t),t=wc.makeFromEncoding(t,e)??t,t=of.parseAllForSortIndex(t,e));const c=e.getDataName(fc.Raw),u=new bc(t,c,fc.Raw,i);if(n[c]=u,t=u,xm(e)){const n=rd.makeFromEncoding(t,e);n&&(t=n,Lu(e)&&(t=new ud(t))),t=Am.makeFromEncoding(t,e)??t,t=Cd.makeFromEncoding(t,e)??t}xm(e)&&(t=Om.make(t,e)??t);const f=e.getDataName(fc.Main),d=new bc(t,f,fc.Main,i);n[f]=d,t=d,xm(e)&&function(e,t){for(const[n,i]of z(e.component.selection??{})){const r=e.getName(`lookup_${n}`);e.component.data.outputNodes[r]=i.materialized=new bc(new Bu(t,e,{param:n}),r,fc.Lookup,e.component.data.outputNodeRefCounts)}}(e,d);let m=null;if($m(e)){const i=e.getName(\"facet\");t=function(e,t){const{row:n,column:i}=t;if(n&&i){let t=null;for(const r of[n,i])if(Co(r.sort)){const{field:n,op:i=zo}=r.sort;e=t=new Nd(e,{joinaggregate:[{op:i,field:n,as:Ym(r,r.sort,{forAs:!0})}],groupby:[oa(r)]})}return t}return null}(t,e.facet)??t,m=new od(t,e,i,d.getSource()),n[i]=m}return{...e.component.data,outputNodes:n,outputNodeRefCounts:i,raw:u,main:d,facetRoot:m,ancestorParse:o}}class Km extends Sm{constructor(e,t,n,i){super(e,\"concat\",t,n,i,e.resolve),qn(this,\"children\",void 0),\"shared\"!==e.resolve?.axis?.x&&\"shared\"!==e.resolve?.axis?.y||$i(\"Axes cannot be shared in concatenated or repeated views yet (https://github.com/vega/vega-lite/issues/2415).\"),this.children=this.getChildren(e).map(((e,t)=>wp(e,this,this.getName(`concat_${t}`),void 0,i)))}parseData(){this.component.data=Jm(this);for(const e of this.children)e.parseData()}parseSelections(){this.component.selection={};for(const e of this.children){e.parseSelections();for(const t of D(e.component.selection))this.component.selection[t]=e.component.selection[t]}}parseMarkGroup(){for(const e of this.children)e.parseMarkGroup()}parseAxesAndHeaders(){for(const e of this.children)e.parseAxesAndHeaders()}getChildren(e){return zs(e)?e.vconcat:Os(e)?e.hconcat:e.concat}parseLayoutSize(){!function(e){Hm(e);const t=1===e.layout.columns?\"width\":\"childWidth\",n=void 0===e.layout.columns?\"height\":\"childHeight\";Vm(e,t),Vm(e,n)}(this)}parseAxisGroup(){return null}assembleSelectionTopLevelSignals(e){return this.children.reduce(((e,t)=>t.assembleSelectionTopLevelSignals(e)),e)}assembleSignals(){return this.children.forEach((e=>e.assembleSignals())),[]}assembleLayoutSignals(){const e=wf(this);for(const t of this.children)e.push(...t.assembleLayoutSignals());return e}assembleSelectionData(e){return this.children.reduce(((e,t)=>t.assembleSelectionData(e)),e)}assembleMarks(){return this.children.map((e=>{const t=e.assembleTitle(),n=e.assembleGroupStyle(),i=e.assembleGroupEncodeEntry(!1);return{type:\"group\",name:e.getName(\"group\"),...t?{title:t}:{},...n?{style:n}:{},...i?{encode:{update:i}}:{},...e.assembleGroup()}}))}assembleGroupStyle(){}assembleDefaultLayout(){const e=this.layout.columns;return{...null!=e?{columns:e}:{},bounds:\"full\",align:\"each\"}}}const Zm={disable:1,gridScale:1,scale:1,..._a,labelExpr:1,encode:1},ep=D(Zm);class tp extends Jl{constructor(){let e=arguments.length>0&&void 0!==arguments[0]?arguments[0]:{},t=arguments.length>1&&void 0!==arguments[1]?arguments[1]:{},n=arguments.length>2&&void 0!==arguments[2]&&arguments[2];super(),this.explicit=e,this.implicit=t,this.mainExtracted=n}clone(){return new tp(l(this.explicit),l(this.implicit),this.mainExtracted)}hasAxisPart(e){return\"axis\"===e||(\"grid\"===e||\"title\"===e?!!this.get(e):!(!1===(t=this.get(e))||null===t));var t}hasOrientSignalRef(){return yn(this.explicit.orient)}}const np={bottom:\"top\",top:\"bottom\",left:\"right\",right:\"left\"};function ip(e,t){if(!e)return t.map((e=>e.clone()));{if(e.length!==t.length)return;const n=e.length;for(let i=0;i{switch(n){case\"title\":return Ln(e,t);case\"gridScale\":return{explicit:e.explicit,value:U(e.value,t.value)}}return tc(e,t,n,\"axis\")}));e.setWithExplicit(n,i)}return e}function op(e,t,n,i,r){if(\"disable\"===t)return void 0!==n;switch(n=n||{},t){case\"titleAngle\":case\"labelAngle\":return e===(yn(n.labelAngle)?n.labelAngle:H(n.labelAngle));case\"values\":return!!n.values;case\"encode\":return!!n.encoding||!!n.labelAngle;case\"title\":if(e===rf(i,r))return!0}return e===n[t]}const ap=new Set([\"grid\",\"translate\",\"format\",\"formatType\",\"orient\",\"labelExpr\",\"tickCount\",\"position\",\"tickMinStep\"]);function sp(e,t){let n=t.axis(e);const i=new tp,r=ga(t.encoding[e]),{mark:o,config:a}=t,s=n?.orient||a[\"x\"===e?\"axisX\":\"axisY\"]?.orient||a.axis?.orient||function(e){return\"x\"===e?\"bottom\":\"left\"}(e),l=t.getScaleComponent(e).get(\"type\"),c=function(e,t,n,i){const r=\"band\"===t?[\"axisDiscrete\",\"axisBand\"]:\"point\"===t?[\"axisDiscrete\",\"axisPoint\"]:hr(t)?[\"axisQuantitative\"]:\"time\"===t||\"utc\"===t?[\"axisTemporal\"]:[],o=\"x\"===e?\"axisX\":\"axisY\",a=yn(n)?\"axisOrient\":`axis${P(n)}`,s=[...r,...r.map((e=>o+e.substr(4)))],l=[\"axis\",a,o];return{vlOnlyAxisConfig:Qu(s,i,e,n),vgAxisConfig:Qu(l,i,e,n),axisConfigStyle:Ju([...l,...s],i)}}(e,l,s,t.config),u=void 0!==n?!n:Ku(\"disable\",a.style,n?.style,c).configValue;if(i.set(\"disable\",u,void 0!==n),u)return i;n=n||{};const f=function(e,t,n,i,r){const o=t?.labelAngle;if(void 0!==o)return yn(o)?o:H(o);{const{configValue:o}=Ku(\"labelAngle\",i,t?.style,r);return void 0!==o?H(o):n!==Z||!p([sr,or],e.type)||Ho(e)&&e.timeUnit?void 0:270}}(r,n,e,a.style,c),d=wo(n.formatType,r,l),m=$o(r,r.type,n.format,n.formatType,a,!0),g={fieldOrDatumDef:r,axis:n,channel:e,model:t,scaleType:l,orient:s,labelAngle:f,format:m,formatType:d,mark:o,config:a};for(const r of ep){const o=r in Zu?Zu[r](g):Ca(r)?n[r]:void 0,s=void 0!==o,l=op(o,r,n,t,e);if(s&&l)i.set(r,o,l);else{const{configValue:e,configFrom:t}=Ca(r)&&\"values\"!==r?Ku(r,a.style,n.style,c):{},u=void 0!==e;s&&!u?i.set(r,o,l):(\"vgAxisConfig\"!==t||ap.has(r)&&u||Fa(e)||yn(e))&&i.set(r,e,!1)}}const h=n.encoding??{},y=za.reduce(((n,r)=>{if(!i.hasAxisPart(r))return n;const o=zf(h[r]??{},t),a=\"labels\"===r?function(e,t,n){const{encoding:i,config:r}=e,o=ga(i[t])??ga(i[it(t)]),a=e.axis(t)||{},{format:s,formatType:l}=a;if(go(l))return{text:xo({fieldOrDatumDef:o,field:\"datum.value\",format:s,formatType:l,config:r}),...n};if(void 0===s&&void 0===l&&r.customFormatTypes){if(\"quantitative\"===Vo(o)){if(ta(o)&&\"normalize\"===o.stack&&r.normalizedNumberFormatType)return{text:xo({fieldOrDatumDef:o,field:\"datum.value\",format:r.normalizedNumberFormat,formatType:r.normalizedNumberFormatType,config:r}),...n};if(r.numberFormatType)return{text:xo({fieldOrDatumDef:o,field:\"datum.value\",format:r.numberFormat,formatType:r.numberFormatType,config:r}),...n}}if(\"temporal\"===Vo(o)&&r.timeFormatType&&Ho(o)&&!o.timeUnit)return{text:xo({fieldOrDatumDef:o,field:\"datum.value\",format:r.timeFormat,formatType:r.timeFormatType,config:r}),...n}}return n}(t,e,o):o;return void 0===a||S(a)||(n[r]={update:a}),n}),{});return S(y)||i.set(\"encode\",y,!!n.encoding||void 0!==n.labelAngle),i}function lp(e,t){const{config:n}=e;return{...du(e,{align:\"ignore\",baseline:\"ignore\",color:\"include\",size:\"include\",orient:\"ignore\",theta:\"ignore\"}),...Zc(\"x\",e,{defaultPos:\"mid\"}),...Zc(\"y\",e,{defaultPos:\"mid\"}),...Xc(\"size\",e),...Xc(\"angle\",e),...cp(e,n,t)}}function cp(e,t,n){return n?{shape:{value:n}}:Xc(\"shape\",e)}const up={vgMark:\"rule\",encodeEntry:e=>{const{markDef:t}=e,n=t.orient;return e.encoding.x||e.encoding.y||e.encoding.latitude||e.encoding.longitude?{...du(e,{align:\"ignore\",baseline:\"ignore\",color:\"include\",orient:\"ignore\",size:\"ignore\",theta:\"ignore\"}),...ru(\"x\",e,{defaultPos:\"horizontal\"===n?\"zeroOrMax\":\"mid\",defaultPos2:\"zeroOrMin\",range:\"vertical\"!==n}),...ru(\"y\",e,{defaultPos:\"vertical\"===n?\"zeroOrMax\":\"mid\",defaultPos2:\"zeroOrMin\",range:\"horizontal\"!==n}),...Xc(\"size\",e,{vgChannel:\"strokeWidth\"})}:{}}};function fp(e,t,n){if(void 0===Cn(\"align\",e,n))return\"center\"}function dp(e,t,n){if(void 0===Cn(\"baseline\",e,n))return\"middle\"}const mp={vgMark:\"rect\",encodeEntry:e=>{const{config:t,markDef:n}=e,i=n.orient,r=\"horizontal\"===i?\"width\":\"height\",o=\"horizontal\"===i?\"height\":\"width\";return{...du(e,{align:\"ignore\",baseline:\"ignore\",color:\"include\",orient:\"ignore\",size:\"ignore\",theta:\"ignore\"}),...Zc(\"x\",e,{defaultPos:\"mid\",vgChannel:\"xc\"}),...Zc(\"y\",e,{defaultPos:\"mid\",vgChannel:\"yc\"}),...Xc(\"size\",e,{defaultValue:pp(e),vgChannel:r}),[o]:Fn(Cn(\"thickness\",n,t))}}};function pp(e){const{config:n,markDef:i}=e,{orient:r}=i,o=\"horizontal\"===r?\"width\":\"height\",a=e.getScaleComponent(\"horizontal\"===r?\"x\":\"y\"),s=Cn(\"size\",i,n,{vgChannel:o})??n.tick.bandSize;if(void 0!==s)return s;{const e=a?a.get(\"range\"):void 0;if(e&&vn(e)&&t.isNumber(e.step))return 3*e.step/4;return 3*js(n.view,o)/4}}const gp={arc:{vgMark:\"arc\",encodeEntry:e=>({...du(e,{align:\"ignore\",baseline:\"ignore\",color:\"include\",size:\"ignore\",orient:\"ignore\",theta:\"ignore\"}),...Zc(\"x\",e,{defaultPos:\"mid\"}),...Zc(\"y\",e,{defaultPos:\"mid\"}),...su(e,\"radius\"),...su(e,\"theta\")})},area:{vgMark:\"area\",encodeEntry:e=>({...du(e,{align:\"ignore\",baseline:\"ignore\",color:\"include\",orient:\"include\",size:\"ignore\",theta:\"ignore\"}),...ru(\"x\",e,{defaultPos:\"zeroOrMin\",defaultPos2:\"zeroOrMin\",range:\"horizontal\"===e.markDef.orient}),...ru(\"y\",e,{defaultPos:\"zeroOrMin\",defaultPos2:\"zeroOrMin\",range:\"vertical\"===e.markDef.orient}),...gu(e)})},bar:{vgMark:\"rect\",encodeEntry:e=>({...du(e,{align:\"ignore\",baseline:\"ignore\",color:\"include\",orient:\"ignore\",size:\"ignore\",theta:\"ignore\"}),...su(e,\"x\"),...su(e,\"y\")})},circle:{vgMark:\"symbol\",encodeEntry:e=>lp(e,\"circle\")},geoshape:{vgMark:\"shape\",encodeEntry:e=>({...du(e,{align:\"ignore\",baseline:\"ignore\",color:\"include\",size:\"ignore\",orient:\"ignore\",theta:\"ignore\"})}),postEncodingTransform:e=>{const{encoding:t}=e,n=t.shape;return[{type:\"geoshape\",projection:e.projectionName(),...n&&Ho(n)&&n.type===lr?{field:oa(n,{expr:\"datum\"})}:{}}]}},image:{vgMark:\"image\",encodeEntry:e=>({...du(e,{align:\"ignore\",baseline:\"ignore\",color:\"ignore\",orient:\"ignore\",size:\"ignore\",theta:\"ignore\"}),...su(e,\"x\"),...su(e,\"y\"),...Rc(e,\"url\")})},line:{vgMark:\"line\",encodeEntry:e=>({...du(e,{align:\"ignore\",baseline:\"ignore\",color:\"include\",size:\"ignore\",orient:\"ignore\",theta:\"ignore\"}),...Zc(\"x\",e,{defaultPos:\"mid\"}),...Zc(\"y\",e,{defaultPos:\"mid\"}),...Xc(\"size\",e,{vgChannel:\"strokeWidth\"}),...gu(e)})},point:{vgMark:\"symbol\",encodeEntry:e=>lp(e)},rect:{vgMark:\"rect\",encodeEntry:e=>({...du(e,{align:\"ignore\",baseline:\"ignore\",color:\"include\",orient:\"ignore\",size:\"ignore\",theta:\"ignore\"}),...su(e,\"x\"),...su(e,\"y\")})},rule:up,square:{vgMark:\"symbol\",encodeEntry:e=>lp(e,\"square\")},text:{vgMark:\"text\",encodeEntry:e=>{const{config:t,encoding:n}=e;return{...du(e,{align:\"include\",baseline:\"include\",color:\"include\",size:\"ignore\",orient:\"ignore\",theta:\"include\"}),...Zc(\"x\",e,{defaultPos:\"mid\"}),...Zc(\"y\",e,{defaultPos:\"mid\"}),...Rc(e),...Xc(\"size\",e,{vgChannel:\"fontSize\"}),...Xc(\"angle\",e),...hu(\"align\",fp(e.markDef,n,t)),...hu(\"baseline\",dp(e.markDef,n,t)),...Zc(\"radius\",e,{defaultPos:null}),...Zc(\"theta\",e,{defaultPos:null})}}},tick:mp,trail:{vgMark:\"trail\",encodeEntry:e=>({...du(e,{align:\"ignore\",baseline:\"ignore\",color:\"include\",size:\"include\",orient:\"ignore\",theta:\"ignore\"}),...Zc(\"x\",e,{defaultPos:\"mid\"}),...Zc(\"y\",e,{defaultPos:\"mid\"}),...Xc(\"size\",e),...gu(e)})}};function hp(e){if(p([Ur,Mr,Vr],e.mark)){const t=Ba(e.mark,e.encoding);if(t.length>0)return function(e,t){return[{name:e.getName(\"pathgroup\"),type:\"group\",from:{facet:{name:yp+e.requestDataName(fc.Main),data:e.requestDataName(fc.Main),groupby:t}},encode:{update:{width:{field:{group:\"width\"}},height:{field:{group:\"height\"}}}},marks:bp(e,{fromPrefix:yp})}]}(e,t)}else if(e.mark===Lr){const t=wn.some((t=>Cn(t,e.markDef,e.config)));if(e.stack&&!e.fieldDef(\"size\")&&t)return function(e){const[t]=bp(e,{fromPrefix:vp}),n=e.scaleName(e.stack.fieldChannel),i=function(){let t=arguments.length>0&&void 0!==arguments[0]?arguments[0]:{};return e.vgField(e.stack.fieldChannel,t)},r=(e,t)=>`${e}(${[i({prefix:\"min\",suffix:\"start\",expr:t}),i({prefix:\"max\",suffix:\"start\",expr:t}),i({prefix:\"min\",suffix:\"end\",expr:t}),i({prefix:\"max\",suffix:\"end\",expr:t})].map((e=>`scale('${n}',${e})`)).join(\",\")})`;let o,a;\"x\"===e.stack.fieldChannel?(o={...u(t.encode.update,[\"y\",\"yc\",\"y2\",\"height\",...wn]),x:{signal:r(\"min\",\"datum\")},x2:{signal:r(\"max\",\"datum\")},clip:{value:!0}},a={x:{field:{group:\"x\"},mult:-1},height:{field:{group:\"height\"}}},t.encode.update={...f(t.encode.update,[\"y\",\"yc\",\"y2\"]),height:{field:{group:\"height\"}}}):(o={...u(t.encode.update,[\"x\",\"xc\",\"x2\",\"width\"]),y:{signal:r(\"min\",\"datum\")},y2:{signal:r(\"max\",\"datum\")},clip:{value:!0}},a={y:{field:{group:\"y\"},mult:-1},width:{field:{group:\"width\"}}},t.encode.update={...f(t.encode.update,[\"x\",\"xc\",\"x2\"]),width:{field:{group:\"width\"}}});for(const n of wn){const i=Pn(n,e.markDef,e.config);t.encode.update[n]?(o[n]=t.encode.update[n],delete t.encode.update[n]):i&&(o[n]=Fn(i)),i&&(t.encode.update[n]={value:0})}const s=[];if(e.stack.groupbyChannels?.length>0)for(const t of e.stack.groupbyChannels){const n=e.fieldDef(t),i=oa(n);i&&s.push(i),(n?.bin||n?.timeUnit)&&s.push(oa(n,{binSuffix:\"end\"}))}o=[\"stroke\",\"strokeWidth\",\"strokeJoin\",\"strokeCap\",\"strokeDash\",\"strokeDashOffset\",\"strokeMiterLimit\",\"strokeOpacity\"].reduce(((n,i)=>{if(t.encode.update[i])return{...n,[i]:t.encode.update[i]};{const t=Pn(i,e.markDef,e.config);return void 0!==t?{...n,[i]:Fn(t)}:n}}),o),o.stroke&&(o.strokeForeground={value:!0},o.strokeOffset={value:0});return[{type:\"group\",from:{facet:{data:e.requestDataName(fc.Main),name:vp+e.requestDataName(fc.Main),groupby:s,aggregate:{fields:[i({suffix:\"start\"}),i({suffix:\"start\"}),i({suffix:\"end\"}),i({suffix:\"end\"})],ops:[\"min\",\"max\",\"min\",\"max\"]}}},encode:{update:o},marks:[{type:\"group\",encode:{update:a},marks:[t]}]}]}(e)}return bp(e)}const yp=\"faceted_path_\";const vp=\"stack_group_\";function bp(e){let n=arguments.length>1&&void 0!==arguments[1]?arguments[1]:{fromPrefix:\"\"};const{mark:i,markDef:r,encoding:o,config:a}=e,s=U(r.clip,function(e){const t=e.getScaleComponent(\"x\"),n=e.getScaleComponent(\"y\");return!(!t?.get(\"selectionExtent\")&&!n?.get(\"selectionExtent\"))||void 0}(e),function(e){const t=e.component.projection;return!(!t||t.isFit)||void 0}(e)),l=Nn(r),c=o.key,u=function(e){const{encoding:n,stack:i,mark:r,markDef:o,config:a}=e,s=n.order;if(!(!t.isArray(s)&&Zo(s)&&m(s.value)||!s&&m(Cn(\"order\",o,a)))){if((t.isArray(s)||Ho(s))&&!i)return Tn(s,{expr:\"datum\"});if(Qr(r)){const i=\"horizontal\"===o.orient?\"y\":\"x\",r=n[i];if(Ho(r)){const n=r.sort;return t.isArray(n)?{field:oa(r,{prefix:i,suffix:\"sort_index\",expr:\"datum\"})}:Co(n)?{field:oa({aggregate:La(e.encoding)?n.op:void 0,field:n.field},{expr:\"datum\"})}:No(n)?{field:oa(e.fieldDef(n.encoding),{expr:\"datum\"}),order:n.order}:null===n?void 0:{field:oa(r,{binSuffix:e.stack?.impute?\"mid\":void 0,expr:\"datum\"})}}}}}(e),f=function(e){if(!e.component.selection)return null;const t=D(e.component.selection).length;let n=t,i=e.parent;for(;i&&0===n;)n=D(i.component.selection).length,i=i.parent;return n?{interactive:t>0||\"geoshape\"===e.mark||!!e.encoding.tooltip}:null}(e),d=Cn(\"aria\",r,a),p=gp[i].postEncodingTransform?gp[i].postEncodingTransform(e):null;return[{name:e.getName(\"marks\"),type:gp[i].vgMark,...s?{clip:!0}:{},...l?{style:l}:{},...c?{key:c.field}:{},...u?{sort:u}:{},...f||{},...!1===d?{aria:d}:{},from:{data:n.fromPrefix+e.requestDataName(fc.Main)},encode:{update:gp[i].encodeEntry(e)},...p?{transform:p}:{}}]}class xp extends Dm{constructor(e,n,i){let r=arguments.length>3&&void 0!==arguments[3]?arguments[3]:{},o=arguments.length>4?arguments[4]:void 0;super(e,\"unit\",n,i,o,void 0,Cs(e)?e.view:void 0),qn(this,\"markDef\",void 0),qn(this,\"encoding\",void 0),qn(this,\"specifiedScales\",{}),qn(this,\"stack\",void 0),qn(this,\"specifiedAxes\",{}),qn(this,\"specifiedLegends\",{}),qn(this,\"specifiedProjection\",{}),qn(this,\"selection\",[]),qn(this,\"children\",[]);const a=Zr(e.mark)?{...e.mark}:{type:e.mark},s=a.type;void 0===a.filled&&(a.filled=function(e,t,n){let{graticule:i}=n;if(i)return!1;const r=Pn(\"filled\",e,t),o=e.type;return U(r,o!==Rr&&o!==Ur&&o!==Br)}(a,o,{graticule:e.data&&uc(e.data)}));const l=this.encoding=function(e,n,i,r){const o={};for(const t of D(e))Ke(t)||$i(`${a=t}-encoding is dropped as ${a} is not a valid encoding channel.`);var a;for(let a of lt){if(!e[a])continue;const s=e[a];if(Pt(a)){const e=st(a),t=o[e];if(Ho(t)&&nr(t.type)&&Ho(s)&&!t.timeUnit){$i(ni(e));continue}}if(\"angle\"!==a||\"arc\"!==n||e.theta||($i(\"Arc marks uses theta channel rather than angle, replacing angle with theta.\"),a=se),Ua(e,a,n)){if(a===ye&&\"line\"===n){const t=pa(e[a]);if(t?.aggregate){$i(\"Line marks cannot encode size with a non-groupby field. You may want to use trail marks instead.\");continue}}if(a===me&&(i?\"fill\"in e:\"stroke\"in e))$i(ri(\"encoding\",{fill:\"fill\"in e,stroke:\"stroke\"in e}));else if(a===Fe||a===De&&!t.isArray(s)&&!Zo(s)||a===Oe&&t.isArray(s)){if(s){if(a===De){const t=e[a];if(Ro(t)){o[a]=t;continue}}o[a]=t.array(s).reduce(((e,t)=>(Ho(t)?e.push(va(t,a)):$i(oi(t,a)),e)),[])}}else{if(a===Oe&&null===s)o[a]=null;else if(!(Ho(s)||Go(s)||Zo(s)||Wo(s)||yn(s))){$i(oi(s,a));continue}o[a]=ha(s,a,r)}}else $i(ai(a,n))}return o}(e.encoding||{},s,a.filled,o);this.markDef=il(a,l,o),this.size=function(e){let{encoding:t,size:n}=e;for(const e of Ft){const i=rt(e);Ns(n[i])&&Yo(t[e])&&(delete n[i],$i(pi(i)))}return n}({encoding:l,size:Cs(e)?{...r,...e.width?{width:e.width}:{},...e.height?{height:e.height}:{}}:r}),this.stack=nl(this.markDef,l),this.specifiedScales=this.initScales(s,l),this.specifiedAxes=this.initAxes(l),this.specifiedLegends=this.initLegends(l),this.specifiedProjection=e.projection,this.selection=(e.params??[]).filter((e=>Ss(e)))}get hasProjection(){const{encoding:e}=this,t=this.mark===Xr,n=e&&Me.some((t=>Jo(e[t])));return t||n}scaleDomain(e){const t=this.specifiedScales[e];return t?t.domain:void 0}axis(e){return this.specifiedAxes[e]}legend(e){return this.specifiedLegends[e]}initScales(e,t){return It.reduce(((e,n)=>{const i=ga(t[n]);return i&&(e[n]=this.initScale(i.scale??{})),e}),{})}initScale(e){const{domain:n,range:i}=e,r=pn(e);return t.isArray(n)&&(r.domain=n.map(Sn)),t.isArray(i)&&(r.range=i.map(Sn)),r}initAxes(e){return Ft.reduce(((t,n)=>{const i=e[n];if(Jo(i)||n===Z&&Jo(e.x2)||n===ee&&Jo(e.y2)){const e=Jo(i)?i.axis:void 0;t[n]=e?this.initAxis({...e}):e}return t}),{})}initAxis(e){const t=D(e),n={};for(const i of t){const t=e[i];n[i]=Fa(t)?kn(t):Sn(t)}return n}initLegends(e){return Wt.reduce(((t,n)=>{const i=ga(e[n]);if(i&&function(e){switch(e){case me:case pe:case ge:case ye:case he:case be:case we:case ke:return!0;case xe:case $e:case ve:return!1}}(n)){const e=i.legend;t[n]=e?pn(e):e}return t}),{})}parseData(){this.component.data=Jm(this)}parseLayoutSize(){!function(e){const{size:t,component:n}=e;for(const i of Ft){const r=rt(i);if(t[r]){const e=t[r];n.layoutSize.set(r,Ns(e)?\"step\":e,!0)}else{const t=Gm(e,r);n.layoutSize.set(r,t,!1)}}}(this)}parseSelections(){this.component.selection=function(e,n){const i={},r=e.config.selection;if(!n||!n.length)return i;for(const o of n){const n=_(o.name),a=o.select,s=t.isString(a)?a:a.type,c=t.isObject(a)?l(a):{type:s},u=r[s];for(const e in u)\"fields\"!==e&&\"encodings\"!==e&&(\"mark\"===e&&(c[e]={...u[e],...c[e]}),void 0!==c[e]&&!0!==c[e]||(c[e]=l(u[e]??c[e])));const f=i[n]={...c,name:n,type:s,init:o.value,bind:o.bind,events:t.isString(c.on)?t.parseSelector(c.on,\"scope\"):t.array(l(c.on))},d=l(o);for(const t of Eu)t.defined(f)&&t.parse&&t.parse(e,f,d)}return i}(this,this.selection)}parseMarkGroup(){this.component.mark=hp(this)}parseAxesAndHeaders(){var e;this.component.axes=(e=this,Ft.reduce(((t,n)=>(e.component.scales[n]&&(t[n]=[sp(n,e)]),t)),{}))}assembleSelectionTopLevelSignals(e){return function(e,n){let i=!1;for(const r of F(e.component.selection??{})){const o=r.name,a=t.stringValue(o+Pu);if(0===n.filter((e=>e.name===o)).length){const e=\"global\"===r.resolve?\"union\":r.resolve,i=\"point\"===r.type?\", true, true)\":\")\";n.push({name:r.name,update:`${Tu}(${a}, ${t.stringValue(e)}${i}`})}i=!0;for(const t of Eu)t.defined(r)&&t.topLevelSignals&&(n=t.topLevelSignals(e,r,n))}i&&0===n.filter((e=>\"unit\"===e.name)).length&&n.unshift({name:\"unit\",value:{},on:[{events:\"pointermove\",update:\"isTuple(group()) ? group() : unit\"}]});return yc(n)}(this,e)}assembleSignals(){return[...Xu(this),...pc(this,[])]}assembleSelectionData(e){return function(e,t){const n=[...t],i=Mu(e,{escape:!1});for(const t of F(e.component.selection??{})){const e={name:t.name+Pu};if(t.project.hasSelectionId&&(e.transform=[{type:\"collect\",sort:{field:xs}}]),t.init){const n=t.project.items.map(dc);e.values=t.project.hasSelectionId?t.init.map((e=>({unit:i,[xs]:mc(e,!1)[0]}))):t.init.map((e=>({unit:i,fields:n,values:mc(e,!1)})))}n.filter((e=>e.name===t.name+Pu)).length||n.push(e)}return n}(this,e)}assembleLayout(){return null}assembleLayoutSignals(){return wf(this)}assembleMarks(){let e=this.component.mark??[];return this.parent&&km(this.parent)||(e=hc(this,e)),e.map(this.correctDataNames)}assembleGroupStyle(){const{style:e}=this.view||{};return void 0!==e?e:this.encoding.x||this.encoding.y?\"cell\":\"view\"}getMapping(){return this.encoding}get mark(){return this.markDef.type}channelHasField(e){return Ta(this.encoding,e)}fieldDef(e){return pa(this.encoding[e])}typedFieldDef(e){const t=this.fieldDef(e);return Ko(t)?t:null}}class $p extends Sm{constructor(e,t,n,i,r){super(e,\"layer\",t,n,r,e.resolve,e.view),qn(this,\"children\",void 0);const o={...i,...e.width?{width:e.width}:{},...e.height?{height:e.height}:{}};this.children=e.layer.map(((e,t)=>{if(Xs(e))return new $p(e,this,this.getName(`layer_${t}`),o,r);if(Aa(e))return new xp(e,this,this.getName(`layer_${t}`),o,r);throw new Error(Bn(e))}))}parseData(){this.component.data=Jm(this);for(const e of this.children)e.parseData()}parseLayoutSize(){var e;Hm(e=this),Vm(e,\"width\"),Vm(e,\"height\")}parseSelections(){this.component.selection={};for(const e of this.children){e.parseSelections();for(const t of D(e.component.selection))this.component.selection[t]=e.component.selection[t]}}parseMarkGroup(){for(const e of this.children)e.parseMarkGroup()}parseAxesAndHeaders(){!function(e){const{axes:t,resolve:n}=e.component,i={top:0,bottom:0,right:0,left:0};for(const i of e.children){i.parseAxesAndHeaders();for(const r of D(i.component.axes))n.axis[r]=_f(e.component.resolve,r),\"shared\"===n.axis[r]&&(t[r]=ip(t[r],i.component.axes[r]),t[r]||(n.axis[r]=\"independent\",delete t[r]))}for(const r of Ft){for(const o of e.children)if(o.component.axes[r]){if(\"independent\"===n.axis[r]){t[r]=(t[r]??[]).concat(o.component.axes[r]);for(const e of o.component.axes[r]){const{value:t,explicit:n}=e.getWithExplicit(\"orient\");if(!yn(t)){if(i[t]>0&&!n){const n=np[t];i[t]>i[n]&&e.set(\"orient\",n,!1)}i[t]++}}}delete o.component.axes[r]}if(\"independent\"===n.axis[r]&&t[r]&&t[r].length>1)for(const[e,n]of(t[r]||[]).entries())e>0&&n.get(\"grid\")&&!n.explicit.grid&&(n.implicit.grid=!1)}}(this)}assembleSelectionTopLevelSignals(e){return this.children.reduce(((e,t)=>t.assembleSelectionTopLevelSignals(e)),e)}assembleSignals(){return this.children.reduce(((e,t)=>e.concat(t.assembleSignals())),Xu(this))}assembleLayoutSignals(){return this.children.reduce(((e,t)=>e.concat(t.assembleLayoutSignals())),wf(this))}assembleSelectionData(e){return this.children.reduce(((e,t)=>t.assembleSelectionData(e)),e)}assembleGroupStyle(){const e=new Set;for(const n of this.children)for(const i of t.array(n.assembleGroupStyle()))e.add(i);const n=Array.from(e);return n.length>1?n:1===n.length?n[0]:void 0}assembleTitle(){let e=super.assembleTitle();if(e)return e;for(const t of this.children)if(e=t.assembleTitle(),e)return e}assembleLayout(){return null}assembleMarks(){return function(e,t){for(const n of e.children)xm(n)&&(t=hc(n,t));return t}(this,this.children.flatMap((e=>e.assembleMarks())))}assembleLegends(){return this.children.reduce(((e,t)=>e.concat(t.assembleLegends())),Vf(this))}}function wp(e,t,n,i,r){if(To(e))return new Xm(e,t,n,r);if(Xs(e))return new $p(e,t,n,i,r);if(Aa(e))return new xp(e,t,n,i,r);if(function(e){return zs(e)||Os(e)||Fs(e)}(e))return new Km(e,t,n,r);throw new Error(Bn(e))}const kp=n;e.accessPathDepth=q,e.accessPathWithDatum=A,e.compile=function(e){let n=arguments.length>1&&void 0!==arguments[1]?arguments[1]:{};var i;n.logger&&(i=n.logger,xi=i),n.fieldTitle&&ca(n.fieldTitle);try{const i=Bs(t.mergeConfig(n.config,e.config)),r=Il(e,i),o=wp(r,null,\"\",void 0,i);o.parse(),function(e,t){Md(e.sources);let n=0,i=0;for(let i=0;i2&&void 0!==arguments[2]?arguments[2]:{},i=arguments.length>3?arguments[3]:void 0;const r=e.config?Gs(e.config):void 0,o=[].concat(e.assembleSelectionData([]),function(e,t){const n=[],i=Um(n);let r=0;for(const t of e.sources){t.hasName()||(t.dataName=\"source_\"+r++);const e=t.assemble();i(t,e)}for(const e of n)0===e.transform.length&&delete e.transform;let o=0;for(const[e,t]of n.entries())0!==(t.transform??[]).length||t.source||n.splice(o++,0,n.splice(e,1)[0]);for(const t of n)for(const n of t.transform??[])\"lookup\"===n.type&&(n.from=e.outputNodes[n.from].getSource());for(const e of n)e.name in t&&(e.values=t[e.name]);return n}(e.component.data,n)),a=e.assembleProjections(),s=e.assembleTitle(),l=e.assembleGroupStyle(),c=e.assembleGroupEncodeEntry(!0);let u=e.assembleLayoutSignals();u=u.filter((e=>\"width\"!==e.name&&\"height\"!==e.name||void 0===e.value||(t[e.name]=+e.value,!1)));const{params:f,...d}=t;return{$schema:\"https://vega.github.io/schema/vega/v5.json\",...e.description?{description:e.description}:{},...d,...s?{title:s}:{},...l?{style:l}:{},...c?{encode:{update:c}}:{},data:o,...a.length>0?{projections:a}:{},...e.assembleGroup([...u,...e.assembleSelectionTopLevelSignals([]),...Ds(f)]),...r?{config:r}:{},...i?{usermeta:i}:{}}}(o,function(e,n,i,r){const o=r.component.layoutSize.get(\"width\"),a=r.component.layoutSize.get(\"height\");void 0===n?(n={type:\"pad\"},r.hasAxisOrientSignalRef()&&(n.resize=!0)):t.isString(n)&&(n={type:n});if(o&&a&&(s=n.type,\"fit\"===s||\"fit-x\"===s||\"fit-y\"===s))if(\"step\"===o&&\"step\"===a)$i(Gn()),n.type=\"pad\";else if(\"step\"===o||\"step\"===a){const e=\"step\"===o?\"width\":\"height\";$i(Gn(Nt(e)));const t=\"width\"===e?\"height\":\"width\";n.type=function(e){return e?`fit-${Nt(e)}`:\"fit\"}(t)}var s;return{...1===D(n).length&&n.type?\"pad\"===n.type?{}:{autosize:n.type}:{autosize:n},...Ql(i,!1),...Ql(e,!0)}}(e,r.autosize,i,o),e.datasets,e.usermeta);return{spec:a,normalized:r}}finally{n.logger&&(xi=bi),n.fieldTitle&&ca(sa)}},e.contains=p,e.deepEqual=Y,e.deleteNestedProperty=C,e.duplicate=l,e.entries=z,e.every=h,e.fieldIntersection=k,e.flatAccessWithDatum=j,e.getFirstDefined=U,e.hasIntersection=$,e.hash=d,e.internalField=B,e.isBoolean=O,e.isEmpty=S,e.isEqual=function(e,t){const n=D(e),i=D(t);if(n.length!==i.length)return!1;for(const i of n)if(e[i]!==t[i])return!1;return!0},e.isInternalField=I,e.isNullOrFalse=m,e.isNumeric=V,e.keys=D,e.logicalExpr=N,e.mergeDeep=y,e.never=c,e.normalize=Il,e.normalizeAngle=H,e.omit=f,e.pick=u,e.prefixGenerator=w,e.removePathFromField=L,e.replaceAll=M,e.replacePathInField=E,e.resetIdCounter=function(){R=42},e.setEqual=x,e.some=g,e.stringify=X,e.titleCase=P,e.unique=b,e.uniqueId=W,e.vals=F,e.varName=_,e.version=kp}));\n//# sourceMappingURL=vega-lite.min.js.map\n" }, { "path": "docs/_static/js/vega@5.js", "content": "!function(t,e){\"object\"==typeof exports&&\"undefined\"!=typeof module?e(exports):\"function\"==typeof define&&define.amd?define([\"exports\"],e):e((t=\"undefined\"!=typeof globalThis?globalThis:t||self).vega={})}(this,(function(t){\"use strict\";function e(t,e,n){return t.fields=e||[],t.fname=n,t}function n(t){return null==t?null:t.fname}function r(t){return null==t?null:t.fields}function i(t){return 1===t.length?o(t[0]):a(t)}const o=t=>function(e){return e[t]},a=t=>{const e=t.length;return function(n){for(let r=0;rr&&c(),u=r=i+1):\"]\"===o&&(u||s(\"Access path missing open bracket: \"+t),u>0&&c(),u=0,r=i+1):i>r?c():r=i+1}return u&&s(\"Access path missing closing bracket: \"+t),a&&s(\"Access path missing closing quote: \"+t),i>r&&(i++,c()),e}function l(t,n,r){const o=u(t);return t=1===o.length?o[0]:t,e((r&&r.get||i)(o),[t],n||t)}const c=l(\"id\"),f=e((t=>t),[],\"identity\"),h=e((()=>0),[],\"zero\"),d=e((()=>1),[],\"one\"),p=e((()=>!0),[],\"true\"),g=e((()=>!1),[],\"false\");function m(t,e,n){const r=[e].concat([].slice.call(n));console[t].apply(console,r)}const y=0,v=1,_=2,x=3,b=4;function w(t,e){let n=arguments.length>2&&void 0!==arguments[2]?arguments[2]:m,r=t||y;return{level(t){return arguments.length?(r=+t,this):r},error(){return r>=v&&n(e||\"error\",\"ERROR\",arguments),this},warn(){return r>=_&&n(e||\"warn\",\"WARN\",arguments),this},info(){return r>=x&&n(e||\"log\",\"INFO\",arguments),this},debug(){return r>=b&&n(e||\"log\",\"DEBUG\",arguments),this}}}var k=Array.isArray;function A(t){return t===Object(t)}const M=t=>\"__proto__\"!==t;function E(){for(var t=arguments.length,e=new Array(t),n=0;n{for(const n in e)if(\"signals\"===n)t.signals=C(t.signals,e.signals);else{const r=\"legend\"===n?{layout:1}:\"style\"===n||null;D(t,n,e[n],r)}return t}),{})}function D(t,e,n,r){if(!M(e))return;let i,o;if(A(n)&&!k(n))for(i in o=A(t[e])?t[e]:t[e]={},n)r&&(!0===r||r[i])?D(o,i,n[i]):M(i)&&(o[i]=n[i]);else t[e]=n}function C(t,e){if(null==t)return e;const n={},r=[];function i(t){n[t.name]||(n[t.name]=1,r.push(t))}return e.forEach(i),t.forEach(i),r}function F(t){return t[t.length-1]}function S(t){return null==t||\"\"===t?null:+t}const $=t=>e=>t*Math.exp(e),T=t=>e=>Math.log(t*e),B=t=>e=>Math.sign(e)*Math.log1p(Math.abs(e/t)),z=t=>e=>Math.sign(e)*Math.expm1(Math.abs(e))*t,N=t=>e=>e<0?-Math.pow(-e,t):Math.pow(e,t);function O(t,e,n,r){const i=n(t[0]),o=n(F(t)),a=(o-i)*e;return[r(i-a),r(o-a)]}function R(t,e){return O(t,e,S,f)}function U(t,e){var n=Math.sign(t[0]);return O(t,e,T(n),$(n))}function L(t,e,n){return O(t,e,N(n),N(1/n))}function q(t,e,n){return O(t,e,B(n),z(n))}function P(t,e,n,r,i){const o=r(t[0]),a=r(F(t)),s=null!=e?r(e):(o+a)/2;return[i(s+(o-s)*n),i(s+(a-s)*n)]}function j(t,e,n){return P(t,e,n,S,f)}function I(t,e,n){const r=Math.sign(t[0]);return P(t,e,n,T(r),$(r))}function W(t,e,n,r){return P(t,e,n,N(r),N(1/r))}function H(t,e,n,r){return P(t,e,n,B(r),z(r))}function Y(t){return 1+~~(new Date(t).getMonth()/3)}function G(t){return 1+~~(new Date(t).getUTCMonth()/3)}function V(t){return null!=t?k(t)?t:[t]:[]}function X(t,e,n){let r,i=t[0],o=t[1];return o=n-e?[e,n]:[i=Math.min(Math.max(i,e),n-r),i+r]}function J(t){return\"function\"==typeof t}const Z=\"descending\";function Q(t,n,i){i=i||{},n=V(n)||[];const o=[],a=[],s={},u=i.comparator||tt;return V(t).forEach(((t,e)=>{null!=t&&(o.push(n[e]===Z?-1:1),a.push(t=J(t)?t:l(t,null,i)),(r(t)||[]).forEach((t=>s[t]=1)))})),0===a.length?null:e(u(a,o),Object.keys(s))}const K=(t,e)=>(te||null==e)&&null!=t?1:(e=e instanceof Date?+e:e,(t=t instanceof Date?+t:t)!==t&&e==e?-1:e!=e&&t==t?1:0),tt=(t,e)=>1===t.length?et(t[0],e[0]):nt(t,e,t.length),et=(t,e)=>function(n,r){return K(t(n),t(r))*e},nt=(t,e,n)=>(e.push(0),function(r,i){let o,a=0,s=-1;for(;0===a&&++st}function it(t,e){let n;return r=>{n&&clearTimeout(n),n=setTimeout((()=>(e(r),n=null)),t)}}function ot(t){for(let e,n,r=1,i=arguments.length;ro&&(o=r))}else{for(r=e(t[a]);ao&&(o=r))}return[i,o]}function st(t,e){const n=t.length;let r,i,o,a,s,u=-1;if(null==e){for(;++u=i){r=o=i;break}if(u===n)return[-1,-1];for(a=s=u;++ui&&(r=i,a=u),o=i){r=o=i;break}if(u===n)return[-1,-1];for(a=s=u;++ui&&(r=i,a=u),or(t)?n[t]:void 0,set(t,e){return r(t)||(++i.size,n[t]===ct&&--i.empty),n[t]=e,this},delete(t){return r(t)&&(--i.size,++i.empty,n[t]=ct),this},clear(){i.size=i.empty=0,i.object=n={}},test(t){return arguments.length?(e=t,i):e},clean(){const t={};let r=0;for(const i in n){const o=n[i];o===ct||e&&e(o)||(t[i]=o,++r)}i.size=r,i.empty=0,i.object=n=t}};return t&&Object.keys(t).forEach((e=>{i.set(e,t[e])})),i}function ht(t,e,n,r,i,o){if(!n&&0!==n)return o;const a=+n;let s,u=t[0],l=F(t);la&&(i=o,o=a,a=i),r=void 0===r||r,((n=void 0===n||n)?o<=t:ot.replace(/\\\\(.)/g,\"$1\"))):V(t));const o=t&&t.length,a=r&&r.get||i,s=t=>a(n?[t]:u(t));let l;if(o)if(1===o){const e=s(t[0]);l=function(t){return\"\"+e(t)}}else{const e=t.map(s);l=function(t){let n=\"\"+e[0](t),r=0;for(;++r{e={},n={},r=0},o=(i,o)=>(++r>t&&(n=e,e={},r=1),e[i]=o);return i(),{clear:i,has:t=>lt(e,t)||lt(n,t),get:t=>lt(e,t)?e[t]:lt(n,t)?o(t,n[t]):void 0,set:(t,n)=>lt(e,t)?e[t]=n:o(t,n)}}function At(t,e,n,r){const i=e.length,o=n.length;if(!o)return e;if(!i)return n;const a=r||new e.constructor(i+o);let s=0,u=0,l=0;for(;s0?n[u++]:e[s++];for(;s=0;)n+=t;return n}function Et(t,e,n,r){const i=n||\" \",o=t+\"\",a=e-o.length;return a<=0?o:\"left\"===r?Mt(i,a)+o:\"center\"===r?Mt(i,~~(a/2))+o+Mt(i,Math.ceil(a/2)):o+Mt(i,a)}function Dt(t){return t&&F(t)-t[0]||0}function Ct(t){return k(t)?\"[\"+t.map(Ct)+\"]\":A(t)||xt(t)?JSON.stringify(t).replace(\"\\u2028\",\"\\\\u2028\").replace(\"\\u2029\",\"\\\\u2029\"):t}function Ft(t){return null==t||\"\"===t?null:!(!t||\"false\"===t||\"0\"===t)&&!!t}const St=t=>vt(t)||mt(t)?t:Date.parse(t);function $t(t,e){return e=e||St,null==t||\"\"===t?null:e(t)}function Tt(t){return null==t||\"\"===t?null:t+\"\"}function Bt(t){const e={},n=t.length;for(let r=0;r9999?\"+\"+It(e,6):It(e,4))+\"-\"+It(t.getUTCMonth()+1,2)+\"-\"+It(t.getUTCDate(),2)+(o?\"T\"+It(n,2)+\":\"+It(r,2)+\":\"+It(i,2)+\".\"+It(o,3)+\"Z\":i?\"T\"+It(n,2)+\":\"+It(r,2)+\":\"+It(i,2)+\"Z\":r||n?\"T\"+It(n,2)+\":\"+It(r,2)+\"Z\":\"\")}function Ht(t){var e=new RegExp('[\"'+t+\"\\n\\r]\"),n=t.charCodeAt(0);function r(t,e){var r,i=[],o=t.length,a=0,s=0,u=o<=0,l=!1;function c(){if(u)return Rt;if(l)return l=!1,Ot;var e,r,i=a;if(t.charCodeAt(i)===Ut){for(;a++=o?u=!0:(r=t.charCodeAt(a++))===Lt?l=!0:r===qt&&(l=!0,t.charCodeAt(a)===Lt&&++a),t.slice(i+1,e-1).replace(/\"\"/g,'\"')}for(;a1)r=function(t,e,n){var r,i=[],o=[];function a(t){var e=t<0?~t:t;(o[e]||(o[e]=[])).push({i:t,g:r})}function s(t){t.forEach(a)}function u(t){t.forEach(s)}function l(t){t.forEach(u)}function c(t){switch(r=t,t.type){case\"GeometryCollection\":t.geometries.forEach(c);break;case\"LineString\":s(t.arcs);break;case\"MultiLineString\":case\"Polygon\":u(t.arcs);break;case\"MultiPolygon\":l(t.arcs)}}return c(e),o.forEach(null==n?function(t){i.push(t[0].i)}:function(t){n(t[0].g,t[t.length-1].g)&&i.push(t[0].i)}),i}(0,e,n);else for(i=0,r=new Array(o=t.arcs.length);ie?1:t>=e?0:NaN}function te(t,e){return null==t||null==e?NaN:et?1:e>=t?0:NaN}function ee(t){let e,n,r;function i(t,r){let i=arguments.length>2&&void 0!==arguments[2]?arguments[2]:0,o=arguments.length>3&&void 0!==arguments[3]?arguments[3]:t.length;if(i>>1;n(t[e],r)<0?i=e+1:o=e}while(iKt(t(e),n),r=(e,n)=>t(e)-n):(e=t===Kt||t===te?t:ne,n=t,r=t),{left:i,center:function(t,e){let n=arguments.length>2&&void 0!==arguments[2]?arguments[2]:0;const o=i(t,e,n,(arguments.length>3&&void 0!==arguments[3]?arguments[3]:t.length)-1);return o>n&&r(t[o-1],e)>-r(t[o],e)?o-1:o},right:function(t,r){let i=arguments.length>2&&void 0!==arguments[2]?arguments[2]:0,o=arguments.length>3&&void 0!==arguments[3]?arguments[3]:t.length;if(i>>1;n(t[e],r)<=0?i=e+1:o=e}while(i0){for(o=t[--i];i>0&&(e=o,n=t[--i],o=e+n,r=n-(o-e),!r););i>0&&(r<0&&t[i-1]<0||r>0&&t[i-1]>0)&&(n=2*r,e=o+n,n==e-o&&(o=e))}return o}}class ue extends Map{constructor(t){let e=arguments.length>1&&void 0!==arguments[1]?arguments[1]:de;if(super(),Object.defineProperties(this,{_intern:{value:new Map},_key:{value:e}}),null!=t)for(const[e,n]of t)this.set(e,n)}get(t){return super.get(ce(this,t))}has(t){return super.has(ce(this,t))}set(t,e){return super.set(fe(this,t),e)}delete(t){return super.delete(he(this,t))}}class le extends Set{constructor(t){let e=arguments.length>1&&void 0!==arguments[1]?arguments[1]:de;if(super(),Object.defineProperties(this,{_intern:{value:new Map},_key:{value:e}}),null!=t)for(const e of t)this.add(e)}has(t){return super.has(ce(this,t))}add(t){return super.add(fe(this,t))}delete(t){return super.delete(he(this,t))}}function ce(t,e){let{_intern:n,_key:r}=t;const i=r(e);return n.has(i)?n.get(i):e}function fe(t,e){let{_intern:n,_key:r}=t;const i=r(e);return n.has(i)?n.get(i):(n.set(i,e),e)}function he(t,e){let{_intern:n,_key:r}=t;const i=r(e);return n.has(i)&&(e=n.get(i),n.delete(i)),e}function de(t){return null!==t&&\"object\"==typeof t?t.valueOf():t}function pe(t,e){return(null==t||!(t>=t))-(null==e||!(e>=e))||(te?1:0)}const ge=Math.sqrt(50),me=Math.sqrt(10),ye=Math.sqrt(2);function ve(t,e,n){const r=(e-t)/Math.max(0,n),i=Math.floor(Math.log10(r)),o=r/Math.pow(10,i),a=o>=ge?10:o>=me?5:o>=ye?2:1;let s,u,l;return i<0?(l=Math.pow(10,-i)/a,s=Math.round(t*l),u=Math.round(e*l),s/le&&--u,l=-l):(l=Math.pow(10,i)*a,s=Math.round(t/l),u=Math.round(e/l),s*le&&--u),u0))return[];if((t=+t)===(e=+e))return[t];const r=e=i))return[];const s=o-i+1,u=new Array(s);if(r)if(a<0)for(let t=0;t=e)&&(n=e);else{let r=-1;for(let i of t)null!=(i=e(i,++r,t))&&(n=i)&&(n=i)}return n}function ke(t,e){let n;if(void 0===e)for(const e of t)null!=e&&(n>e||void 0===n&&e>=e)&&(n=e);else{let r=-1;for(let i of t)null!=(i=e(i,++r,t))&&(n>i||void 0===n&&i>=i)&&(n=i)}return n}function Ae(t,e){let n=arguments.length>2&&void 0!==arguments[2]?arguments[2]:0,r=arguments.length>3&&void 0!==arguments[3]?arguments[3]:1/0,i=arguments.length>4?arguments[4]:void 0;if(e=Math.floor(e),n=Math.floor(Math.max(0,n)),r=Math.floor(Math.min(t.length-1,r)),!(n<=e&&e<=r))return t;for(i=void 0===i?pe:function(){let t=arguments.length>0&&void 0!==arguments[0]?arguments[0]:Kt;if(t===Kt)return pe;if(\"function\"!=typeof t)throw new TypeError(\"compare is not a function\");return(e,n)=>{const r=t(e,n);return r||0===r?r:(0===t(n,n))-(0===t(e,e))}}(i);r>n;){if(r-n>600){const o=r-n+1,a=e-n+1,s=Math.log(o),u=.5*Math.exp(2*s/3),l=.5*Math.sqrt(s*u*(o-u)/o)*(a-o/2<0?-1:1);Ae(t,e,Math.max(n,Math.floor(e-a*u/o+l)),Math.min(r,Math.floor(e+(o-a)*u/o+l)),i)}const o=t[e];let a=n,s=r;for(Me(t,n,e),i(t[r],o)>0&&Me(t,n,r);a0;)--s}0===i(t[n],o)?Me(t,n,s):(++s,Me(t,s,r)),s<=e&&(n=s+1),e<=s&&(r=s-1)}return t}function Me(t,e,n){const r=t[e];t[e]=t[n],t[n]=r}function Ee(t,e,n){if(t=Float64Array.from(function*(t,e){if(void 0===e)for(let e of t)null!=e&&(e=+e)>=e&&(yield e);else{let n=-1;for(let r of t)null!=(r=e(r,++n,t))&&(r=+r)>=r&&(yield r)}}(t,n)),(r=t.length)&&!isNaN(e=+e)){if(e<=0||r<2)return ke(t);if(e>=1)return we(t);var r,i=(r-1)*e,o=Math.floor(i),a=we(Ae(t,o).subarray(0,o+1));return a+(ke(t.subarray(o+1))-a)*(i-o)}}function De(t,e){let n=arguments.length>2&&void 0!==arguments[2]?arguments[2]:re;if((r=t.length)&&!isNaN(e=+e)){if(e<=0||r<2)return+n(t[0],0,t);if(e>=1)return+n(t[r-1],r-1,t);var r,i=(r-1)*e,o=Math.floor(i),a=+n(t[o],o,t);return a+(+n(t[o+1],o+1,t)-a)*(i-o)}}function Ce(t,e){return Ee(t,.5,e)}function Fe(t){return Array.from(function*(t){for(const e of t)yield*e}(t))}function Se(t,e,n){t=+t,e=+e,n=(i=arguments.length)<2?(e=t,t=0,1):i<3?1:+n;for(var r=-1,i=0|Math.max(0,Math.ceil((e-t)/n)),o=new Array(i);++r1?r[0]+r.slice(2):r,+t.slice(n+1)]}function ze(t){return(t=Be(Math.abs(t)))?t[1]:NaN}var Ne,Oe=/^(?:(.)?([<>=^]))?([+\\-( ])?([$#])?(0)?(\\d+)?(,)?(\\.\\d+)?(~)?([a-z%])?$/i;function Re(t){if(!(e=Oe.exec(t)))throw new Error(\"invalid format: \"+t);var e;return new Ue({fill:e[1],align:e[2],sign:e[3],symbol:e[4],zero:e[5],width:e[6],comma:e[7],precision:e[8]&&e[8].slice(1),trim:e[9],type:e[10]})}function Ue(t){this.fill=void 0===t.fill?\" \":t.fill+\"\",this.align=void 0===t.align?\">\":t.align+\"\",this.sign=void 0===t.sign?\"-\":t.sign+\"\",this.symbol=void 0===t.symbol?\"\":t.symbol+\"\",this.zero=!!t.zero,this.width=void 0===t.width?void 0:+t.width,this.comma=!!t.comma,this.precision=void 0===t.precision?void 0:+t.precision,this.trim=!!t.trim,this.type=void 0===t.type?\"\":t.type+\"\"}function Le(t,e){var n=Be(t,e);if(!n)return t+\"\";var r=n[0],i=n[1];return i<0?\"0.\"+new Array(-i).join(\"0\")+r:r.length>i+1?r.slice(0,i+1)+\".\"+r.slice(i+1):r+new Array(i-r.length+2).join(\"0\")}Re.prototype=Ue.prototype,Ue.prototype.toString=function(){return this.fill+this.align+this.sign+this.symbol+(this.zero?\"0\":\"\")+(void 0===this.width?\"\":Math.max(1,0|this.width))+(this.comma?\",\":\"\")+(void 0===this.precision?\"\":\".\"+Math.max(0,0|this.precision))+(this.trim?\"~\":\"\")+this.type};var qe={\"%\":(t,e)=>(100*t).toFixed(e),b:t=>Math.round(t).toString(2),c:t=>t+\"\",d:function(t){return Math.abs(t=Math.round(t))>=1e21?t.toLocaleString(\"en\").replace(/,/g,\"\"):t.toString(10)},e:(t,e)=>t.toExponential(e),f:(t,e)=>t.toFixed(e),g:(t,e)=>t.toPrecision(e),o:t=>Math.round(t).toString(8),p:(t,e)=>Le(100*t,e),r:Le,s:function(t,e){var n=Be(t,e);if(!n)return t+\"\";var r=n[0],i=n[1],o=i-(Ne=3*Math.max(-8,Math.min(8,Math.floor(i/3))))+1,a=r.length;return o===a?r:o>a?r+new Array(o-a+1).join(\"0\"):o>0?r.slice(0,o)+\".\"+r.slice(o):\"0.\"+new Array(1-o).join(\"0\")+Be(t,Math.max(0,e+o-1))[0]},X:t=>Math.round(t).toString(16).toUpperCase(),x:t=>Math.round(t).toString(16)};function Pe(t){return t}var je,Ie,We,He=Array.prototype.map,Ye=[\"y\",\"z\",\"a\",\"f\",\"p\",\"n\",\"µ\",\"m\",\"\",\"k\",\"M\",\"G\",\"T\",\"P\",\"E\",\"Z\",\"Y\"];function Ge(t){var e,n,r=void 0===t.grouping||void 0===t.thousands?Pe:(e=He.call(t.grouping,Number),n=t.thousands+\"\",function(t,r){for(var i=t.length,o=[],a=0,s=e[0],u=0;i>0&&s>0&&(u+s+1>r&&(s=Math.max(1,r-u)),o.push(t.substring(i-=s,i+s)),!((u+=s+1)>r));)s=e[a=(a+1)%e.length];return o.reverse().join(n)}),i=void 0===t.currency?\"\":t.currency[0]+\"\",o=void 0===t.currency?\"\":t.currency[1]+\"\",a=void 0===t.decimal?\".\":t.decimal+\"\",s=void 0===t.numerals?Pe:function(t){return function(e){return e.replace(/[0-9]/g,(function(e){return t[+e]}))}}(He.call(t.numerals,String)),u=void 0===t.percent?\"%\":t.percent+\"\",l=void 0===t.minus?\"−\":t.minus+\"\",c=void 0===t.nan?\"NaN\":t.nan+\"\";function f(t){var e=(t=Re(t)).fill,n=t.align,f=t.sign,h=t.symbol,d=t.zero,p=t.width,g=t.comma,m=t.precision,y=t.trim,v=t.type;\"n\"===v?(g=!0,v=\"g\"):qe[v]||(void 0===m&&(m=12),y=!0,v=\"g\"),(d||\"0\"===e&&\"=\"===n)&&(d=!0,e=\"0\",n=\"=\");var _=\"$\"===h?i:\"#\"===h&&/[boxX]/.test(v)?\"0\"+v.toLowerCase():\"\",x=\"$\"===h?o:/[%p]/.test(v)?u:\"\",b=qe[v],w=/[defgprs%]/.test(v);function k(t){var i,o,u,h=_,k=x;if(\"c\"===v)k=b(t)+k,t=\"\";else{var A=(t=+t)<0||1/t<0;if(t=isNaN(t)?c:b(Math.abs(t),m),y&&(t=function(t){t:for(var e,n=t.length,r=1,i=-1;r0&&(i=0)}return i>0?t.slice(0,i)+t.slice(e+1):t}(t)),A&&0==+t&&\"+\"!==f&&(A=!1),h=(A?\"(\"===f?f:l:\"-\"===f||\"(\"===f?\"\":f)+h,k=(\"s\"===v?Ye[8+Ne/3]:\"\")+k+(A&&\"(\"===f?\")\":\"\"),w)for(i=-1,o=t.length;++i(u=t.charCodeAt(i))||u>57){k=(46===u?a+t.slice(i+1):t.slice(i))+k,t=t.slice(0,i);break}}g&&!d&&(t=r(t,1/0));var M=h.length+t.length+k.length,E=M>1)+h+t+k+E.slice(M);break;default:t=E+h+t+k}return s(t)}return m=void 0===m?6:/[gprs]/.test(v)?Math.max(1,Math.min(21,m)):Math.max(0,Math.min(20,m)),k.toString=function(){return t+\"\"},k}return{format:f,formatPrefix:function(t,e){var n=f(((t=Re(t)).type=\"f\",t)),r=3*Math.max(-8,Math.min(8,Math.floor(ze(e)/3))),i=Math.pow(10,-r),o=Ye[8+r/3];return function(t){return n(i*t)+o}}}}function Ve(t){return Math.max(0,-ze(Math.abs(t)))}function Xe(t,e){return Math.max(0,3*Math.max(-8,Math.min(8,Math.floor(ze(e)/3)))-ze(Math.abs(t)))}function Je(t,e){return t=Math.abs(t),e=Math.abs(e)-t,Math.max(0,ze(e)-ze(t))+1}!function(t){je=Ge(t),Ie=je.format,We=je.formatPrefix}({thousands:\",\",grouping:[3],currency:[\"$\",\"\"]});const Ze=new Date,Qe=new Date;function Ke(t,e,n,r){function i(e){return t(e=0===arguments.length?new Date:new Date(+e)),e}return i.floor=e=>(t(e=new Date(+e)),e),i.ceil=n=>(t(n=new Date(n-1)),e(n,1),t(n),n),i.round=t=>{const e=i(t),n=i.ceil(t);return t-e(e(t=new Date(+t),null==n?1:Math.floor(n)),t),i.range=(n,r,o)=>{const a=[];if(n=i.ceil(n),o=null==o?1:Math.floor(o),!(n0))return a;let s;do{a.push(s=new Date(+n)),e(n,o),t(n)}while(sKe((e=>{if(e>=e)for(;t(e),!n(e);)e.setTime(e-1)}),((t,r)=>{if(t>=t)if(r<0)for(;++r<=0;)for(;e(t,-1),!n(t););else for(;--r>=0;)for(;e(t,1),!n(t););})),n&&(i.count=(e,r)=>(Ze.setTime(+e),Qe.setTime(+r),t(Ze),t(Qe),Math.floor(n(Ze,Qe))),i.every=t=>(t=Math.floor(t),isFinite(t)&&t>0?t>1?i.filter(r?e=>r(e)%t==0:e=>i.count(0,e)%t==0):i:null)),i}const tn=Ke((()=>{}),((t,e)=>{t.setTime(+t+e)}),((t,e)=>e-t));tn.every=t=>(t=Math.floor(t),isFinite(t)&&t>0?t>1?Ke((e=>{e.setTime(Math.floor(e/t)*t)}),((e,n)=>{e.setTime(+e+n*t)}),((e,n)=>(n-e)/t)):tn:null),tn.range;const en=1e3,nn=6e4,rn=36e5,on=864e5,an=6048e5,sn=2592e6,un=31536e6,ln=Ke((t=>{t.setTime(t-t.getMilliseconds())}),((t,e)=>{t.setTime(+t+e*en)}),((t,e)=>(e-t)/en),(t=>t.getUTCSeconds()));ln.range;const cn=Ke((t=>{t.setTime(t-t.getMilliseconds()-t.getSeconds()*en)}),((t,e)=>{t.setTime(+t+e*nn)}),((t,e)=>(e-t)/nn),(t=>t.getMinutes()));cn.range;const fn=Ke((t=>{t.setUTCSeconds(0,0)}),((t,e)=>{t.setTime(+t+e*nn)}),((t,e)=>(e-t)/nn),(t=>t.getUTCMinutes()));fn.range;const hn=Ke((t=>{t.setTime(t-t.getMilliseconds()-t.getSeconds()*en-t.getMinutes()*nn)}),((t,e)=>{t.setTime(+t+e*rn)}),((t,e)=>(e-t)/rn),(t=>t.getHours()));hn.range;const dn=Ke((t=>{t.setUTCMinutes(0,0,0)}),((t,e)=>{t.setTime(+t+e*rn)}),((t,e)=>(e-t)/rn),(t=>t.getUTCHours()));dn.range;const pn=Ke((t=>t.setHours(0,0,0,0)),((t,e)=>t.setDate(t.getDate()+e)),((t,e)=>(e-t-(e.getTimezoneOffset()-t.getTimezoneOffset())*nn)/on),(t=>t.getDate()-1));pn.range;const gn=Ke((t=>{t.setUTCHours(0,0,0,0)}),((t,e)=>{t.setUTCDate(t.getUTCDate()+e)}),((t,e)=>(e-t)/on),(t=>t.getUTCDate()-1));gn.range;const mn=Ke((t=>{t.setUTCHours(0,0,0,0)}),((t,e)=>{t.setUTCDate(t.getUTCDate()+e)}),((t,e)=>(e-t)/on),(t=>Math.floor(t/on)));function yn(t){return Ke((e=>{e.setDate(e.getDate()-(e.getDay()+7-t)%7),e.setHours(0,0,0,0)}),((t,e)=>{t.setDate(t.getDate()+7*e)}),((t,e)=>(e-t-(e.getTimezoneOffset()-t.getTimezoneOffset())*nn)/an))}mn.range;const vn=yn(0),_n=yn(1),xn=yn(2),bn=yn(3),wn=yn(4),kn=yn(5),An=yn(6);function Mn(t){return Ke((e=>{e.setUTCDate(e.getUTCDate()-(e.getUTCDay()+7-t)%7),e.setUTCHours(0,0,0,0)}),((t,e)=>{t.setUTCDate(t.getUTCDate()+7*e)}),((t,e)=>(e-t)/an))}vn.range,_n.range,xn.range,bn.range,wn.range,kn.range,An.range;const En=Mn(0),Dn=Mn(1),Cn=Mn(2),Fn=Mn(3),Sn=Mn(4),$n=Mn(5),Tn=Mn(6);En.range,Dn.range,Cn.range,Fn.range,Sn.range,$n.range,Tn.range;const Bn=Ke((t=>{t.setDate(1),t.setHours(0,0,0,0)}),((t,e)=>{t.setMonth(t.getMonth()+e)}),((t,e)=>e.getMonth()-t.getMonth()+12*(e.getFullYear()-t.getFullYear())),(t=>t.getMonth()));Bn.range;const zn=Ke((t=>{t.setUTCDate(1),t.setUTCHours(0,0,0,0)}),((t,e)=>{t.setUTCMonth(t.getUTCMonth()+e)}),((t,e)=>e.getUTCMonth()-t.getUTCMonth()+12*(e.getUTCFullYear()-t.getUTCFullYear())),(t=>t.getUTCMonth()));zn.range;const Nn=Ke((t=>{t.setMonth(0,1),t.setHours(0,0,0,0)}),((t,e)=>{t.setFullYear(t.getFullYear()+e)}),((t,e)=>e.getFullYear()-t.getFullYear()),(t=>t.getFullYear()));Nn.every=t=>isFinite(t=Math.floor(t))&&t>0?Ke((e=>{e.setFullYear(Math.floor(e.getFullYear()/t)*t),e.setMonth(0,1),e.setHours(0,0,0,0)}),((e,n)=>{e.setFullYear(e.getFullYear()+n*t)})):null,Nn.range;const On=Ke((t=>{t.setUTCMonth(0,1),t.setUTCHours(0,0,0,0)}),((t,e)=>{t.setUTCFullYear(t.getUTCFullYear()+e)}),((t,e)=>e.getUTCFullYear()-t.getUTCFullYear()),(t=>t.getUTCFullYear()));function Rn(t,e,n,r,i,o){const a=[[ln,1,en],[ln,5,5e3],[ln,15,15e3],[ln,30,3e4],[o,1,nn],[o,5,3e5],[o,15,9e5],[o,30,18e5],[i,1,rn],[i,3,108e5],[i,6,216e5],[i,12,432e5],[r,1,on],[r,2,1728e5],[n,1,an],[e,1,sn],[e,3,7776e6],[t,1,un]];function s(e,n,r){const i=Math.abs(n-e)/r,o=ee((t=>{let[,,e]=t;return e})).right(a,i);if(o===a.length)return t.every(be(e/un,n/un,r));if(0===o)return tn.every(Math.max(be(e,n,r),1));const[s,u]=a[i/a[o-1][2]isFinite(t=Math.floor(t))&&t>0?Ke((e=>{e.setUTCFullYear(Math.floor(e.getUTCFullYear()/t)*t),e.setUTCMonth(0,1),e.setUTCHours(0,0,0,0)}),((e,n)=>{e.setUTCFullYear(e.getUTCFullYear()+n*t)})):null,On.range;const[Un,Ln]=Rn(On,zn,En,mn,dn,fn),[qn,Pn]=Rn(Nn,Bn,vn,pn,hn,cn),jn=\"year\",In=\"quarter\",Wn=\"month\",Hn=\"week\",Yn=\"date\",Gn=\"day\",Vn=\"dayofyear\",Xn=\"hours\",Jn=\"minutes\",Zn=\"seconds\",Qn=\"milliseconds\",Kn=[jn,In,Wn,Hn,Yn,Gn,Vn,Xn,Jn,Zn,Qn],tr=Kn.reduce(((t,e,n)=>(t[e]=1+n,t)),{});function er(t){const e=V(t).slice(),n={};e.length||s(\"Missing time unit.\"),e.forEach((t=>{lt(tr,t)?n[t]=1:s(`Invalid time unit: ${t}.`)}));return(n[Hn]||n[Gn]?1:0)+(n[In]||n[Wn]||n[Yn]?1:0)+(n[Vn]?1:0)>1&&s(`Incompatible time units: ${t}`),e.sort(((t,e)=>tr[t]-tr[e])),e}const nr={[jn]:\"%Y \",[In]:\"Q%q \",[Wn]:\"%b \",[Yn]:\"%d \",[Hn]:\"W%U \",[Gn]:\"%a \",[Vn]:\"%j \",[Xn]:\"%H:00\",[Jn]:\"00:%M\",[Zn]:\":%S\",[Qn]:\".%L\",[`${jn}-${Wn}`]:\"%Y-%m \",[`${jn}-${Wn}-${Yn}`]:\"%Y-%m-%d \",[`${Xn}-${Jn}`]:\"%H:%M\"};function rr(t,e){const n=ot({},nr,e),r=er(t),i=r.length;let o,a,s=\"\",u=0;for(u=0;uu;--o)if(a=r.slice(u,o).join(\"-\"),null!=n[a]){s+=n[a],u=o;break}return s.trim()}const ir=new Date;function or(t){return ir.setFullYear(t),ir.setMonth(0),ir.setDate(1),ir.setHours(0,0,0,0),ir}function ar(t){return ur(new Date(t))}function sr(t){return lr(new Date(t))}function ur(t){return pn.count(or(t.getFullYear())-1,t)}function lr(t){return vn.count(or(t.getFullYear())-1,t)}function cr(t){return or(t).getDay()}function fr(t,e,n,r,i,o,a){if(0<=t&&t<100){const s=new Date(-1,e,n,r,i,o,a);return s.setFullYear(t),s}return new Date(t,e,n,r,i,o,a)}function hr(t){return pr(new Date(t))}function dr(t){return gr(new Date(t))}function pr(t){const e=Date.UTC(t.getUTCFullYear(),0,1);return gn.count(e-1,t)}function gr(t){const e=Date.UTC(t.getUTCFullYear(),0,1);return En.count(e-1,t)}function mr(t){return ir.setTime(Date.UTC(t,0,1)),ir.getUTCDay()}function yr(t,e,n,r,i,o,a){if(0<=t&&t<100){const t=new Date(Date.UTC(-1,e,n,r,i,o,a));return t.setUTCFullYear(n.y),t}return new Date(Date.UTC(t,e,n,r,i,o,a))}function vr(t,e,n,r,i){const o=e||1,a=F(t),s=(t,e,i)=>function(t,e,n,r){const i=n<=1?t:r?(e,i)=>r+n*Math.floor((t(e,i)-r)/n):(e,r)=>n*Math.floor(t(e,r)/n);return e?(t,n)=>e(i(t,n),n):i}(n[i=i||t],r[i],t===a&&o,e),u=new Date,l=Bt(t),c=l[jn]?s(jn):rt(2012),f=l[Wn]?s(Wn):l[In]?s(In):h,p=l[Hn]&&l[Gn]?s(Gn,1,Hn+Gn):l[Hn]?s(Hn,1):l[Gn]?s(Gn,1):l[Yn]?s(Yn,1):l[Vn]?s(Vn,1):d,g=l[Xn]?s(Xn):h,m=l[Jn]?s(Jn):h,y=l[Zn]?s(Zn):h,v=l[Qn]?s(Qn):h;return function(t){u.setTime(+t);const e=c(u);return i(e,f(u),p(u,e),g(u),m(u),y(u),v(u))}}function _r(t,e,n){return e+7*t-(n+6)%7}const xr={[jn]:t=>t.getFullYear(),[In]:t=>Math.floor(t.getMonth()/3),[Wn]:t=>t.getMonth(),[Yn]:t=>t.getDate(),[Xn]:t=>t.getHours(),[Jn]:t=>t.getMinutes(),[Zn]:t=>t.getSeconds(),[Qn]:t=>t.getMilliseconds(),[Vn]:t=>ur(t),[Hn]:t=>lr(t),[Hn+Gn]:(t,e)=>_r(lr(t),t.getDay(),cr(e)),[Gn]:(t,e)=>_r(1,t.getDay(),cr(e))},br={[In]:t=>3*t,[Hn]:(t,e)=>_r(t,0,cr(e))};function wr(t,e){return vr(t,e||1,xr,br,fr)}const kr={[jn]:t=>t.getUTCFullYear(),[In]:t=>Math.floor(t.getUTCMonth()/3),[Wn]:t=>t.getUTCMonth(),[Yn]:t=>t.getUTCDate(),[Xn]:t=>t.getUTCHours(),[Jn]:t=>t.getUTCMinutes(),[Zn]:t=>t.getUTCSeconds(),[Qn]:t=>t.getUTCMilliseconds(),[Vn]:t=>pr(t),[Hn]:t=>gr(t),[Gn]:(t,e)=>_r(1,t.getUTCDay(),mr(e)),[Hn+Gn]:(t,e)=>_r(gr(t),t.getUTCDay(),mr(e))},Ar={[In]:t=>3*t,[Hn]:(t,e)=>_r(t,0,mr(e))};function Mr(t,e){return vr(t,e||1,kr,Ar,yr)}const Er={[jn]:Nn,[In]:Bn.every(3),[Wn]:Bn,[Hn]:vn,[Yn]:pn,[Gn]:pn,[Vn]:pn,[Xn]:hn,[Jn]:cn,[Zn]:ln,[Qn]:tn},Dr={[jn]:On,[In]:zn.every(3),[Wn]:zn,[Hn]:En,[Yn]:gn,[Gn]:gn,[Vn]:gn,[Xn]:dn,[Jn]:fn,[Zn]:ln,[Qn]:tn};function Cr(t){return Er[t]}function Fr(t){return Dr[t]}function Sr(t,e,n){return t?t.offset(e,n):void 0}function $r(t,e,n){return Sr(Cr(t),e,n)}function Tr(t,e,n){return Sr(Fr(t),e,n)}function Br(t,e,n,r){return t?t.range(e,n,r):void 0}function zr(t,e,n,r){return Br(Cr(t),e,n,r)}function Nr(t,e,n,r){return Br(Fr(t),e,n,r)}const Or=1e3,Rr=6e4,Ur=36e5,Lr=864e5,qr=2592e6,Pr=31536e6,jr=[jn,Wn,Yn,Xn,Jn,Zn,Qn],Ir=jr.slice(0,-1),Wr=Ir.slice(0,-1),Hr=Wr.slice(0,-1),Yr=Hr.slice(0,-1),Gr=[jn,Wn],Vr=[jn],Xr=[[Ir,1,Or],[Ir,5,5e3],[Ir,15,15e3],[Ir,30,3e4],[Wr,1,Rr],[Wr,5,3e5],[Wr,15,9e5],[Wr,30,18e5],[Hr,1,Ur],[Hr,3,108e5],[Hr,6,216e5],[Hr,12,432e5],[Yr,1,Lr],[[jn,Hn],1,6048e5],[Gr,1,qr],[Gr,3,7776e6],[Vr,1,Pr]];function Jr(t){const e=t.extent,n=t.maxbins||40,r=Math.abs(Dt(e))/n;let i,o,a=ee((t=>t[2])).right(Xr,r);return a===Xr.length?(i=Vr,o=be(e[0]/Pr,e[1]/Pr,n)):a?(a=Xr[r/Xr[a-1][2]=12)]},q:function(t){return 1+~~(t.getMonth()/3)},Q:wo,s:ko,S:ji,u:Ii,U:Wi,V:Yi,w:Gi,W:Vi,x:null,X:null,y:Xi,Y:Zi,Z:Ki,\"%\":bo},x={a:function(t){return a[t.getUTCDay()]},A:function(t){return o[t.getUTCDay()]},b:function(t){return u[t.getUTCMonth()]},B:function(t){return s[t.getUTCMonth()]},c:null,d:to,e:to,f:oo,g:yo,G:_o,H:eo,I:no,j:ro,L:io,m:ao,M:so,p:function(t){return i[+(t.getUTCHours()>=12)]},q:function(t){return 1+~~(t.getUTCMonth()/3)},Q:wo,s:ko,S:uo,u:lo,U:co,V:ho,w:po,W:go,x:null,X:null,y:mo,Y:vo,Z:xo,\"%\":bo},b={a:function(t,e,n){var r=d.exec(e.slice(n));return r?(t.w=p.get(r[0].toLowerCase()),n+r[0].length):-1},A:function(t,e,n){var r=f.exec(e.slice(n));return r?(t.w=h.get(r[0].toLowerCase()),n+r[0].length):-1},b:function(t,e,n){var r=y.exec(e.slice(n));return r?(t.m=v.get(r[0].toLowerCase()),n+r[0].length):-1},B:function(t,e,n){var r=g.exec(e.slice(n));return r?(t.m=m.get(r[0].toLowerCase()),n+r[0].length):-1},c:function(t,n,r){return A(t,e,n,r)},d:Ai,e:Ai,f:Si,g:xi,G:_i,H:Ei,I:Ei,j:Mi,L:Fi,m:ki,M:Di,p:function(t,e,n){var r=l.exec(e.slice(n));return r?(t.p=c.get(r[0].toLowerCase()),n+r[0].length):-1},q:wi,Q:Ti,s:Bi,S:Ci,u:gi,U:mi,V:yi,w:pi,W:vi,x:function(t,e,r){return A(t,n,e,r)},X:function(t,e,n){return A(t,r,e,n)},y:xi,Y:_i,Z:bi,\"%\":$i};function w(t,e){return function(n){var r,i,o,a=[],s=-1,u=0,l=t.length;for(n instanceof Date||(n=new Date(+n));++s53)return null;\"w\"in o||(o.w=1),\"Z\"in o?(i=(r=Qr(Kr(o.y,0,1))).getUTCDay(),r=i>4||0===i?Dn.ceil(r):Dn(r),r=gn.offset(r,7*(o.V-1)),o.y=r.getUTCFullYear(),o.m=r.getUTCMonth(),o.d=r.getUTCDate()+(o.w+6)%7):(i=(r=Zr(Kr(o.y,0,1))).getDay(),r=i>4||0===i?_n.ceil(r):_n(r),r=pn.offset(r,7*(o.V-1)),o.y=r.getFullYear(),o.m=r.getMonth(),o.d=r.getDate()+(o.w+6)%7)}else(\"W\"in o||\"U\"in o)&&(\"w\"in o||(o.w=\"u\"in o?o.u%7:\"W\"in o?1:0),i=\"Z\"in o?Qr(Kr(o.y,0,1)).getUTCDay():Zr(Kr(o.y,0,1)).getDay(),o.m=0,o.d=\"W\"in o?(o.w+6)%7+7*o.W-(i+5)%7:o.w+7*o.U-(i+6)%7);return\"Z\"in o?(o.H+=o.Z/100|0,o.M+=o.Z%100,Qr(o)):Zr(o)}}function A(t,e,n,r){for(var i,o,a=0,s=e.length,u=n.length;a=u)return-1;if(37===(i=e.charCodeAt(a++))){if(i=e.charAt(a++),!(o=b[i in ai?e.charAt(a++):i])||(r=o(t,n,r))<0)return-1}else if(i!=n.charCodeAt(r++))return-1}return r}return _.x=w(n,_),_.X=w(r,_),_.c=w(e,_),x.x=w(n,x),x.X=w(r,x),x.c=w(e,x),{format:function(t){var e=w(t+=\"\",_);return e.toString=function(){return t},e},parse:function(t){var e=k(t+=\"\",!1);return e.toString=function(){return t},e},utcFormat:function(t){var e=w(t+=\"\",x);return e.toString=function(){return t},e},utcParse:function(t){var e=k(t+=\"\",!0);return e.toString=function(){return t},e}}}var ei,ni,ri,ii,oi,ai={\"-\":\"\",_:\" \",0:\"0\"},si=/^\\s*\\d+/,ui=/^%/,li=/[\\\\^$*+?|[\\]().{}]/g;function ci(t,e,n){var r=t<0?\"-\":\"\",i=(r?-t:t)+\"\",o=i.length;return r+(o[t.toLowerCase(),e])))}function pi(t,e,n){var r=si.exec(e.slice(n,n+1));return r?(t.w=+r[0],n+r[0].length):-1}function gi(t,e,n){var r=si.exec(e.slice(n,n+1));return r?(t.u=+r[0],n+r[0].length):-1}function mi(t,e,n){var r=si.exec(e.slice(n,n+2));return r?(t.U=+r[0],n+r[0].length):-1}function yi(t,e,n){var r=si.exec(e.slice(n,n+2));return r?(t.V=+r[0],n+r[0].length):-1}function vi(t,e,n){var r=si.exec(e.slice(n,n+2));return r?(t.W=+r[0],n+r[0].length):-1}function _i(t,e,n){var r=si.exec(e.slice(n,n+4));return r?(t.y=+r[0],n+r[0].length):-1}function xi(t,e,n){var r=si.exec(e.slice(n,n+2));return r?(t.y=+r[0]+(+r[0]>68?1900:2e3),n+r[0].length):-1}function bi(t,e,n){var r=/^(Z)|([+-]\\d\\d)(?::?(\\d\\d))?/.exec(e.slice(n,n+6));return r?(t.Z=r[1]?0:-(r[2]+(r[3]||\"00\")),n+r[0].length):-1}function wi(t,e,n){var r=si.exec(e.slice(n,n+1));return r?(t.q=3*r[0]-3,n+r[0].length):-1}function ki(t,e,n){var r=si.exec(e.slice(n,n+2));return r?(t.m=r[0]-1,n+r[0].length):-1}function Ai(t,e,n){var r=si.exec(e.slice(n,n+2));return r?(t.d=+r[0],n+r[0].length):-1}function Mi(t,e,n){var r=si.exec(e.slice(n,n+3));return r?(t.m=0,t.d=+r[0],n+r[0].length):-1}function Ei(t,e,n){var r=si.exec(e.slice(n,n+2));return r?(t.H=+r[0],n+r[0].length):-1}function Di(t,e,n){var r=si.exec(e.slice(n,n+2));return r?(t.M=+r[0],n+r[0].length):-1}function Ci(t,e,n){var r=si.exec(e.slice(n,n+2));return r?(t.S=+r[0],n+r[0].length):-1}function Fi(t,e,n){var r=si.exec(e.slice(n,n+3));return r?(t.L=+r[0],n+r[0].length):-1}function Si(t,e,n){var r=si.exec(e.slice(n,n+6));return r?(t.L=Math.floor(r[0]/1e3),n+r[0].length):-1}function $i(t,e,n){var r=ui.exec(e.slice(n,n+1));return r?n+r[0].length:-1}function Ti(t,e,n){var r=si.exec(e.slice(n));return r?(t.Q=+r[0],n+r[0].length):-1}function Bi(t,e,n){var r=si.exec(e.slice(n));return r?(t.s=+r[0],n+r[0].length):-1}function zi(t,e){return ci(t.getDate(),e,2)}function Ni(t,e){return ci(t.getHours(),e,2)}function Oi(t,e){return ci(t.getHours()%12||12,e,2)}function Ri(t,e){return ci(1+pn.count(Nn(t),t),e,3)}function Ui(t,e){return ci(t.getMilliseconds(),e,3)}function Li(t,e){return Ui(t,e)+\"000\"}function qi(t,e){return ci(t.getMonth()+1,e,2)}function Pi(t,e){return ci(t.getMinutes(),e,2)}function ji(t,e){return ci(t.getSeconds(),e,2)}function Ii(t){var e=t.getDay();return 0===e?7:e}function Wi(t,e){return ci(vn.count(Nn(t)-1,t),e,2)}function Hi(t){var e=t.getDay();return e>=4||0===e?wn(t):wn.ceil(t)}function Yi(t,e){return t=Hi(t),ci(wn.count(Nn(t),t)+(4===Nn(t).getDay()),e,2)}function Gi(t){return t.getDay()}function Vi(t,e){return ci(_n.count(Nn(t)-1,t),e,2)}function Xi(t,e){return ci(t.getFullYear()%100,e,2)}function Ji(t,e){return ci((t=Hi(t)).getFullYear()%100,e,2)}function Zi(t,e){return ci(t.getFullYear()%1e4,e,4)}function Qi(t,e){var n=t.getDay();return ci((t=n>=4||0===n?wn(t):wn.ceil(t)).getFullYear()%1e4,e,4)}function Ki(t){var e=t.getTimezoneOffset();return(e>0?\"-\":(e*=-1,\"+\"))+ci(e/60|0,\"0\",2)+ci(e%60,\"0\",2)}function to(t,e){return ci(t.getUTCDate(),e,2)}function eo(t,e){return ci(t.getUTCHours(),e,2)}function no(t,e){return ci(t.getUTCHours()%12||12,e,2)}function ro(t,e){return ci(1+gn.count(On(t),t),e,3)}function io(t,e){return ci(t.getUTCMilliseconds(),e,3)}function oo(t,e){return io(t,e)+\"000\"}function ao(t,e){return ci(t.getUTCMonth()+1,e,2)}function so(t,e){return ci(t.getUTCMinutes(),e,2)}function uo(t,e){return ci(t.getUTCSeconds(),e,2)}function lo(t){var e=t.getUTCDay();return 0===e?7:e}function co(t,e){return ci(En.count(On(t)-1,t),e,2)}function fo(t){var e=t.getUTCDay();return e>=4||0===e?Sn(t):Sn.ceil(t)}function ho(t,e){return t=fo(t),ci(Sn.count(On(t),t)+(4===On(t).getUTCDay()),e,2)}function po(t){return t.getUTCDay()}function go(t,e){return ci(Dn.count(On(t)-1,t),e,2)}function mo(t,e){return ci(t.getUTCFullYear()%100,e,2)}function yo(t,e){return ci((t=fo(t)).getUTCFullYear()%100,e,2)}function vo(t,e){return ci(t.getUTCFullYear()%1e4,e,4)}function _o(t,e){var n=t.getUTCDay();return ci((t=n>=4||0===n?Sn(t):Sn.ceil(t)).getUTCFullYear()%1e4,e,4)}function xo(){return\"+0000\"}function bo(){return\"%\"}function wo(t){return+t}function ko(t){return Math.floor(+t/1e3)}function Ao(t){const e={};return n=>e[n]||(e[n]=t(n))}function Mo(t){const e=Ao(t.format),n=t.formatPrefix;return{format:e,formatPrefix:n,formatFloat(t){const n=Re(t||\",\");if(null==n.precision){switch(n.precision=12,n.type){case\"%\":n.precision-=2;break;case\"e\":n.precision-=1}return r=e(n),i=e(\".1f\")(1)[1],t=>{const e=r(t),n=e.indexOf(i);if(n<0)return e;let o=function(t,e){let n,r=t.lastIndexOf(\"e\");if(r>0)return r;for(r=t.length;--r>e;)if(n=t.charCodeAt(r),n>=48&&n<=57)return r+1}(e,n);const a=on;)if(\"0\"!==e[o]){++o;break}return e.slice(0,o)+a}}return e(n);var r,i},formatSpan(t,r,i,o){o=Re(null==o?\",f\":o);const a=be(t,r,i),s=Math.max(Math.abs(t),Math.abs(r));let u;if(null==o.precision)switch(o.type){case\"s\":return isNaN(u=Xe(a,s))||(o.precision=u),n(o,s);case\"\":case\"e\":case\"g\":case\"p\":case\"r\":isNaN(u=Je(a,s))||(o.precision=u-(\"e\"===o.type));break;case\"f\":case\"%\":isNaN(u=Ve(a))||(o.precision=u-2*(\"%\"===o.type))}return e(o)}}}let Eo,Do;function Co(){return Eo=Mo({format:Ie,formatPrefix:We})}function Fo(t){return Mo(Ge(t))}function So(t){return arguments.length?Eo=Fo(t):Eo}function $o(t,e,n){A(n=n||{})||s(`Invalid time multi-format specifier: ${n}`);const r=e(Zn),i=e(Jn),o=e(Xn),a=e(Yn),u=e(Hn),l=e(Wn),c=e(In),f=e(jn),h=t(n[Qn]||\".%L\"),d=t(n[Zn]||\":%S\"),p=t(n[Jn]||\"%I:%M\"),g=t(n[Xn]||\"%I %p\"),m=t(n[Yn]||n[Gn]||\"%a %d\"),y=t(n[Hn]||\"%b %d\"),v=t(n[Wn]||\"%B\"),_=t(n[In]||\"%B\"),x=t(n[jn]||\"%Y\");return t=>(r(t)xt(t)?e(t):$o(e,Cr,t),utcFormat:t=>xt(t)?n(t):$o(n,Fr,t),timeParse:Ao(t.parse),utcParse:Ao(t.utcParse)}}function Bo(){return Do=To({format:ni,parse:ri,utcFormat:ii,utcParse:oi})}function zo(t){return To(ti(t))}function No(t){return arguments.length?Do=zo(t):Do}!function(t){ei=ti(t),ni=ei.format,ri=ei.parse,ii=ei.utcFormat,oi=ei.utcParse}({dateTime:\"%x, %X\",date:\"%-m/%-d/%Y\",time:\"%-I:%M:%S %p\",periods:[\"AM\",\"PM\"],days:[\"Sunday\",\"Monday\",\"Tuesday\",\"Wednesday\",\"Thursday\",\"Friday\",\"Saturday\"],shortDays:[\"Sun\",\"Mon\",\"Tue\",\"Wed\",\"Thu\",\"Fri\",\"Sat\"],months:[\"January\",\"February\",\"March\",\"April\",\"May\",\"June\",\"July\",\"August\",\"September\",\"October\",\"November\",\"December\"],shortMonths:[\"Jan\",\"Feb\",\"Mar\",\"Apr\",\"May\",\"Jun\",\"Jul\",\"Aug\",\"Sep\",\"Oct\",\"Nov\",\"Dec\"]}),Co(),Bo();const Oo=(t,e)=>ot({},t,e);function Ro(t,e){const n=t?Fo(t):So(),r=e?zo(e):No();return Oo(n,r)}function Uo(t,e){const n=arguments.length;return n&&2!==n&&s(\"defaultLocale expects either zero or two arguments.\"),n?Oo(So(t),No(e)):Oo(So(),No())}const Lo=/^(data:|([A-Za-z]+:)?\\/\\/)/,qo=/^(?:(?:(?:f|ht)tps?|mailto|tel|callto|cid|xmpp|file|data):|[^a-z]|[a-z+.\\-]+(?:[^a-z+.\\-:]|$))/i,Po=/[\\u0000-\\u0020\\u00A0\\u1680\\u180E\\u2000-\\u2029\\u205f\\u3000]/g,jo=\"file://\";async function Io(t,e){const n=await this.sanitize(t,e),r=n.href;return n.localFile?this.file(r):this.http(r,e)}async function Wo(t,e){e=ot({},this.options,e);const n=this.fileAccess,r={href:null};let i,o,a;const u=qo.test(t.replace(Po,\"\"));null!=t&&\"string\"==typeof t&&u||s(\"Sanitize failure, invalid URI: \"+Ct(t));const l=Lo.test(t);return(a=e.baseURL)&&!l&&(t.startsWith(\"/\")||a.endsWith(\"/\")||(t=\"/\"+t),t=a+t),o=(i=t.startsWith(jo))||\"file\"===e.mode||\"http\"!==e.mode&&!l&&n,i?t=t.slice(jo.length):t.startsWith(\"//\")&&(\"file\"===e.defaultProtocol?(t=t.slice(2),o=!0):t=(e.defaultProtocol||\"http\")+\":\"+t),Object.defineProperty(r,\"localFile\",{value:!!o}),r.href=t,e.target&&(r.target=e.target+\"\"),e.rel&&(r.rel=e.rel+\"\"),\"image\"===e.context&&e.crossOrigin&&(r.crossOrigin=e.crossOrigin+\"\"),r}function Ho(t){return t?e=>new Promise(((n,r)=>{t.readFile(e,((t,e)=>{t?r(t):n(e)}))})):Yo}async function Yo(){s(\"No file system access.\")}function Go(t){return t?async function(e,n){const r=ot({},this.options.http,n),i=n&&n.response,o=await t(e,r);return o.ok?J(o[i])?o[i]():o.text():s(o.status+\"\"+o.statusText)}:Vo}async function Vo(){s(\"No HTTP fetch method available.\")}const Xo=t=>null!=t&&t==t,Jo=t=>!(Number.isNaN(+t)||t instanceof Date),Zo={boolean:Ft,integer:S,number:S,date:$t,string:Tt,unknown:f},Qo=[t=>\"true\"===t||\"false\"===t||!0===t||!1===t,t=>Jo(t)&&Number.isInteger(+t),Jo,t=>!Number.isNaN(Date.parse(t))],Ko=[\"boolean\",\"integer\",\"number\",\"date\"];function ta(t,e){if(!t||!t.length)return\"unknown\";const n=t.length,r=Qo.length,i=Qo.map(((t,e)=>e+1));for(let o,a,s=0,u=0;s0===t?e:t),0)-1]}function ea(t,e){return e.reduce(((e,n)=>(e[n]=ta(t,n),e)),{})}function na(t){const e=function(e,n){const r={delimiter:t};return ra(e,n?ot(n,r):r)};return e.responseType=\"text\",e}function ra(t,e){return e.header&&(t=e.header.map(Ct).join(e.delimiter)+\"\\n\"+t),Ht(e.delimiter).parse(t+\"\")}function ia(t,e){const n=e&&e.property?l(e.property):f;return!A(t)||(r=t,\"function\"==typeof Buffer&&J(Buffer.isBuffer)&&Buffer.isBuffer(r))?n(JSON.parse(t)):function(t,e){!k(t)&&yt(t)&&(t=[...t]);return e&&e.copy?JSON.parse(JSON.stringify(t)):t}(n(t),e);var r}ra.responseType=\"text\",ia.responseType=\"json\";const oa={interior:(t,e)=>t!==e,exterior:(t,e)=>t===e};function aa(t,e){let n,r,i,o;return t=ia(t,e),e&&e.feature?(n=Gt,i=e.feature):e&&e.mesh?(n=Zt,i=e.mesh,o=oa[e.filter]):s(\"Missing TopoJSON feature or mesh parameter.\"),r=(r=t.objects[i])?n(t,r,o):s(\"Invalid TopoJSON object: \"+i),r&&r.features||[r]}aa.responseType=\"json\";const sa={dsv:ra,csv:na(\",\"),tsv:na(\"\\t\"),json:ia,topojson:aa};function ua(t,e){return arguments.length>1?(sa[t]=e,this):lt(sa,t)?sa[t]:null}function la(t){const e=ua(t);return e&&e.responseType||\"text\"}function ca(t,e,n,r){const i=ua((e=e||{}).type||\"json\");return i||s(\"Unknown data format type: \"+e.type),t=i(t,e),e.parse&&function(t,e,n,r){if(!t.length)return;const i=No();n=n||i.timeParse,r=r||i.utcParse;let o,a,s,u,l,c,f=t.columns||Object.keys(t[0]);\"auto\"===e&&(e=ea(t,f));f=Object.keys(e);const h=f.map((t=>{const i=e[t];let o,a;if(i&&(i.startsWith(\"date:\")||i.startsWith(\"utc:\"))){o=i.split(/:(.+)?/,2),a=o[1],(\"'\"===a[0]&&\"'\"===a[a.length-1]||'\"'===a[0]&&'\"'===a[a.length-1])&&(a=a.slice(1,-1));return(\"utc\"===o[0]?r:n)(a)}if(!Zo[i])throw Error(\"Illegal format pattern: \"+t+\":\"+i);return Zo[i]}));for(s=0,l=t.length,c=f.length;s({options:n||{},sanitize:Wo,load:Io,fileAccess:!!e,file:Ho(e),http:Go(t)})}(\"undefined\"!=typeof fetch&&fetch,null);function ha(t){const e=t||f,n=[],r={};return n.add=t=>{const i=e(t);return r[i]||(r[i]=1,n.push(t)),n},n.remove=t=>{const i=e(t);if(r[i]){r[i]=0;const e=n.indexOf(t);e>=0&&n.splice(e,1)}return n},n}async function da(t,e){try{await e(t)}catch(e){t.error(e)}}const pa=Symbol(\"vega_id\");let ga=1;function ma(t){return!(!t||!ya(t))}function ya(t){return t[pa]}function va(t,e){return t[pa]=e,t}function _a(t){const e=t===Object(t)?t:{data:t};return ya(e)?e:va(e,ga++)}function xa(t){return ba(t,_a({}))}function ba(t,e){for(const n in t)e[n]=t[n];return e}function wa(t,e){return va(e,ya(t))}function ka(t,e){return t?e?(n,r)=>t(n,r)||ya(e(n))-ya(e(r)):(e,n)=>t(e,n)||ya(e)-ya(n):null}function Aa(t){return t&&t.constructor===Ma}function Ma(){const t=[],e=[],n=[],r=[],i=[];let o=null,a=!1;return{constructor:Ma,insert(e){const n=V(e),r=n.length;for(let e=0;e{p(t)&&(l[ya(t)]=-1)}));for(f=0,h=t.length;f0&&(y(g,p,d.value),s.modifies(p));for(f=0,h=i.length;f{p(t)&&l[ya(t)]>0&&y(t,d.field,d.value)})),s.modifies(d.field);if(a)s.mod=e.length||r.length?u.filter((t=>l[ya(t)]>0)):u.slice();else for(m in c)s.mod.push(c[m]);return(o||null==o&&(e.length||r.length))&&s.clean(!0),s}}}const Ea=\"_:mod:_\";function Da(){Object.defineProperty(this,Ea,{writable:!0,value:{}})}Da.prototype={set(t,e,n,r){const i=this,o=i[t],a=i[Ea];return null!=e&&e>=0?(o[e]!==n||r)&&(o[e]=n,a[e+\":\"+t]=-1,a[t]=-1):(o!==n||r)&&(i[t]=n,a[t]=k(n)?1+n.length:-1),i},modified(t,e){const n=this[Ea];if(!arguments.length){for(const t in n)if(n[t])return!0;return!1}if(k(t)){for(let e=0;e=0?e+1{a instanceof Sa?(a!==this&&(e&&a.targets().add(this),o.push(a)),i.push({op:a,name:t,index:n})):r.set(t,n,a)};for(a in t)if(u=t[a],\"pulse\"===a)V(u).forEach((t=>{t instanceof Sa?t!==this&&(t.targets().add(this),o.push(t)):s(\"Pulse parameters must be operator instances.\")})),this.source=u;else if(k(u))for(r.set(a,-1,Array(l=u.length)),c=0;c{const n=Date.now();return n-e>t?(e=n,1):0}))},debounce(t){const e=za();return this.targets().add(za(null,null,it(t,(t=>{const n=t.dataflow;e.receive(t),n&&n.run&&n.run()})))),e},between(t,e){let n=!1;return t.targets().add(za(null,null,(()=>n=!0))),e.targets().add(za(null,null,(()=>n=!1))),this.filter((()=>n))},detach(){this._filter=p,this._targets=null}};const Na={skip:!0};function Oa(t,e,n,r,i,o){const a=ot({},o,Na);let s,u;J(n)||(n=rt(n)),void 0===r?s=e=>t.touch(n(e)):J(r)?(u=new Sa(null,r,i,!1),s=e=>{u.evaluate(e);const r=n(e),i=u.value;Aa(i)?t.pulse(r,i,o):t.update(r,i,a)}):s=e=>t.update(n(e),r,a),e.apply(s)}function Ra(t,e,n,r,i,o){if(void 0===r)e.targets().add(n);else{const a=o||{},s=new Sa(null,function(t,e){return e=J(e)?e:rt(e),t?function(n,r){const i=e(n,r);return t.skip()||(t.skip(i!==this.value).value=i),i}:e}(n,r),i,!1);s.modified(a.force),s.rank=e.rank,e.targets().add(s),n&&(s.skip(!0),s.value=n.value,s.targets().add(n),t.connect(n,[s]))}}const Ua={};function La(t,e,n){this.dataflow=t,this.stamp=null==e?-1:e,this.add=[],this.rem=[],this.mod=[],this.fields=null,this.encode=n||null}function qa(t,e){const n=[];return Nt(t,e,(t=>n.push(t))),n}function Pa(t,e){const n={};return t.visit(e,(t=>{n[ya(t)]=1})),t=>n[ya(t)]?null:t}function ja(t,e){return t?(n,r)=>t(n,r)&&e(n,r):e}function Ia(t,e,n,r){const i=this;let o=0;this.dataflow=t,this.stamp=e,this.fields=null,this.encode=r||null,this.pulses=n;for(const t of n)if(t.stamp===e){if(t.fields){const e=i.fields||(i.fields={});for(const n in t.fields)e[n]=1}t.changed(i.ADD)&&(o|=i.ADD),t.changed(i.REM)&&(o|=i.REM),t.changed(i.MOD)&&(o|=i.MOD)}this.changes=o}function Wa(t){return t.error(\"Dataflow already running. Use runAsync() to chain invocations.\"),t}La.prototype={StopPropagation:Ua,ADD:1,REM:2,MOD:4,ADD_REM:3,ADD_MOD:5,ALL:7,REFLOW:8,SOURCE:16,NO_SOURCE:32,NO_FIELDS:64,fork(t){return new La(this.dataflow).init(this,t)},clone(){const t=this.fork(7);return t.add=t.add.slice(),t.rem=t.rem.slice(),t.mod=t.mod.slice(),t.source&&(t.source=t.source.slice()),t.materialize(23)},addAll(){let t=this;return!t.source||t.add===t.rem||!t.rem.length&&t.source.length===t.add.length||(t=new La(this.dataflow).init(this),t.add=t.source,t.rem=[]),t},init(t,e){const n=this;return n.stamp=t.stamp,n.encode=t.encode,!t.fields||64&e||(n.fields=t.fields),1&e?(n.addF=t.addF,n.add=t.add):(n.addF=null,n.add=[]),2&e?(n.remF=t.remF,n.rem=t.rem):(n.remF=null,n.rem=[]),4&e?(n.modF=t.modF,n.mod=t.mod):(n.modF=null,n.mod=[]),32&e?(n.srcF=null,n.source=null):(n.srcF=t.srcF,n.source=t.source,t.cleans&&(n.cleans=t.cleans)),n},runAfter(t){this.dataflow.runAfter(t)},changed(t){const e=t||7;return 1&e&&this.add.length||2&e&&this.rem.length||4&e&&this.mod.length},reflow(t){if(t)return this.fork(7).reflow();const e=this.add.length,n=this.source&&this.source.length;return n&&n!==e&&(this.mod=this.source,e&&this.filter(4,Pa(this,1))),this},clean(t){return arguments.length?(this.cleans=!!t,this):this.cleans},modifies(t){const e=this.fields||(this.fields={});return k(t)?t.forEach((t=>e[t]=!0)):e[t]=!0,this},modified(t,e){const n=this.fields;return!(!e&&!this.mod.length||!n)&&(arguments.length?k(t)?t.some((t=>n[t])):n[t]:!!n)},filter(t,e){const n=this;return 1&t&&(n.addF=ja(n.addF,e)),2&t&&(n.remF=ja(n.remF,e)),4&t&&(n.modF=ja(n.modF,e)),16&t&&(n.srcF=ja(n.srcF,e)),n},materialize(t){const e=this;return 1&(t=t||7)&&e.addF&&(e.add=qa(e.add,e.addF),e.addF=null),2&t&&e.remF&&(e.rem=qa(e.rem,e.remF),e.remF=null),4&t&&e.modF&&(e.mod=qa(e.mod,e.modF),e.modF=null),16&t&&e.srcF&&(e.source=e.source.filter(e.srcF),e.srcF=null),e},visit(t,e){const n=this,r=e;if(16&t)return Nt(n.source,n.srcF,r),n;1&t&&Nt(n.add,n.addF,r),2&t&&Nt(n.rem,n.remF,r),4&t&&Nt(n.mod,n.modF,r);const i=n.source;if(8&t&&i){const t=n.add.length+n.mod.length;t===i.length||Nt(i,t?Pa(n,5):n.srcF,r)}return n}},dt(Ia,La,{fork(t){const e=new La(this.dataflow).init(this,t&this.NO_FIELDS);return void 0!==t&&(t&e.ADD&&this.visit(e.ADD,(t=>e.add.push(t))),t&e.REM&&this.visit(e.REM,(t=>e.rem.push(t))),t&e.MOD&&this.visit(e.MOD,(t=>e.mod.push(t)))),e},changed(t){return this.changes&t},modified(t){const e=this,n=e.fields;return n&&e.changes&e.MOD?k(t)?t.some((t=>n[t])):n[t]:0},filter(){s(\"MultiPulse does not support filtering.\")},materialize(){s(\"MultiPulse does not support materialization.\")},visit(t,e){const n=this,r=n.pulses,i=r.length;let o=0;if(t&n.SOURCE)for(;oe=[],size:()=>e.length,peek:()=>e[0],push:n=>(e.push(n),Ga(e,0,e.length-1,t)),pop:()=>{const n=e.pop();let r;return e.length?(r=e[0],e[0]=n,function(t,e,n){const r=e,i=t.length,o=t[e];let a,s=1+(e<<1);for(;s=0&&(s=a),t[e]=t[s],s=1+((e=s)<<1);t[e]=o,Ga(t,r,e,n)}(e,0,t)):r=n,r}}}function Ga(t,e,n,r){let i,o;const a=t[n];for(;n>e&&(o=n-1>>1,i=t[o],r(a,i)<0);)t[n]=i,n=o;return t[n]=a}function Va(){this.logger(w()),this.logLevel(v),this._clock=0,this._rank=0,this._locale=Uo();try{this._loader=fa()}catch(t){}this._touched=ha(c),this._input={},this._pulse=null,this._heap=Ya(((t,e)=>t.qrank-e.qrank)),this._postrun=[]}function Xa(t){return function(){return this._log[t].apply(this,arguments)}}function Ja(t,e){Sa.call(this,t,null,e)}Va.prototype={stamp(){return this._clock},loader(t){return arguments.length?(this._loader=t,this):this._loader},locale(t){return arguments.length?(this._locale=t,this):this._locale},logger(t){return arguments.length?(this._log=t,this):this._log},error:Xa(\"error\"),warn:Xa(\"warn\"),info:Xa(\"info\"),debug:Xa(\"debug\"),logLevel:Xa(\"level\"),cleanThreshold:1e4,add:function(t,e,n,r){let i,o=1;return t instanceof Sa?i=t:t&&t.prototype instanceof Sa?i=new t:J(t)?i=new Sa(null,t):(o=0,i=new Sa(t,e)),this.rank(i),o&&(r=n,n=e),n&&this.connect(i,i.parameters(n,r)),this.touch(i),i},connect:function(t,e){const n=t.rank,r=e.length;for(let i=0;i=0;)e.push(n=r[i]),n===t&&s(\"Cycle detected in dataflow graph.\")},pulse:function(t,e,n){this.touch(t,n||Ha);const r=new La(this,this._clock+(this._pulse?0:1)),i=t.pulse&&t.pulse.source||[];return r.target=t,this._input[t.id]=e.pulse(r,i),this},touch:function(t,e){const n=e||Ha;return this._pulse?this._enqueue(t):this._touched.add(t),n.skip&&t.skip(!0),this},update:function(t,e,n){const r=n||Ha;return(t.set(e)||r.force)&&this.touch(t,r),this},changeset:Ma,ingest:function(t,e,n){return e=this.parse(e,n),this.pulse(t,this.changeset().insert(e))},parse:function(t,e){const n=this.locale();return ca(t,e,n.timeParse,n.utcParse)},preload:async function(t,e,n){const r=this,i=r._pending||function(t){let e;const n=new Promise((t=>e=t));return n.requests=0,n.done=()=>{0==--n.requests&&(t._pending=null,e(t))},t._pending=n}(r);i.requests+=1;const o=await r.request(e,n);return r.pulse(t,r.changeset().remove(p).insert(o.data||[])),i.done(),o},request:async function(t,e){const n=this;let r,i=0;try{r=await n.loader().load(t,{context:\"dataflow\",response:la(e&&e.type)});try{r=n.parse(r,e)}catch(e){i=-2,n.warn(\"Data ingestion failed\",t,e)}}catch(e){i=-1,n.warn(\"Loading failed\",t,e)}return{data:r,status:i}},events:function(t,e,n,r){const i=this,o=za(n,r),a=function(t){t.dataflow=i;try{o.receive(t)}catch(t){i.error(t)}finally{i.run()}};let s;s=\"string\"==typeof t&&\"undefined\"!=typeof document?document.querySelectorAll(t):V(t);const u=s.length;for(let t=0;tr._enqueue(t,!0))),r._touched=ha(c);let a,s,u,l=0;try{for(;r._heap.size()>0;)a=r._heap.pop(),a.rank===a.qrank?(s=a.run(r._getPulse(a,t)),s.then?s=await s:s.async&&(i.push(s.async),s=Ua),s!==Ua&&a._targets&&a._targets.forEach((t=>r._enqueue(t))),++l):r._enqueue(a,!0)}catch(t){r._heap.clear(),u=t}if(r._input={},r._pulse=null,r.debug(`Pulse ${o}: ${l} operators`),u&&(r._postrun=[],r.error(u)),r._postrun.length){const t=r._postrun.sort(((t,e)=>e.priority-t.priority));r._postrun=[];for(let e=0;er.runAsync(null,(()=>{t.forEach((t=>{try{t(r)}catch(t){r.error(t)}}))})))),r},run:function(t,e,n){return this._pulse?Wa(this):(this.evaluate(t,e,n),this)},runAsync:async function(t,e,n){for(;this._running;)await this._running;const r=()=>this._running=null;return(this._running=this.evaluate(t,e,n)).then(r,r),this._running},runAfter:function(t,e,n){if(this._pulse||e)this._postrun.push({priority:n||0,callback:t});else try{t(this)}catch(t){this.error(t)}},_enqueue:function(t,e){const n=t.stampt.pulse)),e):this._input[t.id]||function(t,e){if(e&&e.stamp===t.stamp)return e;t=t.fork(),e&&e!==Ua&&(t.source=e.source);return t}(this._pulse,n&&n.pulse)}},dt(Ja,Sa,{run(t){if(t.stampthis.pulse=t)):e!==t.StopPropagation&&(this.pulse=e),e},evaluate(t){const e=this.marshall(t.stamp),n=this.transform(e,t);return e.clear(),n},transform(){}});const Za={};function Qa(t){const e=Ka(t);return e&&e.Definition||null}function Ka(t){return t=t&&t.toLowerCase(),lt(Za,t)?Za[t]:null}function*ts(t,e){if(null==e)for(let e of t)null!=e&&\"\"!==e&&(e=+e)>=e&&(yield e);else{let n=-1;for(let r of t)r=e(r,++n,t),null!=r&&\"\"!==r&&(r=+r)>=r&&(yield r)}}function es(t,e,n){const r=Float64Array.from(ts(t,n));return r.sort(Kt),e.map((t=>De(r,t)))}function ns(t,e){return es(t,[.25,.5,.75],e)}function rs(t,e){const n=t.length,r=function(t,e){const n=function(t,e){let n,r=0,i=0,o=0;if(void 0===e)for(let e of t)null!=e&&(e=+e)>=e&&(n=e-i,i+=n/++r,o+=n*(e-i));else{let a=-1;for(let s of t)null!=(s=e(s,++a,t))&&(s=+s)>=s&&(n=s-i,i+=n/++r,o+=n*(s-i))}if(r>1)return o/(r-1)}(t,e);return n?Math.sqrt(n):n}(t,e),i=ns(t,e),o=(i[2]-i[0])/1.34;return 1.06*(Math.min(r,o)||r||Math.abs(i[0])||1)*Math.pow(n,-.2)}function is(t){const e=t.maxbins||20,n=t.base||10,r=Math.log(n),i=t.divide||[5,2];let o,a,s,u,l,c,f=t.extent[0],h=t.extent[1];const d=t.span||h-f||Math.abs(f)||1;if(t.step)o=t.step;else if(t.steps){for(u=d/e,l=0,c=t.steps.length;le;)o*=n;for(l=0,c=i.length;l=s&&d/u<=e&&(o=u)}u=Math.log(o);const p=u>=0?0:1+~~(-u/r),g=Math.pow(n,-p-1);return(t.nice||void 0===t.nice)&&(u=Math.floor(f/o+g)*o,f=ft);const i=t.length,o=new Float64Array(i);let a,s=0,u=1,l=r(t[0]),c=l,f=l+e;for(;u=f){for(c=(l+c)/2;s>1);ia;)t[i--]=t[o]}o=a,a=r}return t}(o,e+e/4):o}t.random=Math.random;const ss=Math.sqrt(2*Math.PI),us=Math.SQRT2;let ls=NaN;function cs(e,n){e=e||0,n=null==n?1:n;let r,i,o=0,a=0;if(ls==ls)o=ls,ls=NaN;else{do{o=2*t.random()-1,a=2*t.random()-1,r=o*o+a*a}while(0===r||r>1);i=Math.sqrt(-2*Math.log(r)/r),o*=i,ls=a*i}return e+o*n}function fs(t,e,n){const r=(t-(e||0))/(n=null==n?1:n);return Math.exp(-.5*r*r)/(n*ss)}function hs(t,e,n){const r=(t-(e=e||0))/(n=null==n?1:n),i=Math.abs(r);let o;if(i>37)o=0;else{const t=Math.exp(-i*i/2);let e;i<7.07106781186547?(e=.0352624965998911*i+.700383064443688,e=e*i+6.37396220353165,e=e*i+33.912866078383,e=e*i+112.079291497871,e=e*i+221.213596169931,e=e*i+220.206867912376,o=t*e,e=.0883883476483184*i+1.75566716318264,e=e*i+16.064177579207,e=e*i+86.7807322029461,e=e*i+296.564248779674,e=e*i+637.333633378831,e=e*i+793.826512519948,e=e*i+440.413735824752,o/=e):(e=i+.65,e=i+4/e,e=i+3/e,e=i+2/e,e=i+1/e,o=t/e/2.506628274631)}return r>0?1-o:o}function ds(t,e,n){return t<0||t>1?NaN:(e||0)+(null==n?1:n)*us*function(t){let e,n=-Math.log((1-t)*(1+t));n<6.25?(n-=3.125,e=-364441206401782e-35,e=e*n-16850591381820166e-35,e=128584807152564e-32+e*n,e=11157877678025181e-33+e*n,e=e*n-1333171662854621e-31,e=20972767875968562e-33+e*n,e=6637638134358324e-30+e*n,e=e*n-4054566272975207e-29,e=e*n-8151934197605472e-29,e=26335093153082323e-28+e*n,e=e*n-12975133253453532e-27,e=e*n-5415412054294628e-26,e=1.0512122733215323e-9+e*n,e=e*n-4.112633980346984e-9,e=e*n-2.9070369957882005e-8,e=4.2347877827932404e-7+e*n,e=e*n-13654692000834679e-22,e=e*n-13882523362786469e-21,e=.00018673420803405714+e*n,e=e*n-.000740702534166267,e=e*n-.006033670871430149,e=.24015818242558962+e*n,e=1.6536545626831027+e*n):n<16?(n=Math.sqrt(n)-3.25,e=2.2137376921775787e-9,e=9.075656193888539e-8+e*n,e=e*n-2.7517406297064545e-7,e=1.8239629214389228e-8+e*n,e=15027403968909828e-22+e*n,e=e*n-4013867526981546e-21,e=29234449089955446e-22+e*n,e=12475304481671779e-21+e*n,e=e*n-47318229009055734e-21,e=6828485145957318e-20+e*n,e=24031110387097894e-21+e*n,e=e*n-.0003550375203628475,e=.0009532893797373805+e*n,e=e*n-.0016882755560235047,e=.002491442096107851+e*n,e=e*n-.003751208507569241,e=.005370914553590064+e*n,e=1.0052589676941592+e*n,e=3.0838856104922208+e*n):Number.isFinite(n)?(n=Math.sqrt(n)-5,e=-27109920616438573e-27,e=e*n-2.555641816996525e-10,e=1.5076572693500548e-9+e*n,e=e*n-3.789465440126737e-9,e=7.61570120807834e-9+e*n,e=e*n-1.496002662714924e-8,e=2.914795345090108e-8+e*n,e=e*n-6.771199775845234e-8,e=2.2900482228026655e-7+e*n,e=e*n-9.9298272942317e-7,e=4526062597223154e-21+e*n,e=e*n-1968177810553167e-20,e=7599527703001776e-20+e*n,e=e*n-.00021503011930044477,e=e*n-.00013871931833623122,e=1.0103004648645344+e*n,e=4.849906401408584+e*n):e=1/0;return e*t}(2*t-1)}function ps(t,e){let n,r;const i={mean(t){return arguments.length?(n=t||0,i):n},stdev(t){return arguments.length?(r=null==t?1:t,i):r},sample:()=>cs(n,r),pdf:t=>fs(t,n,r),cdf:t=>hs(t,n,r),icdf:t=>ds(t,n,r)};return i.mean(t).stdev(e)}function gs(e,n){const r=ps();let i=0;const o={data(t){return arguments.length?(e=t,i=t?t.length:0,o.bandwidth(n)):e},bandwidth(t){return arguments.length?(!(n=t)&&e&&(n=rs(e)),o):n},sample:()=>e[~~(t.random()*i)]+n*r.sample(),pdf(t){let o=0,a=0;for(;ams(n,r),pdf:t=>ys(t,n,r),cdf:t=>vs(t,n,r),icdf:t=>_s(t,n,r)};return i.mean(t).stdev(e)}function bs(e,n){let r,i=0;const o={weights(t){return arguments.length?(r=function(t){const e=[];let n,r=0;for(n=0;n=e&&t<=n?1/(n-e):0}function As(t,e,n){return null==n&&(n=null==e?1:e,e=0),tn?1:(t-e)/(n-e)}function Ms(t,e,n){return null==n&&(n=null==e?1:e,e=0),t>=0&&t<=1?e+t*(n-e):NaN}function Es(t,e){let n,r;const i={min(t){return arguments.length?(n=t||0,i):n},max(t){return arguments.length?(r=null==t?1:t,i):r},sample:()=>ws(n,r),pdf:t=>ks(t,n,r),cdf:t=>As(t,n,r),icdf:t=>Ms(t,n,r)};return null==e&&(e=null==t?1:t,t=0),i.min(t).max(e)}function Ds(t,e,n){let r=0,i=0;for(const o of t){const t=n(o);null==e(o)||null==t||isNaN(t)||(r+=(t-r)/++i)}return{coef:[r],predict:()=>r,rSquared:0}}function Cs(t,e,n,r){const i=r-t*t,o=Math.abs(i)<1e-24?0:(n-t*e)/i;return[e-o*t,o]}function Fs(t,e,n,r){t=t.filter((t=>{let r=e(t),i=n(t);return null!=r&&(r=+r)>=r&&null!=i&&(i=+i)>=i})),r&&t.sort(((t,n)=>e(t)-e(n)));const i=t.length,o=new Float64Array(i),a=new Float64Array(i);let s,u,l,c=0,f=0,h=0;for(l of t)o[c]=s=+e(l),a[c]=u=+n(l),++c,f+=(s-f)/c,h+=(u-h)/c;for(c=0;c=i&&null!=o&&(o=+o)>=o&&r(i,o,++a)}function $s(t,e,n,r,i){let o=0,a=0;return Ss(t,e,n,((t,e)=>{const n=e-i(t),s=e-r;o+=n*n,a+=s*s})),1-o/a}function Ts(t,e,n){let r=0,i=0,o=0,a=0,s=0;Ss(t,e,n,((t,e)=>{++s,r+=(t-r)/s,i+=(e-i)/s,o+=(t*e-o)/s,a+=(t*t-a)/s}));const u=Cs(r,i,o,a),l=t=>u[0]+u[1]*t;return{coef:u,predict:l,rSquared:$s(t,e,n,i,l)}}function Bs(t,e,n){let r=0,i=0,o=0,a=0,s=0;Ss(t,e,n,((t,e)=>{++s,t=Math.log(t),r+=(t-r)/s,i+=(e-i)/s,o+=(t*e-o)/s,a+=(t*t-a)/s}));const u=Cs(r,i,o,a),l=t=>u[0]+u[1]*Math.log(t);return{coef:u,predict:l,rSquared:$s(t,e,n,i,l)}}function zs(t,e,n){const[r,i,o,a]=Fs(t,e,n);let s,u,l,c=0,f=0,h=0,d=0,p=0;Ss(t,e,n,((t,e)=>{s=r[p++],u=Math.log(e),l=s*e,c+=(e*u-c)/p,f+=(l-f)/p,h+=(l*u-h)/p,d+=(s*l-d)/p}));const[g,m]=Cs(f/a,c/a,h/a,d/a),y=t=>Math.exp(g+m*(t-o));return{coef:[Math.exp(g-m*o),m],predict:y,rSquared:$s(t,e,n,a,y)}}function Ns(t,e,n){let r=0,i=0,o=0,a=0,s=0,u=0;Ss(t,e,n,((t,e)=>{const n=Math.log(t),l=Math.log(e);++u,r+=(n-r)/u,i+=(l-i)/u,o+=(n*l-o)/u,a+=(n*n-a)/u,s+=(e-s)/u}));const l=Cs(r,i,o,a),c=t=>l[0]*Math.pow(t,l[1]);return l[0]=Math.exp(l[0]),{coef:l,predict:c,rSquared:$s(t,e,n,s,c)}}function Os(t,e,n){const[r,i,o,a]=Fs(t,e,n),s=r.length;let u,l,c,f,h=0,d=0,p=0,g=0,m=0;for(u=0;u_*(t-=o)*t+x*t+b+a;return{coef:[b-x*o+_*o*o+a,x-2*_*o,_],predict:w,rSquared:$s(t,e,n,a,w)}}function Rs(t,e,n,r){if(0===r)return Ds(t,e,n);if(1===r)return Ts(t,e,n);if(2===r)return Os(t,e,n);const[i,o,a,s]=Fs(t,e,n),u=i.length,l=[],c=[],f=r+1;let h,d,p,g,m;for(h=0;hMath.abs(t[r][a])&&(a=i);for(o=r;o=r;o--)t[o][i]-=t[o][r]*t[r][i]/t[r][r]}for(i=e-1;i>=0;--i){for(s=0,o=i+1;o{t-=a;let e=s+y[0]+y[1]*t+y[2]*t*t;for(h=3;h=0;--o)for(s=e[o],u=1,i[o]+=s,a=1;a<=o;++a)u*=(o+1-a)/a,i[o-a]+=s*Math.pow(n,a)*u;return i[0]+=r,i}function Ls(t,e,n,r){const[i,o,a,s]=Fs(t,e,n,!0),u=i.length,l=Math.max(2,~~(r*u)),c=new Float64Array(u),f=new Float64Array(u),h=new Float64Array(u).fill(1);for(let t=-1;++t<=2;){const e=[0,l-1];for(let t=0;ti[a]-n?r:a;let u=0,l=0,d=0,p=0,g=0;const m=1/Math.abs(i[s]-n||1);for(let t=r;t<=a;++t){const e=i[t],r=o[t],a=qs(Math.abs(n-e)*m)*h[t],s=e*a;u+=a,l+=s,d+=r*a,p+=r*s,g+=e*s}const[y,v]=Cs(l/u,d/u,p/u,g/u);c[t]=y+v*n,f[t]=Math.abs(o[t]-c[t]),Ps(i,t+1,e)}if(2===t)break;const n=Ce(f);if(Math.abs(n)<1e-12)break;for(let t,e,r=0;r=1?1e-12:(e=1-t*t)*e}return function(t,e,n,r){const i=t.length,o=[];let a,s=0,u=0,l=[];for(;s=t.length))for(;e>i&&t[o]-r<=r-t[i];)n[0]=++i,n[1]=o,++o}const js=.5*Math.PI/180;function Is(t,e,n,r){n=n||25,r=Math.max(n,r||200);const i=e=>[e,t(e)],o=e[0],a=e[1],s=a-o,u=s/r,l=[i(o)],c=[];if(n===r){for(let t=1;t0;)c.push(i(o+t/n*s));let f=l[0],h=c[c.length-1];const d=1/s,p=function(t,e){let n=t,r=t;const i=e.length;for(let t=0;tr&&(r=i)}return 1/(r-n)}(f[1],c);for(;h;){const t=i((f[0]+h[0])/2);t[0]-f[0]>=u&&Ws(f,t,h,d,p)>js?c.push(t):(f=h,l.push(h),c.pop()),h=c[c.length-1]}return l}function Ws(t,e,n,r,i){const o=Math.atan2(i*(n[1]-t[1]),r*(n[0]-t[0])),a=Math.atan2(i*(e[1]-t[1]),r*(e[0]-t[0]));return Math.abs(o-a)}function Hs(t){return t&&t.length?1===t.length?t[0]:(e=t,t=>{const n=e.length;let r=1,i=String(e[0](t));for(;r{},Vs={init:Gs,add:Gs,rem:Gs,idx:0},Xs={values:{init:t=>t.cell.store=!0,value:t=>t.cell.data.values(),idx:-1},count:{value:t=>t.cell.num},__count__:{value:t=>t.missing+t.valid},missing:{value:t=>t.missing},valid:{value:t=>t.valid},sum:{init:t=>t.sum=0,value:t=>t.valid?t.sum:void 0,add:(t,e)=>t.sum+=+e,rem:(t,e)=>t.sum-=e},product:{init:t=>t.product=1,value:t=>t.valid?t.product:void 0,add:(t,e)=>t.product*=e,rem:(t,e)=>t.product/=e},mean:{init:t=>t.mean=0,value:t=>t.valid?t.mean:void 0,add:(t,e)=>(t.mean_d=e-t.mean,t.mean+=t.mean_d/t.valid),rem:(t,e)=>(t.mean_d=e-t.mean,t.mean-=t.valid?t.mean_d/t.valid:t.mean)},average:{value:t=>t.valid?t.mean:void 0,req:[\"mean\"],idx:1},variance:{init:t=>t.dev=0,value:t=>t.valid>1?t.dev/(t.valid-1):void 0,add:(t,e)=>t.dev+=t.mean_d*(e-t.mean),rem:(t,e)=>t.dev-=t.mean_d*(e-t.mean),req:[\"mean\"],idx:1},variancep:{value:t=>t.valid>1?t.dev/t.valid:void 0,req:[\"variance\"],idx:2},stdev:{value:t=>t.valid>1?Math.sqrt(t.dev/(t.valid-1)):void 0,req:[\"variance\"],idx:2},stdevp:{value:t=>t.valid>1?Math.sqrt(t.dev/t.valid):void 0,req:[\"variance\"],idx:2},stderr:{value:t=>t.valid>1?Math.sqrt(t.dev/(t.valid*(t.valid-1))):void 0,req:[\"variance\"],idx:2},distinct:{value:t=>t.cell.data.distinct(t.get),req:[\"values\"],idx:3},ci0:{value:t=>t.cell.data.ci0(t.get),req:[\"values\"],idx:3},ci1:{value:t=>t.cell.data.ci1(t.get),req:[\"values\"],idx:3},median:{value:t=>t.cell.data.q2(t.get),req:[\"values\"],idx:3},q1:{value:t=>t.cell.data.q1(t.get),req:[\"values\"],idx:3},q3:{value:t=>t.cell.data.q3(t.get),req:[\"values\"],idx:3},min:{init:t=>t.min=void 0,value:t=>t.min=Number.isNaN(t.min)?t.cell.data.min(t.get):t.min,add:(t,e)=>{(e{e<=t.min&&(t.min=NaN)},req:[\"values\"],idx:4},max:{init:t=>t.max=void 0,value:t=>t.max=Number.isNaN(t.max)?t.cell.data.max(t.get):t.max,add:(t,e)=>{(e>t.max||void 0===t.max)&&(t.max=e)},rem:(t,e)=>{e>=t.max&&(t.max=NaN)},req:[\"values\"],idx:4},argmin:{init:t=>t.argmin=void 0,value:t=>t.argmin||t.cell.data.argmin(t.get),add:(t,e,n)=>{e{e<=t.min&&(t.argmin=void 0)},req:[\"min\",\"values\"],idx:3},argmax:{init:t=>t.argmax=void 0,value:t=>t.argmax||t.cell.data.argmax(t.get),add:(t,e,n)=>{e>t.max&&(t.argmax=n)},rem:(t,e)=>{e>=t.max&&(t.argmax=void 0)},req:[\"max\",\"values\"],idx:3},exponential:{init:(t,e)=>{t.exp=0,t.exp_r=e},value:t=>t.valid?t.exp*(1-t.exp_r)/(1-t.exp_r**t.valid):void 0,add:(t,e)=>t.exp=t.exp_r*t.exp+e,rem:(t,e)=>t.exp=(t.exp-e/t.exp_r**(t.valid-1))/t.exp_r},exponentialb:{value:t=>t.valid?t.exp*(1-t.exp_r):void 0,req:[\"exponential\"],idx:1}},Js=Object.keys(Xs).filter((t=>\"__count__\"!==t));function Zs(t,e,n){return Xs[t](n,e)}function Qs(t,e){return t.idx-e.idx}function Ks(){this.valid=0,this.missing=0,this._ops.forEach((t=>null==t.aggregate_param?t.init(this):t.init(this,t.aggregate_param)))}function tu(t,e){null!=t&&\"\"!==t?t==t&&(++this.valid,this._ops.forEach((n=>n.add(this,t,e)))):++this.missing}function eu(t,e){null!=t&&\"\"!==t?t==t&&(--this.valid,this._ops.forEach((n=>n.rem(this,t,e)))):--this.missing}function nu(t){return this._out.forEach((e=>t[e.out]=e.value(this))),t}function ru(t,e){const n=e||f,r=function(t){const e={};t.forEach((t=>e[t.name]=t));const n=t=>{t.req&&t.req.forEach((t=>{e[t]||n(e[t]=Xs[t]())}))};return t.forEach(n),Object.values(e).sort(Qs)}(t),i=t.slice().sort(Qs);function o(t){this._ops=r,this._out=i,this.cell=t,this.init()}return o.prototype.init=Ks,o.prototype.add=tu,o.prototype.rem=eu,o.prototype.set=nu,o.prototype.get=n,o.fields=t.map((t=>t.out)),o}function iu(t){this._key=t?l(t):ya,this.reset()}[...Js,\"__count__\"].forEach((t=>{Xs[t]=function(t,e){return(n,r)=>ot({name:t,aggregate_param:r,out:n||t},Vs,e)}(t,Xs[t])}));const ou=iu.prototype;function au(t){Ja.call(this,null,t),this._adds=[],this._mods=[],this._alen=0,this._mlen=0,this._drop=!0,this._cross=!1,this._dims=[],this._dnames=[],this._measures=[],this._countOnly=!1,this._counts=null,this._prev=null,this._inputs=null,this._outputs=null}ou.reset=function(){this._add=[],this._rem=[],this._ext=null,this._get=null,this._q=null},ou.add=function(t){this._add.push(t)},ou.rem=function(t){this._rem.push(t)},ou.values=function(){if(this._get=null,0===this._rem.length)return this._add;const t=this._add,e=this._rem,n=this._key,r=t.length,i=e.length,o=Array(r-i),a={};let s,u,l;for(s=0;s=0;)r=t(e[i])+\"\",lt(n,r)||(n[r]=1,++o);return o},ou.extent=function(t){if(this._get!==t||!this._ext){const e=this.values(),n=st(e,t);this._ext=[e[n[0]],e[n[1]]],this._get=t}return this._ext},ou.argmin=function(t){return this.extent(t)[0]||{}},ou.argmax=function(t){return this.extent(t)[1]||{}},ou.min=function(t){const e=this.extent(t)[0];return null!=e?t(e):void 0},ou.max=function(t){const e=this.extent(t)[1];return null!=e?t(e):void 0},ou.quartile=function(t){return this._get===t&&this._q||(this._q=ns(this.values(),t),this._get=t),this._q},ou.q1=function(t){return this.quartile(t)[0]},ou.q2=function(t){return this.quartile(t)[1]},ou.q3=function(t){return this.quartile(t)[2]},ou.ci=function(t){return this._get===t&&this._ci||(this._ci=os(this.values(),1e3,.05,t),this._get=t),this._ci},ou.ci0=function(t){return this.ci(t)[0]},ou.ci1=function(t){return this.ci(t)[1]},au.Definition={type:\"Aggregate\",metadata:{generates:!0,changes:!0},params:[{name:\"groupby\",type:\"field\",array:!0},{name:\"ops\",type:\"enum\",array:!0,values:Js},{name:\"aggregate_params\",type:\"number\",null:!0,array:!0},{name:\"fields\",type:\"field\",null:!0,array:!0},{name:\"as\",type:\"string\",null:!0,array:!0},{name:\"drop\",type:\"boolean\",default:!0},{name:\"cross\",type:\"boolean\",default:!1},{name:\"key\",type:\"field\"}]},dt(au,Ja,{transform(t,e){const n=this,r=e.fork(e.NO_SOURCE|e.NO_FIELDS),i=t.modified();return n.stamp=r.stamp,n.value&&(i||e.modified(n._inputs,!0))?(n._prev=n.value,n.value=i?n.init(t):Object.create(null),e.visit(e.SOURCE,(t=>n.add(t)))):(n.value=n.value||n.init(t),e.visit(e.REM,(t=>n.rem(t))),e.visit(e.ADD,(t=>n.add(t)))),r.modifies(n._outputs),n._drop=!1!==t.drop,t.cross&&n._dims.length>1&&(n._drop=!1,n.cross()),e.clean()&&n._drop&&r.clean(!0).runAfter((()=>this.clean())),n.changes(r)},cross(){const t=this,e=t.value,n=t._dnames,r=n.map((()=>({}))),i=n.length;function o(t){let e,o,a,s;for(e in t)for(a=t[e].tuple,o=0;o{const e=n(t);return a(t),i.push(e),e})),this.cellkey=t.key?t.key:Hs(this._dims),this._countOnly=!0,this._counts=[],this._measures=[];const u=t.fields||[null],l=t.ops||[\"count\"],c=t.aggregate_params||[null],f=t.as||[],h=u.length,d={};let p,g,m,y,v,_,x;for(h!==l.length&&s(\"Unmatched number of fields and aggregate ops.\"),x=0;xru(t,t.field))),Object.create(null)},cellkey:Hs(),cell(t,e){let n=this.value[t];return n?0===n.num&&this._drop&&n.stampo.push(t),remove:t=>a[r(t)]=++s,size:()=>i.length,data:(t,e)=>(s&&(i=i.filter((t=>!a[r(t)])),a={},s=0),e&&t&&i.sort(t),o.length&&(i=t?At(t,i,o.sort(t)):i.concat(o),o=[]),i)}}function lu(t){Ja.call(this,[],t)}function cu(t){Sa.call(this,null,fu,t)}function fu(t){return this.value&&!t.modified()?this.value:Q(t.fields,t.orders)}function hu(t){Ja.call(this,null,t)}function du(t){Ja.call(this,null,t)}su.Definition={type:\"Bin\",metadata:{modifies:!0},params:[{name:\"field\",type:\"field\",required:!0},{name:\"interval\",type:\"boolean\",default:!0},{name:\"anchor\",type:\"number\"},{name:\"maxbins\",type:\"number\",default:20},{name:\"base\",type:\"number\",default:10},{name:\"divide\",type:\"number\",array:!0,default:[5,2]},{name:\"extent\",type:\"number\",array:!0,length:2,required:!0},{name:\"span\",type:\"number\"},{name:\"step\",type:\"number\"},{name:\"steps\",type:\"number\",array:!0},{name:\"minstep\",type:\"number\",default:0},{name:\"nice\",type:\"boolean\",default:!0},{name:\"name\",type:\"string\"},{name:\"as\",type:\"string\",array:!0,length:2,default:[\"bin0\",\"bin1\"]}]},dt(su,Ja,{transform(t,e){const n=!1!==t.interval,i=this._bins(t),o=i.start,a=i.step,s=t.as||[\"bin0\",\"bin1\"],u=s[0],l=s[1];let c;return c=t.modified()?(e=e.reflow(!0)).SOURCE:e.modified(r(t.field))?e.ADD_MOD:e.ADD,e.visit(c,n?t=>{const e=i(t);t[u]=e,t[l]=null==e?null:o+a*(1+(e-o)/a)}:t=>t[u]=i(t)),e.modifies(n?s:u)},_bins(t){if(this.value&&!t.modified())return this.value;const i=t.field,o=is(t),a=o.step;let s,u,l=o.start,c=l+Math.ceil((o.stop-l)/a)*a;null!=(s=t.anchor)&&(u=s-(l+a*Math.floor((s-l)/a)),l+=u,c+=u);const f=function(t){let e=S(i(t));return null==e?null:ec?1/0:(e=Math.max(l,Math.min(e,c-a)),l+a*Math.floor(1e-14+(e-l)/a))};return f.start=l,f.stop=o.stop,f.step=a,this.value=e(f,r(i),t.name||\"bin_\"+n(i))}}),lu.Definition={type:\"Collect\",metadata:{source:!0},params:[{name:\"sort\",type:\"compare\"}]},dt(lu,Ja,{transform(t,e){const n=e.fork(e.ALL),r=uu(ya,this.value,n.materialize(n.ADD).add),i=t.sort,o=e.changed()||i&&(t.modified(\"sort\")||e.modified(i.fields));return n.visit(n.REM,r.remove),this.modified(o),this.value=n.source=r.data(ka(i),o),e.source&&e.source.root&&(this.value.root=e.source.root),n}}),dt(cu,Sa),hu.Definition={type:\"CountPattern\",metadata:{generates:!0,changes:!0},params:[{name:\"field\",type:\"field\",required:!0},{name:\"case\",type:\"enum\",values:[\"upper\",\"lower\",\"mixed\"],default:\"mixed\"},{name:\"pattern\",type:\"string\",default:'[\\\\w\"]+'},{name:\"stopwords\",type:\"string\",default:\"\"},{name:\"as\",type:\"string\",array:!0,length:2,default:[\"text\",\"count\"]}]},dt(hu,Ja,{transform(t,e){const n=e=>n=>{for(var r,i=function(t,e,n){switch(e){case\"upper\":t=t.toUpperCase();break;case\"lower\":t=t.toLowerCase()}return t.match(n)}(s(n),t.case,o)||[],u=0,l=i.length;ui[t]=1+(i[t]||0))),c=n((t=>i[t]-=1));return r?e.visit(e.SOURCE,l):(e.visit(e.ADD,l),e.visit(e.REM,c)),this._finish(e,u)},_parameterCheck(t,e){let n=!1;return!t.modified(\"stopwords\")&&this._stop||(this._stop=new RegExp(\"^\"+(t.stopwords||\"\")+\"$\",\"i\"),n=!0),!t.modified(\"pattern\")&&this._match||(this._match=new RegExp(t.pattern||\"[\\\\w']+\",\"g\"),n=!0),(t.modified(\"field\")||e.modified(t.field.fields))&&(n=!0),n&&(this._counts={}),n},_finish(t,e){const n=this._counts,r=this._tuples||(this._tuples={}),i=e[0],o=e[1],a=t.fork(t.NO_SOURCE|t.NO_FIELDS);let s,u,l;for(s in n)u=r[s],l=n[s]||0,!u&&l?(r[s]=u=_a({}),u[i]=s,u[o]=l,a.add.push(u)):0===l?(u&&a.rem.push(u),n[s]=null,r[s]=null):u[o]!==l&&(u[o]=l,a.mod.push(u));return a.modifies(e)}}),du.Definition={type:\"Cross\",metadata:{generates:!0},params:[{name:\"filter\",type:\"expr\"},{name:\"as\",type:\"string\",array:!0,length:2,default:[\"a\",\"b\"]}]},dt(du,Ja,{transform(t,e){const n=e.fork(e.NO_SOURCE),r=t.as||[\"a\",\"b\"],i=r[0],o=r[1],a=!this.value||e.changed(e.ADD_REM)||t.modified(\"as\")||t.modified(\"filter\");let s=this.value;return a?(s&&(n.rem=s),s=e.materialize(e.SOURCE).source,n.add=this.value=function(t,e,n,r){for(var i,o,a=[],s={},u=t.length,l=0;lmu(t,e)))):typeof r[n]===gu&&r[n](t[n]);return r}function yu(t){Ja.call(this,null,t)}const vu=[{key:{function:\"normal\"},params:[{name:\"mean\",type:\"number\",default:0},{name:\"stdev\",type:\"number\",default:1}]},{key:{function:\"lognormal\"},params:[{name:\"mean\",type:\"number\",default:0},{name:\"stdev\",type:\"number\",default:1}]},{key:{function:\"uniform\"},params:[{name:\"min\",type:\"number\",default:0},{name:\"max\",type:\"number\",default:1}]},{key:{function:\"kde\"},params:[{name:\"field\",type:\"field\",required:!0},{name:\"from\",type:\"data\"},{name:\"bandwidth\",type:\"number\",default:0}]}],_u={key:{function:\"mixture\"},params:[{name:\"distributions\",type:\"param\",array:!0,params:vu},{name:\"weights\",type:\"number\",array:!0}]};function xu(t,e){return t?t.map(((t,r)=>e[r]||n(t))):null}function bu(t,e,n){const r=[],i=t=>t(u);let o,a,s,u,l,c;if(null==e)r.push(t.map(n));else for(o={},a=0,s=t.length;at.materialize(t.SOURCE).source}(e)),i=t.steps||t.minsteps||25,o=t.steps||t.maxsteps||200;let a=t.method||\"pdf\";\"pdf\"!==a&&\"cdf\"!==a&&s(\"Invalid density method: \"+a),t.extent||r.data||s(\"Missing density extent parameter.\"),a=r[a];const u=t.as||[\"value\",\"density\"],l=Is(a,t.extent||at(r.data()),i,o).map((t=>{const e={};return e[u[0]]=t[0],e[u[1]]=t[1],_a(e)}));this.value&&(n.rem=this.value),this.value=n.add=n.source=l}return n}});function wu(t){Ja.call(this,null,t)}wu.Definition={type:\"DotBin\",metadata:{modifies:!0},params:[{name:\"field\",type:\"field\",required:!0},{name:\"groupby\",type:\"field\",array:!0},{name:\"step\",type:\"number\"},{name:\"smooth\",type:\"boolean\",default:!1},{name:\"as\",type:\"string\",default:\"bin\"}]};function ku(t){Sa.call(this,null,Au,t),this.modified(!0)}function Au(t){const i=t.expr;return this.value&&!t.modified(\"expr\")?this.value:e((e=>i(e,t)),r(i),n(i))}function Mu(t){Ja.call(this,[void 0,void 0],t)}function Eu(t,e){Sa.call(this,t),this.parent=e,this.count=0}function Du(t){Ja.call(this,{},t),this._keys=ft();const e=this._targets=[];e.active=0,e.forEach=t=>{for(let n=0,r=e.active;nl(t))):l(t.name,t.as)}function Su(t){Ja.call(this,ft(),t)}function $u(t){Ja.call(this,[],t)}function Tu(t){Ja.call(this,[],t)}function Bu(t){Ja.call(this,null,t)}function zu(t){Ja.call(this,[],t)}dt(wu,Ja,{transform(t,e){if(this.value&&!t.modified()&&!e.changed())return e;const n=e.materialize(e.SOURCE).source,r=bu(e.source,t.groupby,f),i=t.smooth||!1,o=t.field,a=t.step||((t,e)=>Dt(at(t,e))/30)(n,o),s=ka(((t,e)=>o(t)-o(e))),u=t.as||\"bin\",l=r.length;let c,h=1/0,d=-1/0,p=0;for(;pd&&(d=e),t[++c][u]=e}return this.value={start:h,stop:d,step:a},e.reflow(!0).modifies(u)}}),dt(ku,Sa),Mu.Definition={type:\"Extent\",metadata:{},params:[{name:\"field\",type:\"field\",required:!0}]},dt(Mu,Ja,{transform(t,e){const r=this.value,i=t.field,o=e.changed()||e.modified(i.fields)||t.modified(\"field\");let a=r[0],s=r[1];if((o||null==a)&&(a=1/0,s=-1/0),e.visit(o?e.SOURCE:e.ADD,(t=>{const e=S(i(t));null!=e&&(es&&(s=e))})),!Number.isFinite(a)||!Number.isFinite(s)){let t=n(i);t&&(t=` for field \"${t}\"`),e.dataflow.warn(`Infinite extent${t}: [${a}, ${s}]`),a=s=void 0}this.value=[a,s]}}),dt(Eu,Sa,{connect(t){return this.detachSubflow=t.detachSubflow,this.targets().add(t),t.source=this},add(t){this.count+=1,this.value.add.push(t)},rem(t){this.count-=1,this.value.rem.push(t)},mod(t){this.value.mod.push(t)},init(t){this.value.init(t,t.NO_SOURCE)},evaluate(){return this.value}}),dt(Du,Ja,{activate(t){this._targets[this._targets.active++]=t},subflow(t,e,n,r){const i=this.value;let o,a,s=lt(i,t)&&i[t];return s?s.value.stampt&&t.count>0));this.initTargets(t)}},initTargets(t){const e=this._targets,n=e.length,r=t?t.length:0;let i=0;for(;ithis.subflow(t,i,e);return this._group=t.group||{},this.initTargets(),e.visit(e.REM,(t=>{const e=ya(t),n=o.get(e);void 0!==n&&(o.delete(e),s(n).rem(t))})),e.visit(e.ADD,(t=>{const e=r(t);o.set(ya(t),e),s(e).add(t)})),a||e.modified(r.fields)?e.visit(e.MOD,(t=>{const e=ya(t),n=o.get(e),i=r(t);n===i?s(i).mod(t):(o.set(e,i),s(n).rem(t),s(i).add(t))})):e.changed(e.MOD)&&e.visit(e.MOD,(t=>{s(o.get(ya(t))).mod(t)})),a&&e.visit(e.REFLOW,(t=>{const e=ya(t),n=o.get(e),i=r(t);n!==i&&(o.set(e,i),s(n).rem(t),s(i).add(t))})),e.clean()?n.runAfter((()=>{this.clean(),o.clean()})):o.empty>n.cleanThreshold&&n.runAfter(o.clean),e}}),dt(Cu,Sa),Su.Definition={type:\"Filter\",metadata:{changes:!0},params:[{name:\"expr\",type:\"expr\",required:!0}]},dt(Su,Ja,{transform(t,e){const n=e.dataflow,r=this.value,i=e.fork(),o=i.add,a=i.rem,s=i.mod,u=t.expr;let l=!0;function c(e){const n=ya(e),i=u(e,t),c=r.get(n);i&&c?(r.delete(n),o.push(e)):i||c?l&&i&&!c&&s.push(e):(r.set(n,1),a.push(e))}return e.visit(e.REM,(t=>{const e=ya(t);r.has(e)?r.delete(e):a.push(t)})),e.visit(e.ADD,(e=>{u(e,t)?o.push(e):r.set(ya(e),1)})),e.visit(e.MOD,c),t.modified()&&(l=!1,e.visit(e.REFLOW,c)),r.empty>n.cleanThreshold&&n.runAfter(r.clean),i}}),$u.Definition={type:\"Flatten\",metadata:{generates:!0},params:[{name:\"fields\",type:\"field\",array:!0,required:!0},{name:\"index\",type:\"string\"},{name:\"as\",type:\"string\",array:!0}]},dt($u,Ja,{transform(t,e){const n=e.fork(e.NO_SOURCE),r=t.fields,i=xu(r,t.as||[]),o=t.index||null,a=i.length;return n.rem=this.value,e.visit(e.SOURCE,(t=>{const e=r.map((e=>e(t))),s=e.reduce(((t,e)=>Math.max(t,e.length)),0);let u,l,c,f=0;for(;f{for(let e,n=0;ne[r]=n(e,t)))}}),dt(zu,Ja,{transform(t,e){const n=e.fork(e.ALL),r=t.generator;let i,o,a,s=this.value,u=t.size-s.length;if(u>0){for(i=[];--u>=0;)i.push(a=_a(r(t))),s.push(a);n.add=n.add.length?n.materialize(n.ADD).add.concat(i):i}else o=s.slice(0,-u),n.rem=n.rem.length?n.materialize(n.REM).rem.concat(o):o,s=s.slice(-u);return n.source=this.value=s,n}});const Nu={value:\"value\",median:Ce,mean:function(t,e){let n=0,r=0;if(void 0===e)for(let e of t)null!=e&&(e=+e)>=e&&(++n,r+=e);else{let i=-1;for(let o of t)null!=(o=e(o,++i,t))&&(o=+o)>=o&&(++n,r+=o)}if(n)return r/n},min:ke,max:we},Ou=[];function Ru(t){Ja.call(this,[],t)}function Uu(t){au.call(this,t)}function Lu(t){Ja.call(this,null,t)}function qu(t){Sa.call(this,null,Pu,t)}function Pu(t){return this.value&&!t.modified()?this.value:bt(t.fields,t.flat)}function ju(t){Ja.call(this,[],t),this._pending=null}function Iu(t,e,n){n.forEach(_a);const r=e.fork(e.NO_FIELDS&e.NO_SOURCE);return r.rem=t.value,t.value=r.source=r.add=n,t._pending=null,r.rem.length&&r.clean(!0),r}function Wu(t){Ja.call(this,{},t)}function Hu(t){Sa.call(this,null,Yu,t)}function Yu(t){if(this.value&&!t.modified())return this.value;const e=t.extents,n=e.length;let r,i,o=1/0,a=-1/0;for(r=0;ra&&(a=i[1]);return[o,a]}function Gu(t){Sa.call(this,null,Vu,t)}function Vu(t){return this.value&&!t.modified()?this.value:t.values.reduce(((t,e)=>t.concat(e)),[])}function Xu(t){Ja.call(this,null,t)}function Ju(t){au.call(this,t)}function Zu(t){Du.call(this,t)}function Qu(t){Ja.call(this,null,t)}function Ku(t){Ja.call(this,null,t)}function tl(t){Ja.call(this,null,t)}Ru.Definition={type:\"Impute\",metadata:{changes:!0},params:[{name:\"field\",type:\"field\",required:!0},{name:\"key\",type:\"field\",required:!0},{name:\"keyvals\",array:!0},{name:\"groupby\",type:\"field\",array:!0},{name:\"method\",type:\"enum\",default:\"value\",values:[\"value\",\"mean\",\"median\",\"max\",\"min\"]},{name:\"value\",default:0}]},dt(Ru,Ja,{transform(t,e){var r,i,o,a,u,l,c,f,h,d,p=e.fork(e.ALL),g=function(t){var e,n=t.method||Nu.value;if(null!=Nu[n])return n===Nu.value?(e=void 0!==t.value?t.value:0,()=>e):Nu[n];s(\"Unrecognized imputation method: \"+n)}(t),m=function(t){const e=t.field;return t=>t?e(t):NaN}(t),y=n(t.field),v=n(t.key),_=(t.groupby||[]).map(n),x=function(t,e,n,r){var i,o,a,s,u,l,c,f,h=t=>t(f),d=[],p=r?r.slice():[],g={},m={};for(p.forEach(((t,e)=>g[t]=e+1)),s=0,c=t.length;sn.add(t)))):(i=n.value=n.value||this.init(t),e.visit(e.REM,(t=>n.rem(t))),e.visit(e.ADD,(t=>n.add(t)))),n.changes(),e.visit(e.SOURCE,(t=>{ot(t,i[n.cellkey(t)].tuple)})),e.reflow(r).modifies(this._outputs)},changes(){const t=this._adds,e=this._mods;let n,r;for(n=0,r=this._alen;n{const n=gs(e,u)[l],r=t.counts?e.length:1;Is(n,h||at(e),d,p).forEach((t=>{const n={};for(let t=0;t(this._pending=V(t.data),t=>t.touch(this))));return{async:e}}return n.request(t.url,t.format).then((t=>Iu(this,e,V(t.data))))}}),Wu.Definition={type:\"Lookup\",metadata:{modifies:!0},params:[{name:\"index\",type:\"index\",params:[{name:\"from\",type:\"data\",required:!0},{name:\"key\",type:\"field\",required:!0}]},{name:\"values\",type:\"field\",array:!0},{name:\"fields\",type:\"field\",array:!0,required:!0},{name:\"as\",type:\"string\",array:!0},{name:\"default\",default:null}]},dt(Wu,Ja,{transform(t,e){const r=t.fields,i=t.index,o=t.values,a=null==t.default?null:t.default,u=t.modified(),l=r.length;let c,f,h,d=u?e.SOURCE:e.ADD,p=e,g=t.as;return o?(f=o.length,l>1&&!g&&s('Multi-field lookup requires explicit \"as\" parameter.'),g&&g.length!==l*f&&s('The \"as\" parameter has too few output field names.'),g=g||o.map(n),c=function(t){for(var e,n,s=0,u=0;se.modified(t.fields))),d|=h?e.MOD:0),e.visit(d,c),p.modifies(g)}}),dt(Hu,Sa),dt(Gu,Sa),dt(Xu,Ja,{transform(t,e){return this.modified(t.modified()),this.value=t,e.fork(e.NO_SOURCE|e.NO_FIELDS)}}),Ju.Definition={type:\"Pivot\",metadata:{generates:!0,changes:!0},params:[{name:\"groupby\",type:\"field\",array:!0},{name:\"field\",type:\"field\",required:!0},{name:\"value\",type:\"field\",required:!0},{name:\"op\",type:\"enum\",values:Js,default:\"sum\"},{name:\"limit\",type:\"number\",default:0},{name:\"key\",type:\"field\"}]},dt(Ju,au,{_transform:au.prototype.transform,transform(t,n){return this._transform(function(t,n){const i=t.field,o=t.value,a=(\"count\"===t.op?\"__count__\":t.op)||\"sum\",s=r(i).concat(r(o)),u=function(t,e,n){const r={},i=[];return n.visit(n.SOURCE,(e=>{const n=t(e);r[n]||(r[n]=1,i.push(n))})),i.sort(K),e?i.slice(0,e):i}(i,t.limit||0,n);n.changed()&&t.set(\"__pivot__\",null,null,!0);return{key:t.key,groupby:t.groupby,ops:u.map((()=>a)),fields:u.map((t=>function(t,n,r,i){return e((e=>n(e)===t?r(e):NaN),i,t+\"\")}(t,i,o,s))),as:u.map((t=>t+\"\")),modified:t.modified.bind(t)}}(t,n),n)}}),dt(Zu,Du,{transform(t,e){const n=t.subflow,i=t.field,o=t=>this.subflow(ya(t),n,e,t);return(t.modified(\"field\")||i&&e.modified(r(i)))&&s(\"PreFacet does not support field modification.\"),this.initTargets(),i?(e.visit(e.MOD,(t=>{const e=o(t);i(t).forEach((t=>e.mod(t)))})),e.visit(e.ADD,(t=>{const e=o(t);i(t).forEach((t=>e.add(_a(t))))})),e.visit(e.REM,(t=>{const e=o(t);i(t).forEach((t=>e.rem(t)))}))):(e.visit(e.MOD,(t=>o(t).mod(t))),e.visit(e.ADD,(t=>o(t).add(t))),e.visit(e.REM,(t=>o(t).rem(t)))),e.clean()&&e.runAfter((()=>this.clean())),e}}),Qu.Definition={type:\"Project\",metadata:{generates:!0,changes:!0},params:[{name:\"fields\",type:\"field\",array:!0},{name:\"as\",type:\"string\",null:!0,array:!0}]},dt(Qu,Ja,{transform(t,e){const n=e.fork(e.NO_SOURCE),r=t.fields,i=xu(t.fields,t.as||[]),o=r?(t,e)=>function(t,e,n,r){for(let i=0,o=n.length;i{const e=ya(t);n.rem.push(a[e]),a[e]=null})),e.visit(e.ADD,(t=>{const e=o(t,_a({}));a[ya(t)]=e,n.add.push(e)})),e.visit(e.MOD,(t=>{n.mod.push(o(t,a[ya(t)]))})),n}}),dt(Ku,Ja,{transform(t,e){return this.value=t.value,t.modified(\"value\")?e.fork(e.NO_SOURCE|e.NO_FIELDS):e.StopPropagation}}),tl.Definition={type:\"Quantile\",metadata:{generates:!0,changes:!0},params:[{name:\"groupby\",type:\"field\",array:!0},{name:\"field\",type:\"field\",required:!0},{name:\"probs\",type:\"number\",array:!0},{name:\"step\",type:\"number\",default:.01},{name:\"as\",type:\"string\",array:!0,default:[\"prob\",\"value\"]}]};function el(t){Ja.call(this,null,t)}function nl(t){Ja.call(this,[],t),this.count=0}function rl(t){Ja.call(this,null,t)}function il(t){Ja.call(this,null,t),this.modified(!0)}function ol(t){Ja.call(this,null,t)}dt(tl,Ja,{transform(t,e){const r=e.fork(e.NO_SOURCE|e.NO_FIELDS),i=t.as||[\"prob\",\"value\"];if(this.value&&!t.modified()&&!e.changed())return r.source=this.value,r;const o=bu(e.materialize(e.SOURCE).source,t.groupby,t.field),a=(t.groupby||[]).map(n),s=[],u=t.step||.01,l=t.probs||Se(u/2,1-1e-14,u),c=l.length;return o.forEach((t=>{const e=es(t,l);for(let n=0;n{const e=ya(t);n.rem.push(r[e]),r[e]=null})),e.visit(e.ADD,(t=>{const e=xa(t);r[ya(t)]=e,n.add.push(e)})),e.visit(e.MOD,(t=>{const e=r[ya(t)];for(const r in t)e[r]=t[r],n.modifies(r);n.mod.push(e)}))),n}}),nl.Definition={type:\"Sample\",metadata:{},params:[{name:\"size\",type:\"number\",default:1e3}]},dt(nl,Ja,{transform(e,n){const r=n.fork(n.NO_SOURCE),i=e.modified(\"size\"),o=e.size,a=this.value.reduce(((t,e)=>(t[ya(e)]=1,t)),{});let s=this.value,u=this.count,l=0;function c(e){let n,i;s.length=l&&(n=s[i],a[ya(n)]&&r.rem.push(n),s[i]=e)),++u}if(n.rem.length&&(n.visit(n.REM,(t=>{const e=ya(t);a[e]&&(a[e]=-1,r.rem.push(t)),--u})),s=s.filter((t=>-1!==a[ya(t)]))),(n.rem.length||i)&&s.length{a[ya(t)]||c(t)})),l=-1),i&&s.length>o){const t=s.length-o;for(let e=0;e{a[ya(t)]&&r.mod.push(t)})),n.add.length&&n.visit(n.ADD,c),(n.add.length||l<0)&&(r.add=s.filter((t=>!a[ya(t)]))),this.count=u,this.value=r.source=s,r}}),rl.Definition={type:\"Sequence\",metadata:{generates:!0,changes:!0},params:[{name:\"start\",type:\"number\",required:!0},{name:\"stop\",type:\"number\",required:!0},{name:\"step\",type:\"number\",default:1},{name:\"as\",type:\"string\",default:\"data\"}]},dt(rl,Ja,{transform(t,e){if(this.value&&!t.modified())return;const n=e.materialize().fork(e.MOD),r=t.as||\"data\";return n.rem=this.value?e.rem.concat(this.value):e.rem,this.value=Se(t.start,t.stop,t.step||1).map((t=>{const e={};return e[r]=t,_a(e)})),n.add=e.add.concat(this.value),n}}),dt(il,Ja,{transform(t,e){return this.value=e.source,e.changed()?e.fork(e.NO_SOURCE|e.NO_FIELDS):e.StopPropagation}});const al=[\"unit0\",\"unit1\"];function sl(t){Ja.call(this,ft(),t)}function ul(t){Ja.call(this,null,t)}ol.Definition={type:\"TimeUnit\",metadata:{modifies:!0},params:[{name:\"field\",type:\"field\",required:!0},{name:\"interval\",type:\"boolean\",default:!0},{name:\"units\",type:\"enum\",values:Kn,array:!0},{name:\"step\",type:\"number\",default:1},{name:\"maxbins\",type:\"number\",default:40},{name:\"extent\",type:\"date\",array:!0},{name:\"timezone\",type:\"enum\",default:\"local\",values:[\"local\",\"utc\"]},{name:\"as\",type:\"string\",array:!0,length:2,default:al}]},dt(ol,Ja,{transform(t,e){const n=t.field,i=!1!==t.interval,o=\"utc\"===t.timezone,a=this._floor(t,e),s=(o?Fr:Cr)(a.unit).offset,u=t.as||al,l=u[0],c=u[1],f=a.step;let h=a.start||1/0,d=a.stop||-1/0,p=e.ADD;return(t.modified()||e.changed(e.REM)||e.modified(r(n)))&&(p=(e=e.reflow(!0)).SOURCE,h=1/0,d=-1/0),e.visit(p,(t=>{const e=n(t);let r,o;null==e?(t[l]=null,i&&(t[c]=null)):(t[l]=r=o=a(e),i&&(t[c]=o=s(r,f)),rd&&(d=o))})),a.start=h,a.stop=d,e.modifies(i?u:l)},_floor(t,e){const n=\"utc\"===t.timezone,{units:r,step:i}=t.units?{units:t.units,step:t.step||1}:Jr({extent:t.extent||at(e.materialize(e.SOURCE).source,t.field),maxbins:t.maxbins}),o=er(r),a=this.value||{},s=(n?Mr:wr)(o,i);return s.unit=F(o),s.units=o,s.step=i,s.start=a.start,s.stop=a.stop,this.value=s}}),dt(sl,Ja,{transform(t,e){const n=e.dataflow,r=t.field,i=this.value,o=t=>i.set(r(t),t);let a=!0;return t.modified(\"field\")||e.modified(r.fields)?(i.clear(),e.visit(e.SOURCE,o)):e.changed()?(e.visit(e.REM,(t=>i.delete(r(t)))),e.visit(e.ADD,o)):a=!1,this.modified(a),i.empty>n.cleanThreshold&&n.runAfter(i.clean),e.fork()}}),dt(ul,Ja,{transform(t,e){(!this.value||t.modified(\"field\")||t.modified(\"sort\")||e.changed()||t.sort&&e.modified(t.sort.fields))&&(this.value=(t.sort?e.source.slice().sort(ka(t.sort)):e.source).map(t.field))}});const ll={row_number:function(){return{next:t=>t.index+1}},rank:function(){let t;return{init:()=>t=1,next:e=>{const n=e.index,r=e.data;return n&&e.compare(r[n-1],r[n])?t=n+1:t}}},dense_rank:function(){let t;return{init:()=>t=1,next:e=>{const n=e.index,r=e.data;return n&&e.compare(r[n-1],r[n])?++t:t}}},percent_rank:function(){const t=ll.rank(),e=t.next;return{init:t.init,next:t=>(e(t)-1)/(t.data.length-1)}},cume_dist:function(){let t;return{init:()=>t=0,next:e=>{const n=e.data,r=e.compare;let i=e.index;if(t0||s(\"ntile num must be greater than zero.\");const n=ll.cume_dist(),r=n.next;return{init:n.init,next:t=>Math.ceil(e*r(t))}},lag:function(t,e){return e=+e||1,{next:n=>{const r=n.index-e;return r>=0?t(n.data[r]):null}}},lead:function(t,e){return e=+e||1,{next:n=>{const r=n.index+e,i=n.data;return rt(e.data[e.i0])}},last_value:function(t){return{next:e=>t(e.data[e.i1-1])}},nth_value:function(t,e){return(e=+e)>0||s(\"nth_value nth must be greater than zero.\"),{next:n=>{const r=n.i0+(e-1);return re=null,next:n=>{const r=t(n.data[n.index]);return null!=r?e=r:e}}},next_value:function(t){let e,n;return{init:()=>(e=null,n=-1),next:r=>{const i=r.data;return r.index<=n?e:(n=function(t,e,n){for(let r=e.length;nf[t]=1))}y(t.sort),e.forEach(((t,e)=>{const r=i[e],f=o[e],v=a[e]||null,_=n(r),x=Ys(t,_,u[e]);if(y(r),l.push(x),lt(ll,t))c.push(function(t,e,n,r){const i=ll[t](e,n);return{init:i.init||h,update:function(t,e){e[r]=i.next(t)}}}(t,r,f,x));else{if(null==r&&\"count\"!==t&&s(\"Null aggregate field specified.\"),\"count\"===t)return void p.push(x);m=!1;let e=d[_];e||(e=d[_]=[],e.field=r,g.push(e)),e.push(Zs(t,v,x))}})),(p.length||g.length)&&(this.cell=function(t,e,n){t=t.map((t=>ru(t,t.field)));const r={num:0,agg:null,store:!1,count:e};if(!n)for(var i=t.length,o=r.agg=Array(i),a=0;a0&&!i(o[n],o[n-1])&&(t.i0=e.left(o,o[n])),rt.init())),this.cell&&this.cell.init()},hl.update=function(t,e){const n=this.cell,r=this.windows,i=t.data,o=r&&r.length;let a;if(n){for(a=t.p0;athis.group(i(t));let a=this.state;a&&!n||(a=this.state=new fl(t)),n||e.modified(a.inputs)?(this.value={},e.visit(e.SOURCE,(t=>o(t).add(t)))):(e.visit(e.REM,(t=>o(t).remove(t))),e.visit(e.ADD,(t=>o(t).add(t))));for(let e=0,n=this._mlen;e=1?Cl:t<=-1?-Cl:Math.asin(t)}const $l=Math.PI,Tl=2*$l,Bl=1e-6,zl=Tl-Bl;function Nl(t){this._+=t[0];for(let e=1,n=t.length;e=0))throw new Error(`invalid digits: ${t}`);if(e>15)return Nl;const n=10**e;return function(t){this._+=t[0];for(let e=1,r=t.length;eBl)if(Math.abs(c*s-u*l)>Bl&&i){let h=n-o,d=r-a,p=s*s+u*u,g=h*h+d*d,m=Math.sqrt(p),y=Math.sqrt(f),v=i*Math.tan(($l-Math.acos((p+f-g)/(2*m*y)))/2),_=v/y,x=v/m;Math.abs(_-1)>Bl&&this._append`L${t+_*l},${e+_*c}`,this._append`A${i},${i},0,0,${+(c*h>l*d)},${this._x1=t+x*s},${this._y1=e+x*u}`}else this._append`L${this._x1=t},${this._y1=e}`;else;}arc(t,e,n,r,i,o){if(t=+t,e=+e,o=!!o,(n=+n)<0)throw new Error(`negative radius: ${n}`);let a=n*Math.cos(r),s=n*Math.sin(r),u=t+a,l=e+s,c=1^o,f=o?r-i:i-r;null===this._x1?this._append`M${u},${l}`:(Math.abs(this._x1-u)>Bl||Math.abs(this._y1-l)>Bl)&&this._append`L${u},${l}`,n&&(f<0&&(f=f%Tl+Tl),f>zl?this._append`A${n},${n},0,1,${c},${t-a},${e-s}A${n},${n},0,1,${c},${this._x1=u},${this._y1=l}`:f>Bl&&this._append`A${n},${n},0,${+(f>=$l)},${c},${this._x1=t+n*Math.cos(i)},${this._y1=e+n*Math.sin(i)}`)}rect(t,e,n,r){this._append`M${this._x0=this._x1=+t},${this._y0=this._y1=+e}h${n=+n}v${+r}h${-n}Z`}toString(){return this._}};function Rl(){return new Ol}function Ul(t){let e=3;return t.digits=function(n){if(!arguments.length)return e;if(null==n)e=null;else{const t=Math.floor(n);if(!(t>=0))throw new RangeError(`invalid digits: ${n}`);e=t}return t},()=>new Ol(e)}function Ll(t){return t.innerRadius}function ql(t){return t.outerRadius}function Pl(t){return t.startAngle}function jl(t){return t.endAngle}function Il(t){return t&&t.padAngle}function Wl(t,e,n,r,i,o,a){var s=t-n,u=e-r,l=(a?o:-o)/Ml(s*s+u*u),c=l*u,f=-l*s,h=t+c,d=e+f,p=n+c,g=r+f,m=(h+p)/2,y=(d+g)/2,v=p-h,_=g-d,x=v*v+_*_,b=i-o,w=h*g-p*d,k=(_<0?-1:1)*Ml(wl(0,b*b*x-w*w)),A=(w*_-v*k)/x,M=(-w*v-_*k)/x,E=(w*_+v*k)/x,D=(-w*v+_*k)/x,C=A-m,F=M-y,S=E-m,$=D-y;return C*C+F*F>S*S+$*$&&(A=E,M=D),{cx:A,cy:M,x01:-c,y01:-f,x11:A*(i/b-1),y11:M*(i/b-1)}}function Hl(t){return\"object\"==typeof t&&\"length\"in t?t:Array.from(t)}function Yl(t){this._context=t}function Gl(t){return new Yl(t)}function Vl(t){return t[0]}function Xl(t){return t[1]}function Jl(t,e){var n=vl(!0),r=null,i=Gl,o=null,a=Ul(s);function s(s){var u,l,c,f=(s=Hl(s)).length,h=!1;for(null==r&&(o=i(c=a())),u=0;u<=f;++u)!(u=f;--h)s.point(y[h],v[h]);s.lineEnd(),s.areaEnd()}m&&(y[c]=+t(d,c,l),v[c]=+e(d,c,l),s.point(r?+r(d,c,l):y[c],n?+n(d,c,l):v[c]))}if(p)return s=null,p+\"\"||null}function c(){return Jl().defined(i).curve(a).context(o)}return t=\"function\"==typeof t?t:void 0===t?Vl:vl(+t),e=\"function\"==typeof e?e:vl(void 0===e?0:+e),n=\"function\"==typeof n?n:void 0===n?Xl:vl(+n),l.x=function(e){return arguments.length?(t=\"function\"==typeof e?e:vl(+e),r=null,l):t},l.x0=function(e){return arguments.length?(t=\"function\"==typeof e?e:vl(+e),l):t},l.x1=function(t){return arguments.length?(r=null==t?null:\"function\"==typeof t?t:vl(+t),l):r},l.y=function(t){return arguments.length?(e=\"function\"==typeof t?t:vl(+t),n=null,l):e},l.y0=function(t){return arguments.length?(e=\"function\"==typeof t?t:vl(+t),l):e},l.y1=function(t){return arguments.length?(n=null==t?null:\"function\"==typeof t?t:vl(+t),l):n},l.lineX0=l.lineY0=function(){return c().x(t).y(e)},l.lineY1=function(){return c().x(t).y(n)},l.lineX1=function(){return c().x(r).y(e)},l.defined=function(t){return arguments.length?(i=\"function\"==typeof t?t:vl(!!t),l):i},l.curve=function(t){return arguments.length?(a=t,null!=o&&(s=a(o)),l):a},l.context=function(t){return arguments.length?(null==t?o=s=null:s=a(o=t),l):o},l}Rl.prototype=Ol.prototype,Yl.prototype={areaStart:function(){this._line=0},areaEnd:function(){this._line=NaN},lineStart:function(){this._point=0},lineEnd:function(){(this._line||0!==this._line&&1===this._point)&&this._context.closePath(),this._line=1-this._line},point:function(t,e){switch(t=+t,e=+e,this._point){case 0:this._point=1,this._line?this._context.lineTo(t,e):this._context.moveTo(t,e);break;case 1:this._point=2;default:this._context.lineTo(t,e)}}};var Ql={draw(t,e){const n=Ml(e/Dl);t.moveTo(n,0),t.arc(0,0,n,0,Fl)}};function Kl(){}function tc(t,e,n){t._context.bezierCurveTo((2*t._x0+t._x1)/3,(2*t._y0+t._y1)/3,(t._x0+2*t._x1)/3,(t._y0+2*t._y1)/3,(t._x0+4*t._x1+e)/6,(t._y0+4*t._y1+n)/6)}function ec(t){this._context=t}function nc(t){this._context=t}function rc(t){this._context=t}function ic(t,e){this._basis=new ec(t),this._beta=e}ec.prototype={areaStart:function(){this._line=0},areaEnd:function(){this._line=NaN},lineStart:function(){this._x0=this._x1=this._y0=this._y1=NaN,this._point=0},lineEnd:function(){switch(this._point){case 3:tc(this,this._x1,this._y1);case 2:this._context.lineTo(this._x1,this._y1)}(this._line||0!==this._line&&1===this._point)&&this._context.closePath(),this._line=1-this._line},point:function(t,e){switch(t=+t,e=+e,this._point){case 0:this._point=1,this._line?this._context.lineTo(t,e):this._context.moveTo(t,e);break;case 1:this._point=2;break;case 2:this._point=3,this._context.lineTo((5*this._x0+this._x1)/6,(5*this._y0+this._y1)/6);default:tc(this,t,e)}this._x0=this._x1,this._x1=t,this._y0=this._y1,this._y1=e}},nc.prototype={areaStart:Kl,areaEnd:Kl,lineStart:function(){this._x0=this._x1=this._x2=this._x3=this._x4=this._y0=this._y1=this._y2=this._y3=this._y4=NaN,this._point=0},lineEnd:function(){switch(this._point){case 1:this._context.moveTo(this._x2,this._y2),this._context.closePath();break;case 2:this._context.moveTo((this._x2+2*this._x3)/3,(this._y2+2*this._y3)/3),this._context.lineTo((this._x3+2*this._x2)/3,(this._y3+2*this._y2)/3),this._context.closePath();break;case 3:this.point(this._x2,this._y2),this.point(this._x3,this._y3),this.point(this._x4,this._y4)}},point:function(t,e){switch(t=+t,e=+e,this._point){case 0:this._point=1,this._x2=t,this._y2=e;break;case 1:this._point=2,this._x3=t,this._y3=e;break;case 2:this._point=3,this._x4=t,this._y4=e,this._context.moveTo((this._x0+4*this._x1+t)/6,(this._y0+4*this._y1+e)/6);break;default:tc(this,t,e)}this._x0=this._x1,this._x1=t,this._y0=this._y1,this._y1=e}},rc.prototype={areaStart:function(){this._line=0},areaEnd:function(){this._line=NaN},lineStart:function(){this._x0=this._x1=this._y0=this._y1=NaN,this._point=0},lineEnd:function(){(this._line||0!==this._line&&3===this._point)&&this._context.closePath(),this._line=1-this._line},point:function(t,e){switch(t=+t,e=+e,this._point){case 0:this._point=1;break;case 1:this._point=2;break;case 2:this._point=3;var n=(this._x0+4*this._x1+t)/6,r=(this._y0+4*this._y1+e)/6;this._line?this._context.lineTo(n,r):this._context.moveTo(n,r);break;case 3:this._point=4;default:tc(this,t,e)}this._x0=this._x1,this._x1=t,this._y0=this._y1,this._y1=e}},ic.prototype={lineStart:function(){this._x=[],this._y=[],this._basis.lineStart()},lineEnd:function(){var t=this._x,e=this._y,n=t.length-1;if(n>0)for(var r,i=t[0],o=e[0],a=t[n]-i,s=e[n]-o,u=-1;++u<=n;)r=u/n,this._basis.point(this._beta*t[u]+(1-this._beta)*(i+r*a),this._beta*e[u]+(1-this._beta)*(o+r*s));this._x=this._y=null,this._basis.lineEnd()},point:function(t,e){this._x.push(+t),this._y.push(+e)}};var oc=function t(e){function n(t){return 1===e?new ec(t):new ic(t,e)}return n.beta=function(e){return t(+e)},n}(.85);function ac(t,e,n){t._context.bezierCurveTo(t._x1+t._k*(t._x2-t._x0),t._y1+t._k*(t._y2-t._y0),t._x2+t._k*(t._x1-e),t._y2+t._k*(t._y1-n),t._x2,t._y2)}function sc(t,e){this._context=t,this._k=(1-e)/6}sc.prototype={areaStart:function(){this._line=0},areaEnd:function(){this._line=NaN},lineStart:function(){this._x0=this._x1=this._x2=this._y0=this._y1=this._y2=NaN,this._point=0},lineEnd:function(){switch(this._point){case 2:this._context.lineTo(this._x2,this._y2);break;case 3:ac(this,this._x1,this._y1)}(this._line||0!==this._line&&1===this._point)&&this._context.closePath(),this._line=1-this._line},point:function(t,e){switch(t=+t,e=+e,this._point){case 0:this._point=1,this._line?this._context.lineTo(t,e):this._context.moveTo(t,e);break;case 1:this._point=2,this._x1=t,this._y1=e;break;case 2:this._point=3;default:ac(this,t,e)}this._x0=this._x1,this._x1=this._x2,this._x2=t,this._y0=this._y1,this._y1=this._y2,this._y2=e}};var uc=function t(e){function n(t){return new sc(t,e)}return n.tension=function(e){return t(+e)},n}(0);function lc(t,e){this._context=t,this._k=(1-e)/6}lc.prototype={areaStart:Kl,areaEnd:Kl,lineStart:function(){this._x0=this._x1=this._x2=this._x3=this._x4=this._x5=this._y0=this._y1=this._y2=this._y3=this._y4=this._y5=NaN,this._point=0},lineEnd:function(){switch(this._point){case 1:this._context.moveTo(this._x3,this._y3),this._context.closePath();break;case 2:this._context.lineTo(this._x3,this._y3),this._context.closePath();break;case 3:this.point(this._x3,this._y3),this.point(this._x4,this._y4),this.point(this._x5,this._y5)}},point:function(t,e){switch(t=+t,e=+e,this._point){case 0:this._point=1,this._x3=t,this._y3=e;break;case 1:this._point=2,this._context.moveTo(this._x4=t,this._y4=e);break;case 2:this._point=3,this._x5=t,this._y5=e;break;default:ac(this,t,e)}this._x0=this._x1,this._x1=this._x2,this._x2=t,this._y0=this._y1,this._y1=this._y2,this._y2=e}};var cc=function t(e){function n(t){return new lc(t,e)}return n.tension=function(e){return t(+e)},n}(0);function fc(t,e){this._context=t,this._k=(1-e)/6}fc.prototype={areaStart:function(){this._line=0},areaEnd:function(){this._line=NaN},lineStart:function(){this._x0=this._x1=this._x2=this._y0=this._y1=this._y2=NaN,this._point=0},lineEnd:function(){(this._line||0!==this._line&&3===this._point)&&this._context.closePath(),this._line=1-this._line},point:function(t,e){switch(t=+t,e=+e,this._point){case 0:this._point=1;break;case 1:this._point=2;break;case 2:this._point=3,this._line?this._context.lineTo(this._x2,this._y2):this._context.moveTo(this._x2,this._y2);break;case 3:this._point=4;default:ac(this,t,e)}this._x0=this._x1,this._x1=this._x2,this._x2=t,this._y0=this._y1,this._y1=this._y2,this._y2=e}};var hc=function t(e){function n(t){return new fc(t,e)}return n.tension=function(e){return t(+e)},n}(0);function dc(t,e,n){var r=t._x1,i=t._y1,o=t._x2,a=t._y2;if(t._l01_a>El){var s=2*t._l01_2a+3*t._l01_a*t._l12_a+t._l12_2a,u=3*t._l01_a*(t._l01_a+t._l12_a);r=(r*s-t._x0*t._l12_2a+t._x2*t._l01_2a)/u,i=(i*s-t._y0*t._l12_2a+t._y2*t._l01_2a)/u}if(t._l23_a>El){var l=2*t._l23_2a+3*t._l23_a*t._l12_a+t._l12_2a,c=3*t._l23_a*(t._l23_a+t._l12_a);o=(o*l+t._x1*t._l23_2a-e*t._l12_2a)/c,a=(a*l+t._y1*t._l23_2a-n*t._l12_2a)/c}t._context.bezierCurveTo(r,i,o,a,t._x2,t._y2)}function pc(t,e){this._context=t,this._alpha=e}pc.prototype={areaStart:function(){this._line=0},areaEnd:function(){this._line=NaN},lineStart:function(){this._x0=this._x1=this._x2=this._y0=this._y1=this._y2=NaN,this._l01_a=this._l12_a=this._l23_a=this._l01_2a=this._l12_2a=this._l23_2a=this._point=0},lineEnd:function(){switch(this._point){case 2:this._context.lineTo(this._x2,this._y2);break;case 3:this.point(this._x2,this._y2)}(this._line||0!==this._line&&1===this._point)&&this._context.closePath(),this._line=1-this._line},point:function(t,e){if(t=+t,e=+e,this._point){var n=this._x2-t,r=this._y2-e;this._l23_a=Math.sqrt(this._l23_2a=Math.pow(n*n+r*r,this._alpha))}switch(this._point){case 0:this._point=1,this._line?this._context.lineTo(t,e):this._context.moveTo(t,e);break;case 1:this._point=2;break;case 2:this._point=3;default:dc(this,t,e)}this._l01_a=this._l12_a,this._l12_a=this._l23_a,this._l01_2a=this._l12_2a,this._l12_2a=this._l23_2a,this._x0=this._x1,this._x1=this._x2,this._x2=t,this._y0=this._y1,this._y1=this._y2,this._y2=e}};var gc=function t(e){function n(t){return e?new pc(t,e):new sc(t,0)}return n.alpha=function(e){return t(+e)},n}(.5);function mc(t,e){this._context=t,this._alpha=e}mc.prototype={areaStart:Kl,areaEnd:Kl,lineStart:function(){this._x0=this._x1=this._x2=this._x3=this._x4=this._x5=this._y0=this._y1=this._y2=this._y3=this._y4=this._y5=NaN,this._l01_a=this._l12_a=this._l23_a=this._l01_2a=this._l12_2a=this._l23_2a=this._point=0},lineEnd:function(){switch(this._point){case 1:this._context.moveTo(this._x3,this._y3),this._context.closePath();break;case 2:this._context.lineTo(this._x3,this._y3),this._context.closePath();break;case 3:this.point(this._x3,this._y3),this.point(this._x4,this._y4),this.point(this._x5,this._y5)}},point:function(t,e){if(t=+t,e=+e,this._point){var n=this._x2-t,r=this._y2-e;this._l23_a=Math.sqrt(this._l23_2a=Math.pow(n*n+r*r,this._alpha))}switch(this._point){case 0:this._point=1,this._x3=t,this._y3=e;break;case 1:this._point=2,this._context.moveTo(this._x4=t,this._y4=e);break;case 2:this._point=3,this._x5=t,this._y5=e;break;default:dc(this,t,e)}this._l01_a=this._l12_a,this._l12_a=this._l23_a,this._l01_2a=this._l12_2a,this._l12_2a=this._l23_2a,this._x0=this._x1,this._x1=this._x2,this._x2=t,this._y0=this._y1,this._y1=this._y2,this._y2=e}};var yc=function t(e){function n(t){return e?new mc(t,e):new lc(t,0)}return n.alpha=function(e){return t(+e)},n}(.5);function vc(t,e){this._context=t,this._alpha=e}vc.prototype={areaStart:function(){this._line=0},areaEnd:function(){this._line=NaN},lineStart:function(){this._x0=this._x1=this._x2=this._y0=this._y1=this._y2=NaN,this._l01_a=this._l12_a=this._l23_a=this._l01_2a=this._l12_2a=this._l23_2a=this._point=0},lineEnd:function(){(this._line||0!==this._line&&3===this._point)&&this._context.closePath(),this._line=1-this._line},point:function(t,e){if(t=+t,e=+e,this._point){var n=this._x2-t,r=this._y2-e;this._l23_a=Math.sqrt(this._l23_2a=Math.pow(n*n+r*r,this._alpha))}switch(this._point){case 0:this._point=1;break;case 1:this._point=2;break;case 2:this._point=3,this._line?this._context.lineTo(this._x2,this._y2):this._context.moveTo(this._x2,this._y2);break;case 3:this._point=4;default:dc(this,t,e)}this._l01_a=this._l12_a,this._l12_a=this._l23_a,this._l01_2a=this._l12_2a,this._l12_2a=this._l23_2a,this._x0=this._x1,this._x1=this._x2,this._x2=t,this._y0=this._y1,this._y1=this._y2,this._y2=e}};var _c=function t(e){function n(t){return e?new vc(t,e):new fc(t,0)}return n.alpha=function(e){return t(+e)},n}(.5);function xc(t){this._context=t}function bc(t){return t<0?-1:1}function wc(t,e,n){var r=t._x1-t._x0,i=e-t._x1,o=(t._y1-t._y0)/(r||i<0&&-0),a=(n-t._y1)/(i||r<0&&-0),s=(o*i+a*r)/(r+i);return(bc(o)+bc(a))*Math.min(Math.abs(o),Math.abs(a),.5*Math.abs(s))||0}function kc(t,e){var n=t._x1-t._x0;return n?(3*(t._y1-t._y0)/n-e)/2:e}function Ac(t,e,n){var r=t._x0,i=t._y0,o=t._x1,a=t._y1,s=(o-r)/3;t._context.bezierCurveTo(r+s,i+s*e,o-s,a-s*n,o,a)}function Mc(t){this._context=t}function Ec(t){this._context=new Dc(t)}function Dc(t){this._context=t}function Cc(t){this._context=t}function Fc(t){var e,n,r=t.length-1,i=new Array(r),o=new Array(r),a=new Array(r);for(i[0]=0,o[0]=2,a[0]=t[0]+2*t[1],e=1;e=0;--e)i[e]=(a[e]-i[e+1])/o[e];for(o[r-1]=(t[r]+i[r-1])/2,e=0;e=0&&(this._t=1-this._t,this._line=1-this._line)},point:function(t,e){switch(t=+t,e=+e,this._point){case 0:this._point=1,this._line?this._context.lineTo(t,e):this._context.moveTo(t,e);break;case 1:this._point=2;default:if(this._t<=0)this._context.lineTo(this._x,e),this._context.lineTo(t,e);else{var n=this._x*(1-this._t)+t*this._t;this._context.lineTo(n,this._y),this._context.lineTo(n,e)}}this._x=t,this._y=e}};const Tc=()=>\"undefined\"!=typeof Image?Image:null;function Bc(t,e){switch(arguments.length){case 0:break;case 1:this.range(t);break;default:this.range(e).domain(t)}return this}function zc(t,e){switch(arguments.length){case 0:break;case 1:\"function\"==typeof t?this.interpolator(t):this.range(t);break;default:this.domain(t),\"function\"==typeof e?this.interpolator(e):this.range(e)}return this}const Nc=Symbol(\"implicit\");function Oc(){var t=new ue,e=[],n=[],r=Nc;function i(i){let o=t.get(i);if(void 0===o){if(r!==Nc)return r;t.set(i,o=e.push(i)-1)}return n[o%n.length]}return i.domain=function(n){if(!arguments.length)return e.slice();e=[],t=new ue;for(const r of n)t.has(r)||t.set(r,e.push(r)-1);return i},i.range=function(t){return arguments.length?(n=Array.from(t),i):n.slice()},i.unknown=function(t){return arguments.length?(r=t,i):r},i.copy=function(){return Oc(e,n).unknown(r)},Bc.apply(i,arguments),i}function Rc(t,e,n){t.prototype=e.prototype=n,n.constructor=t}function Uc(t,e){var n=Object.create(t.prototype);for(var r in e)n[r]=e[r];return n}function Lc(){}var qc=.7,Pc=1/qc,jc=\"\\\\s*([+-]?\\\\d+)\\\\s*\",Ic=\"\\\\s*([+-]?(?:\\\\d*\\\\.)?\\\\d+(?:[eE][+-]?\\\\d+)?)\\\\s*\",Wc=\"\\\\s*([+-]?(?:\\\\d*\\\\.)?\\\\d+(?:[eE][+-]?\\\\d+)?)%\\\\s*\",Hc=/^#([0-9a-f]{3,8})$/,Yc=new RegExp(`^rgb\\\\(${jc},${jc},${jc}\\\\)$`),Gc=new RegExp(`^rgb\\\\(${Wc},${Wc},${Wc}\\\\)$`),Vc=new RegExp(`^rgba\\\\(${jc},${jc},${jc},${Ic}\\\\)$`),Xc=new RegExp(`^rgba\\\\(${Wc},${Wc},${Wc},${Ic}\\\\)$`),Jc=new RegExp(`^hsl\\\\(${Ic},${Wc},${Wc}\\\\)$`),Zc=new RegExp(`^hsla\\\\(${Ic},${Wc},${Wc},${Ic}\\\\)$`),Qc={aliceblue:15792383,antiquewhite:16444375,aqua:65535,aquamarine:8388564,azure:15794175,beige:16119260,bisque:16770244,black:0,blanchedalmond:16772045,blue:255,blueviolet:9055202,brown:10824234,burlywood:14596231,cadetblue:6266528,chartreuse:8388352,chocolate:13789470,coral:16744272,cornflowerblue:6591981,cornsilk:16775388,crimson:14423100,cyan:65535,darkblue:139,darkcyan:35723,darkgoldenrod:12092939,darkgray:11119017,darkgreen:25600,darkgrey:11119017,darkkhaki:12433259,darkmagenta:9109643,darkolivegreen:5597999,darkorange:16747520,darkorchid:10040012,darkred:9109504,darksalmon:15308410,darkseagreen:9419919,darkslateblue:4734347,darkslategray:3100495,darkslategrey:3100495,darkturquoise:52945,darkviolet:9699539,deeppink:16716947,deepskyblue:49151,dimgray:6908265,dimgrey:6908265,dodgerblue:2003199,firebrick:11674146,floralwhite:16775920,forestgreen:2263842,fuchsia:16711935,gainsboro:14474460,ghostwhite:16316671,gold:16766720,goldenrod:14329120,gray:8421504,green:32768,greenyellow:11403055,grey:8421504,honeydew:15794160,hotpink:16738740,indianred:13458524,indigo:4915330,ivory:16777200,khaki:15787660,lavender:15132410,lavenderblush:16773365,lawngreen:8190976,lemonchiffon:16775885,lightblue:11393254,lightcoral:15761536,lightcyan:14745599,lightgoldenrodyellow:16448210,lightgray:13882323,lightgreen:9498256,lightgrey:13882323,lightpink:16758465,lightsalmon:16752762,lightseagreen:2142890,lightskyblue:8900346,lightslategray:7833753,lightslategrey:7833753,lightsteelblue:11584734,lightyellow:16777184,lime:65280,limegreen:3329330,linen:16445670,magenta:16711935,maroon:8388608,mediumaquamarine:6737322,mediumblue:205,mediumorchid:12211667,mediumpurple:9662683,mediumseagreen:3978097,mediumslateblue:8087790,mediumspringgreen:64154,mediumturquoise:4772300,mediumvioletred:13047173,midnightblue:1644912,mintcream:16121850,mistyrose:16770273,moccasin:16770229,navajowhite:16768685,navy:128,oldlace:16643558,olive:8421376,olivedrab:7048739,orange:16753920,orangered:16729344,orchid:14315734,palegoldenrod:15657130,palegreen:10025880,paleturquoise:11529966,palevioletred:14381203,papayawhip:16773077,peachpuff:16767673,peru:13468991,pink:16761035,plum:14524637,powderblue:11591910,purple:8388736,rebeccapurple:6697881,red:16711680,rosybrown:12357519,royalblue:4286945,saddlebrown:9127187,salmon:16416882,sandybrown:16032864,seagreen:3050327,seashell:16774638,sienna:10506797,silver:12632256,skyblue:8900331,slateblue:6970061,slategray:7372944,slategrey:7372944,snow:16775930,springgreen:65407,steelblue:4620980,tan:13808780,teal:32896,thistle:14204888,tomato:16737095,turquoise:4251856,violet:15631086,wheat:16113331,white:16777215,whitesmoke:16119285,yellow:16776960,yellowgreen:10145074};function Kc(){return this.rgb().formatHex()}function tf(){return this.rgb().formatRgb()}function ef(t){var e,n;return t=(t+\"\").trim().toLowerCase(),(e=Hc.exec(t))?(n=e[1].length,e=parseInt(e[1],16),6===n?nf(e):3===n?new sf(e>>8&15|e>>4&240,e>>4&15|240&e,(15&e)<<4|15&e,1):8===n?rf(e>>24&255,e>>16&255,e>>8&255,(255&e)/255):4===n?rf(e>>12&15|e>>8&240,e>>8&15|e>>4&240,e>>4&15|240&e,((15&e)<<4|15&e)/255):null):(e=Yc.exec(t))?new sf(e[1],e[2],e[3],1):(e=Gc.exec(t))?new sf(255*e[1]/100,255*e[2]/100,255*e[3]/100,1):(e=Vc.exec(t))?rf(e[1],e[2],e[3],e[4]):(e=Xc.exec(t))?rf(255*e[1]/100,255*e[2]/100,255*e[3]/100,e[4]):(e=Jc.exec(t))?df(e[1],e[2]/100,e[3]/100,1):(e=Zc.exec(t))?df(e[1],e[2]/100,e[3]/100,e[4]):Qc.hasOwnProperty(t)?nf(Qc[t]):\"transparent\"===t?new sf(NaN,NaN,NaN,0):null}function nf(t){return new sf(t>>16&255,t>>8&255,255&t,1)}function rf(t,e,n,r){return r<=0&&(t=e=n=NaN),new sf(t,e,n,r)}function of(t){return t instanceof Lc||(t=ef(t)),t?new sf((t=t.rgb()).r,t.g,t.b,t.opacity):new sf}function af(t,e,n,r){return 1===arguments.length?of(t):new sf(t,e,n,null==r?1:r)}function sf(t,e,n,r){this.r=+t,this.g=+e,this.b=+n,this.opacity=+r}function uf(){return`#${hf(this.r)}${hf(this.g)}${hf(this.b)}`}function lf(){const t=cf(this.opacity);return`${1===t?\"rgb(\":\"rgba(\"}${ff(this.r)}, ${ff(this.g)}, ${ff(this.b)}${1===t?\")\":`, ${t})`}`}function cf(t){return isNaN(t)?1:Math.max(0,Math.min(1,t))}function ff(t){return Math.max(0,Math.min(255,Math.round(t)||0))}function hf(t){return((t=ff(t))<16?\"0\":\"\")+t.toString(16)}function df(t,e,n,r){return r<=0?t=e=n=NaN:n<=0||n>=1?t=e=NaN:e<=0&&(t=NaN),new mf(t,e,n,r)}function pf(t){if(t instanceof mf)return new mf(t.h,t.s,t.l,t.opacity);if(t instanceof Lc||(t=ef(t)),!t)return new mf;if(t instanceof mf)return t;var e=(t=t.rgb()).r/255,n=t.g/255,r=t.b/255,i=Math.min(e,n,r),o=Math.max(e,n,r),a=NaN,s=o-i,u=(o+i)/2;return s?(a=e===o?(n-r)/s+6*(n0&&u<1?0:a,new mf(a,s,u,t.opacity)}function gf(t,e,n,r){return 1===arguments.length?pf(t):new mf(t,e,n,null==r?1:r)}function mf(t,e,n,r){this.h=+t,this.s=+e,this.l=+n,this.opacity=+r}function yf(t){return(t=(t||0)%360)<0?t+360:t}function vf(t){return Math.max(0,Math.min(1,t||0))}function _f(t,e,n){return 255*(t<60?e+(n-e)*t/60:t<180?n:t<240?e+(n-e)*(240-t)/60:e)}Rc(Lc,ef,{copy(t){return Object.assign(new this.constructor,this,t)},displayable(){return this.rgb().displayable()},hex:Kc,formatHex:Kc,formatHex8:function(){return this.rgb().formatHex8()},formatHsl:function(){return pf(this).formatHsl()},formatRgb:tf,toString:tf}),Rc(sf,af,Uc(Lc,{brighter(t){return t=null==t?Pc:Math.pow(Pc,t),new sf(this.r*t,this.g*t,this.b*t,this.opacity)},darker(t){return t=null==t?qc:Math.pow(qc,t),new sf(this.r*t,this.g*t,this.b*t,this.opacity)},rgb(){return this},clamp(){return new sf(ff(this.r),ff(this.g),ff(this.b),cf(this.opacity))},displayable(){return-.5<=this.r&&this.r<255.5&&-.5<=this.g&&this.g<255.5&&-.5<=this.b&&this.b<255.5&&0<=this.opacity&&this.opacity<=1},hex:uf,formatHex:uf,formatHex8:function(){return`#${hf(this.r)}${hf(this.g)}${hf(this.b)}${hf(255*(isNaN(this.opacity)?1:this.opacity))}`},formatRgb:lf,toString:lf})),Rc(mf,gf,Uc(Lc,{brighter(t){return t=null==t?Pc:Math.pow(Pc,t),new mf(this.h,this.s,this.l*t,this.opacity)},darker(t){return t=null==t?qc:Math.pow(qc,t),new mf(this.h,this.s,this.l*t,this.opacity)},rgb(){var t=this.h%360+360*(this.h<0),e=isNaN(t)||isNaN(this.s)?0:this.s,n=this.l,r=n+(n<.5?n:1-n)*e,i=2*n-r;return new sf(_f(t>=240?t-240:t+120,i,r),_f(t,i,r),_f(t<120?t+240:t-120,i,r),this.opacity)},clamp(){return new mf(yf(this.h),vf(this.s),vf(this.l),cf(this.opacity))},displayable(){return(0<=this.s&&this.s<=1||isNaN(this.s))&&0<=this.l&&this.l<=1&&0<=this.opacity&&this.opacity<=1},formatHsl(){const t=cf(this.opacity);return`${1===t?\"hsl(\":\"hsla(\"}${yf(this.h)}, ${100*vf(this.s)}%, ${100*vf(this.l)}%${1===t?\")\":`, ${t})`}`}}));const xf=Math.PI/180,bf=180/Math.PI,wf=.96422,kf=1,Af=.82521,Mf=4/29,Ef=6/29,Df=3*Ef*Ef,Cf=Ef*Ef*Ef;function Ff(t){if(t instanceof $f)return new $f(t.l,t.a,t.b,t.opacity);if(t instanceof Rf)return Uf(t);t instanceof sf||(t=of(t));var e,n,r=Nf(t.r),i=Nf(t.g),o=Nf(t.b),a=Tf((.2225045*r+.7168786*i+.0606169*o)/kf);return r===i&&i===o?e=n=a:(e=Tf((.4360747*r+.3850649*i+.1430804*o)/wf),n=Tf((.0139322*r+.0971045*i+.7141733*o)/Af)),new $f(116*a-16,500*(e-a),200*(a-n),t.opacity)}function Sf(t,e,n,r){return 1===arguments.length?Ff(t):new $f(t,e,n,null==r?1:r)}function $f(t,e,n,r){this.l=+t,this.a=+e,this.b=+n,this.opacity=+r}function Tf(t){return t>Cf?Math.pow(t,1/3):t/Df+Mf}function Bf(t){return t>Ef?t*t*t:Df*(t-Mf)}function zf(t){return 255*(t<=.0031308?12.92*t:1.055*Math.pow(t,1/2.4)-.055)}function Nf(t){return(t/=255)<=.04045?t/12.92:Math.pow((t+.055)/1.055,2.4)}function Of(t,e,n,r){return 1===arguments.length?function(t){if(t instanceof Rf)return new Rf(t.h,t.c,t.l,t.opacity);if(t instanceof $f||(t=Ff(t)),0===t.a&&0===t.b)return new Rf(NaN,0=1?(n=1,e-1):Math.floor(n*e),i=t[r],o=t[r+1],a=r>0?t[r-1]:2*i-o,s=r()=>t;function Kf(t,e){return function(n){return t+n*e}}function th(t,e){var n=e-t;return n?Kf(t,n>180||n<-180?n-360*Math.round(n/360):n):Qf(isNaN(t)?e:t)}function eh(t){return 1==(t=+t)?nh:function(e,n){return n-e?function(t,e,n){return t=Math.pow(t,n),e=Math.pow(e,n)-t,n=1/n,function(r){return Math.pow(t+r*e,n)}}(e,n,t):Qf(isNaN(e)?n:e)}}function nh(t,e){var n=e-t;return n?Kf(t,n):Qf(isNaN(t)?e:t)}var rh=function t(e){var n=eh(e);function r(t,e){var r=n((t=af(t)).r,(e=af(e)).r),i=n(t.g,e.g),o=n(t.b,e.b),a=nh(t.opacity,e.opacity);return function(e){return t.r=r(e),t.g=i(e),t.b=o(e),t.opacity=a(e),t+\"\"}}return r.gamma=t,r}(1);function ih(t){return function(e){var n,r,i=e.length,o=new Array(i),a=new Array(i),s=new Array(i);for(n=0;no&&(i=e.slice(o,i),s[a]?s[a]+=i:s[++a]=i),(n=n[0])===(r=r[0])?s[a]?s[a]+=r:s[++a]=r:(s[++a]=null,u.push({i:a,x:fh(n,r)})),o=ph.lastIndex;return o180?e+=360:e-t>180&&(t+=360),o.push({i:n.push(i(n)+\"rotate(\",null,r)-2,x:fh(t,e)})):e&&n.push(i(n)+\"rotate(\"+e+r)}(o.rotate,a.rotate,s,u),function(t,e,n,o){t!==e?o.push({i:n.push(i(n)+\"skewX(\",null,r)-2,x:fh(t,e)}):e&&n.push(i(n)+\"skewX(\"+e+r)}(o.skewX,a.skewX,s,u),function(t,e,n,r,o,a){if(t!==n||e!==r){var s=o.push(i(o)+\"scale(\",null,\",\",null,\")\");a.push({i:s-4,x:fh(t,n)},{i:s-2,x:fh(e,r)})}else 1===n&&1===r||o.push(i(o)+\"scale(\"+n+\",\"+r+\")\")}(o.scaleX,o.scaleY,a.scaleX,a.scaleY,s,u),o=a=null,function(t){for(var e,n=-1,r=u.length;++ne&&(n=t,t=e,e=n),function(n){return Math.max(t,Math.min(e,n))}}(a[0],a[t-1])),r=t>2?Ih:jh,i=o=null,f}function f(e){return null==e||isNaN(e=+e)?n:(i||(i=r(a.map(t),s,u)))(t(l(e)))}return f.invert=function(n){return l(e((o||(o=r(s,a.map(t),fh)))(n)))},f.domain=function(t){return arguments.length?(a=Array.from(t,Uh),c()):a.slice()},f.range=function(t){return arguments.length?(s=Array.from(t),c()):s.slice()},f.rangeRound=function(t){return s=Array.from(t),u=yh,c()},f.clamp=function(t){return arguments.length?(l=!!t||qh,c()):l!==qh},f.interpolate=function(t){return arguments.length?(u=t,c()):u},f.unknown=function(t){return arguments.length?(n=t,f):n},function(n,r){return t=n,e=r,c()}}function Yh(){return Hh()(qh,qh)}function Gh(t,e,n,r){var i,o=be(t,e,n);switch((r=Re(null==r?\",f\":r)).type){case\"s\":var a=Math.max(Math.abs(t),Math.abs(e));return null!=r.precision||isNaN(i=Xe(o,a))||(r.precision=i),We(r,a);case\"\":case\"e\":case\"g\":case\"p\":case\"r\":null!=r.precision||isNaN(i=Je(o,Math.max(Math.abs(t),Math.abs(e))))||(r.precision=i-(\"e\"===r.type));break;case\"f\":case\"%\":null!=r.precision||isNaN(i=Ve(o))||(r.precision=i-2*(\"%\"===r.type))}return Ie(r)}function Vh(t){var e=t.domain;return t.ticks=function(t){var n=e();return _e(n[0],n[n.length-1],null==t?10:t)},t.tickFormat=function(t,n){var r=e();return Gh(r[0],r[r.length-1],null==t?10:t,n)},t.nice=function(n){null==n&&(n=10);var r,i,o=e(),a=0,s=o.length-1,u=o[a],l=o[s],c=10;for(l0;){if((i=xe(u,l,n))===r)return o[a]=u,o[s]=l,e(o);if(i>0)u=Math.floor(u/i)*i,l=Math.ceil(l/i)*i;else{if(!(i<0))break;u=Math.ceil(u*i)/i,l=Math.floor(l*i)/i}r=i}return t},t}function Xh(t,e){var n,r=0,i=(t=t.slice()).length-1,o=t[r],a=t[i];return a-t(-e,n)}function nd(t){const e=t(Jh,Zh),n=e.domain;let r,i,o=10;function a(){return r=function(t){return t===Math.E?Math.log:10===t&&Math.log10||2===t&&Math.log2||(t=Math.log(t),e=>Math.log(e)/t)}(o),i=function(t){return 10===t?td:t===Math.E?Math.exp:e=>Math.pow(t,e)}(o),n()[0]<0?(r=ed(r),i=ed(i),t(Qh,Kh)):t(Jh,Zh),e}return e.base=function(t){return arguments.length?(o=+t,a()):o},e.domain=function(t){return arguments.length?(n(t),a()):n()},e.ticks=t=>{const e=n();let a=e[0],s=e[e.length-1];const u=s0){for(;f<=h;++f)for(l=1;ls)break;p.push(c)}}else for(;f<=h;++f)for(l=o-1;l>=1;--l)if(c=f>0?l/i(-f):l*i(f),!(cs)break;p.push(c)}2*p.length{if(null==t&&(t=10),null==n&&(n=10===o?\"s\":\",\"),\"function\"!=typeof n&&(o%1||null!=(n=Re(n)).precision||(n.trim=!0),n=Ie(n)),t===1/0)return n;const a=Math.max(1,o*t/e.ticks().length);return t=>{let e=t/i(Math.round(r(t)));return e*on(Xh(n(),{floor:t=>i(Math.floor(r(t))),ceil:t=>i(Math.ceil(r(t)))})),e}function rd(t){return function(e){return Math.sign(e)*Math.log1p(Math.abs(e/t))}}function id(t){return function(e){return Math.sign(e)*Math.expm1(Math.abs(e))*t}}function od(t){var e=1,n=t(rd(e),id(e));return n.constant=function(n){return arguments.length?t(rd(e=+n),id(e)):e},Vh(n)}function ad(t){return function(e){return e<0?-Math.pow(-e,t):Math.pow(e,t)}}function sd(t){return t<0?-Math.sqrt(-t):Math.sqrt(t)}function ud(t){return t<0?-t*t:t*t}function ld(t){var e=t(qh,qh),n=1;return e.exponent=function(e){return arguments.length?1===(n=+e)?t(qh,qh):.5===n?t(sd,ud):t(ad(n),ad(1/n)):n},Vh(e)}function cd(){var t=ld(Hh());return t.copy=function(){return Wh(t,cd()).exponent(t.exponent())},Bc.apply(t,arguments),t}function fd(t){return new Date(t)}function hd(t){return t instanceof Date?+t:+new Date(+t)}function dd(t,e,n,r,i,o,a,s,u,l){var c=Yh(),f=c.invert,h=c.domain,d=l(\".%L\"),p=l(\":%S\"),g=l(\"%I:%M\"),m=l(\"%I %p\"),y=l(\"%a %d\"),v=l(\"%b %d\"),_=l(\"%B\"),x=l(\"%Y\");function b(t){return(u(t)0?r:1:0}const bd=\"linear\",wd=\"log\",kd=\"pow\",Ad=\"sqrt\",Md=\"symlog\",Ed=\"time\",Dd=\"utc\",Cd=\"sequential\",Fd=\"diverging\",Sd=\"quantile\",$d=\"quantize\",Td=\"threshold\",Bd=\"ordinal\",zd=\"point\",Nd=\"band\",Od=\"bin-ordinal\",Rd=\"continuous\",Ud=\"discrete\",Ld=\"discretizing\",qd=\"interpolating\",Pd=\"temporal\";function jd(){const t=Oc().unknown(void 0),e=t.domain,n=t.range;let r,i,o=[0,1],a=!1,s=0,u=0,l=.5;function c(){const t=e().length,c=o[1]d+r*t));return n(c?p.reverse():p)}return delete t.unknown,t.domain=function(t){return arguments.length?(e(t),c()):e()},t.range=function(t){return arguments.length?(o=[+t[0],+t[1]],c()):o.slice()},t.rangeRound=function(t){return o=[+t[0],+t[1]],a=!0,c()},t.bandwidth=function(){return i},t.step=function(){return r},t.round=function(t){return arguments.length?(a=!!t,c()):a},t.padding=function(t){return arguments.length?(u=Math.max(0,Math.min(1,t)),s=u,c()):s},t.paddingInner=function(t){return arguments.length?(s=Math.max(0,Math.min(1,t)),c()):s},t.paddingOuter=function(t){return arguments.length?(u=Math.max(0,Math.min(1,t)),c()):u},t.align=function(t){return arguments.length?(l=Math.max(0,Math.min(1,t)),c()):l},t.invertRange=function(t){if(null==t[0]||null==t[1])return;const r=o[1]o[1-r])?void 0:(u=Math.max(0,oe(a,f)-1),l=f===h?u:oe(a,h)-1,f-a[u]>i+1e-10&&++u,r&&(c=u,u=s-l,l=s-c),u>l?void 0:e().slice(u,l+1))},t.invert=function(e){const n=t.invertRange([e,e]);return n?n[0]:n},t.copy=function(){return jd().domain(e()).range(o).round(a).paddingInner(s).paddingOuter(u).align(l)},c()}function Id(t){const e=t.copy;return t.padding=t.paddingOuter,delete t.paddingInner,t.copy=function(){return Id(e())},t}var Wd=Array.prototype.map;const Hd=Array.prototype.slice;const Yd=new Map,Gd=Symbol(\"vega_scale\");function Vd(t){return t[Gd]=!0,t}function Xd(t,e,n){return arguments.length>1?(Yd.set(t,function(t,e,n){const r=function(){const n=e();return n.invertRange||(n.invertRange=n.invert?function(t){return function(e){let n,r=e[0],i=e[1];return i=s&&n[o]<=u&&(l<0&&(l=o),r=o);if(!(l<0))return s=t.invertExtent(n[l]),u=t.invertExtent(n[r]),[void 0===s[0]?s[1]:s[0],void 0===u[1]?u[0]:u[1]]}}(n):void 0),n.type=t,Vd(n)};return r.metadata=Bt(V(n)),r}(t,e,n)),this):Jd(t)?Yd.get(t):void 0}function Jd(t){return Yd.has(t)}function Zd(t,e){const n=Yd.get(t);return n&&n.metadata[e]}function Qd(t){return Zd(t,Rd)}function Kd(t){return Zd(t,Ud)}function tp(t){return Zd(t,Ld)}function ep(t){return Zd(t,wd)}function np(t){return Zd(t,qd)}function rp(t){return Zd(t,Sd)}Xd(\"identity\",(function t(e){var n;function r(t){return null==t||isNaN(t=+t)?n:t}return r.invert=r,r.domain=r.range=function(t){return arguments.length?(e=Array.from(t,Uh),r):e.slice()},r.unknown=function(t){return arguments.length?(n=t,r):n},r.copy=function(){return t(e).unknown(n)},e=arguments.length?Array.from(e,Uh):[0,1],Vh(r)})),Xd(bd,(function t(){var e=Yh();return e.copy=function(){return Wh(e,t())},Bc.apply(e,arguments),Vh(e)}),Rd),Xd(wd,(function t(){const e=nd(Hh()).domain([1,10]);return e.copy=()=>Wh(e,t()).base(e.base()),Bc.apply(e,arguments),e}),[Rd,wd]),Xd(kd,cd,Rd),Xd(Ad,(function(){return cd.apply(null,arguments).exponent(.5)}),Rd),Xd(Md,(function t(){var e=od(Hh());return e.copy=function(){return Wh(e,t()).constant(e.constant())},Bc.apply(e,arguments)}),Rd),Xd(Ed,(function(){return Bc.apply(dd(qn,Pn,Nn,Bn,vn,pn,hn,cn,ln,ni).domain([new Date(2e3,0,1),new Date(2e3,0,2)]),arguments)}),[Rd,Pd]),Xd(Dd,(function(){return Bc.apply(dd(Un,Ln,On,zn,En,gn,dn,fn,ln,ii).domain([Date.UTC(2e3,0,1),Date.UTC(2e3,0,2)]),arguments)}),[Rd,Pd]),Xd(Cd,md,[Rd,qd]),Xd(`${Cd}-${bd}`,md,[Rd,qd]),Xd(`${Cd}-${wd}`,(function t(){var e=nd(pd()).domain([1,10]);return e.copy=function(){return gd(e,t()).base(e.base())},zc.apply(e,arguments)}),[Rd,qd,wd]),Xd(`${Cd}-${kd}`,yd,[Rd,qd]),Xd(`${Cd}-${Ad}`,(function(){return yd.apply(null,arguments).exponent(.5)}),[Rd,qd]),Xd(`${Cd}-${Md}`,(function t(){var e=od(pd());return e.copy=function(){return gd(e,t()).constant(e.constant())},zc.apply(e,arguments)}),[Rd,qd]),Xd(`${Fd}-${bd}`,(function t(){var e=Vh(vd()(qh));return e.copy=function(){return gd(e,t())},zc.apply(e,arguments)}),[Rd,qd]),Xd(`${Fd}-${wd}`,(function t(){var e=nd(vd()).domain([.1,1,10]);return e.copy=function(){return gd(e,t()).base(e.base())},zc.apply(e,arguments)}),[Rd,qd,wd]),Xd(`${Fd}-${kd}`,_d,[Rd,qd]),Xd(`${Fd}-${Ad}`,(function(){return _d.apply(null,arguments).exponent(.5)}),[Rd,qd]),Xd(`${Fd}-${Md}`,(function t(){var e=od(vd());return e.copy=function(){return gd(e,t()).constant(e.constant())},zc.apply(e,arguments)}),[Rd,qd]),Xd(Sd,(function t(){var e,n=[],r=[],i=[];function o(){var t=0,e=Math.max(1,r.length);for(i=new Array(e-1);++t0?i[e-1]:n[0],e=i?[o[i-1],r]:[o[e-1],o[e]]},s.unknown=function(t){return arguments.length?(e=t,s):s},s.thresholds=function(){return o.slice()},s.copy=function(){return t().domain([n,r]).range(a).unknown(e)},Bc.apply(Vh(s),arguments)}),Ld),Xd(Td,(function t(){var e,n=[.5],r=[0,1],i=1;function o(t){return null!=t&&t<=t?r[oe(n,t,0,i)]:e}return o.domain=function(t){return arguments.length?(n=Array.from(t),i=Math.min(n.length,r.length-1),o):n.slice()},o.range=function(t){return arguments.length?(r=Array.from(t),i=Math.min(n.length,r.length-1),o):r.slice()},o.invertExtent=function(t){var e=r.indexOf(t);return[n[e-1],n[e]]},o.unknown=function(t){return arguments.length?(e=t,o):e},o.copy=function(){return t().domain(n).range(r).unknown(e)},Bc.apply(o,arguments)}),Ld),Xd(Od,(function t(){let e=[],n=[];function r(t){return null==t||t!=t?void 0:n[(oe(e,t)-1)%n.length]}return r.domain=function(t){return arguments.length?(e=function(t){return Wd.call(t,S)}(t),r):e.slice()},r.range=function(t){return arguments.length?(n=Hd.call(t),r):n.slice()},r.tickFormat=function(t,n){return Gh(e[0],F(e),null==t?10:t,n)},r.copy=function(){return t().domain(r.domain()).range(r.range())},r}),[Ud,Ld]),Xd(Bd,Oc,Ud),Xd(Nd,jd,Ud),Xd(zd,(function(){return Id(jd().paddingInner(1))}),Ud);const ip=[\"clamp\",\"base\",\"constant\",\"exponent\"];function op(t,e){const n=e[0],r=F(e)-n;return function(e){return t(n+e*r)}}function ap(t,e,n){return Oh(lp(e||\"rgb\",n),t)}function sp(t,e){const n=new Array(e),r=e+1;for(let i=0;it[e]?a[e](t[e]()):0)),a):rt(.5)}function lp(t,e){const n=Rh[function(t){return\"interpolate\"+t.toLowerCase().split(\"-\").map((t=>t[0].toUpperCase()+t.slice(1))).join(\"\")}(t)];return null!=e&&n&&n.gamma?n.gamma(e):n}function cp(t){const e=t.length/6|0,n=new Array(e);for(let r=0;r1?(hp[t]=e,this):hp[t]}fp({category10:\"1f77b4ff7f0e2ca02cd627289467bd8c564be377c27f7f7fbcbd2217becf\",category20:\"1f77b4aec7e8ff7f0effbb782ca02c98df8ad62728ff98969467bdc5b0d58c564bc49c94e377c2f7b6d27f7f7fc7c7c7bcbd22dbdb8d17becf9edae5\",category20b:\"393b795254a36b6ecf9c9ede6379398ca252b5cf6bcedb9c8c6d31bd9e39e7ba52e7cb94843c39ad494ad6616be7969c7b4173a55194ce6dbdde9ed6\",category20c:\"3182bd6baed69ecae1c6dbefe6550dfd8d3cfdae6bfdd0a231a35474c476a1d99bc7e9c0756bb19e9ac8bcbddcdadaeb636363969696bdbdbdd9d9d9\",tableau10:\"4c78a8f58518e4575672b7b254a24beeca3bb279a2ff9da69d755dbab0ac\",tableau20:\"4c78a89ecae9f58518ffbf7954a24b88d27ab79a20f2cf5b43989483bcb6e45756ff9d9879706ebab0acd67195fcbfd2b279a2d6a5c99e765fd8b5a5\",accent:\"7fc97fbeaed4fdc086ffff99386cb0f0027fbf5b17666666\",dark2:\"1b9e77d95f027570b3e7298a66a61ee6ab02a6761d666666\",paired:\"a6cee31f78b4b2df8a33a02cfb9a99e31a1cfdbf6fff7f00cab2d66a3d9affff99b15928\",pastel1:\"fbb4aeb3cde3ccebc5decbe4fed9a6ffffcce5d8bdfddaecf2f2f2\",pastel2:\"b3e2cdfdcdaccbd5e8f4cae4e6f5c9fff2aef1e2cccccccc\",set1:\"e41a1c377eb84daf4a984ea3ff7f00ffff33a65628f781bf999999\",set2:\"66c2a5fc8d628da0cbe78ac3a6d854ffd92fe5c494b3b3b3\",set3:\"8dd3c7ffffb3bebadafb807280b1d3fdb462b3de69fccde5d9d9d9bc80bdccebc5ffed6f\"},cp),fp({blues:\"cfe1f2bed8eca8cee58fc1de74b2d75ba3cf4592c63181bd206fb2125ca40a4a90\",greens:\"d3eecdc0e6baabdda594d3917bc77d60ba6c46ab5e329a512089430e7735036429\",greys:\"e2e2e2d4d4d4c4c4c4b1b1b19d9d9d8888887575756262624d4d4d3535351e1e1e\",oranges:\"fdd8b3fdc998fdb87bfda55efc9244f87f2cf06b18e4580bd14904b93d029f3303\",purples:\"e2e1efd4d4e8c4c5e0b4b3d6a3a0cc928ec3827cb97566ae684ea25c3696501f8c\",reds:\"fdc9b4fcb49afc9e80fc8767fa7051f6573fec3f2fdc2a25c81b1db21218970b13\",blueGreen:\"d5efedc1e8e0a7ddd18bd2be70c6a958ba9144ad77319c5d2089460e7736036429\",bluePurple:\"ccddecbad0e4a8c2dd9ab0d4919cc98d85be8b6db28a55a6873c99822287730f71\",greenBlue:\"d3eecec5e8c3b1e1bb9bd8bb82cec269c2ca51b2cd3c9fc7288abd1675b10b60a1\",orangeRed:\"fddcaffdcf9bfdc18afdad77fb9562f67d53ee6545e24932d32d1ebf130da70403\",purpleBlue:\"dbdaebc8cee4b1c3de97b7d87bacd15b9fc93a90c01e7fb70b70ab056199045281\",purpleBlueGreen:\"dbd8eac8cee4b0c3de93b7d872acd1549fc83892bb1c88a3097f8702736b016353\",purpleRed:\"dcc9e2d3b3d7ce9eccd186c0da6bb2e14da0e23189d91e6fc61159ab07498f023a\",redPurple:\"fccfccfcbec0faa9b8f98faff571a5ec539ddb3695c41b8aa908808d0179700174\",yellowGreen:\"e4f4acd1eca0b9e2949ed68880c97c62bb6e47aa5e3297502083440e723b036034\",yellowOrangeBrown:\"feeaa1fedd84fecc63feb746fca031f68921eb7215db5e0bc54c05ab3d038f3204\",yellowOrangeRed:\"fee087fed16ffebd59fea849fd903efc7335f9522bee3423de1b20ca0b22af0225\",blueOrange:\"134b852f78b35da2cb9dcae1d2e5eff2f0ebfce0bafbbf74e8932fc5690d994a07\",brownBlueGreen:\"704108a0651ac79548e3c78af3e6c6eef1eac9e9e48ed1c74da79e187a72025147\",purpleGreen:\"5b1667834792a67fb6c9aed3e6d6e8eff0efd9efd5aedda971bb75368e490e5e29\",purpleOrange:\"4114696647968f83b7b9b4d6dadbebf3eeeafce0bafbbf74e8932fc5690d994a07\",redBlue:\"8c0d25bf363adf745ef4ae91fbdbc9f2efeed2e5ef9dcae15da2cb2f78b3134b85\",redGrey:\"8c0d25bf363adf745ef4ae91fcdccbfaf4f1e2e2e2c0c0c0969696646464343434\",yellowGreenBlue:\"eff9bddbf1b4bde5b594d5b969c5be45b4c22c9ec02182b82163aa23479c1c3185\",redYellowBlue:\"a50026d4322cf16e43fcac64fedd90faf8c1dcf1ecabd6e875abd04a74b4313695\",redYellowGreen:\"a50026d4322cf16e43fcac63fedd8df9f7aed7ee8ea4d86e64bc6122964f006837\",pinkYellowGreen:\"8e0152c0267edd72adf0b3d6faddedf5f3efe1f2cab6de8780bb474f9125276419\",spectral:\"9e0142d13c4bf0704afcac63fedd8dfbf8b0e0f3a1a9dda269bda94288b55e4fa2\",viridis:\"440154470e61481a6c482575472f7d443a834144873d4e8a39568c35608d31688e2d708e2a788e27818e23888e21918d1f988b1fa08822a8842ab07f35b77943bf7154c56866cc5d7ad1518fd744a5db36bcdf27d2e21be9e51afde725\",magma:\"0000040404130b0924150e3720114b2c11603b0f704a107957157e651a80721f817f24828c29819a2e80a8327db6377ac43c75d1426fde4968e95462f1605df76f5cfa7f5efc8f65fe9f6dfeaf78febf84fece91fddea0fcedaffcfdbf\",inferno:\"0000040403130c0826170c3b240c4f330a5f420a68500d6c5d126e6b176e781c6d86216b932667a12b62ae305cbb3755c73e4cd24644dd513ae65c30ed6925f3771af8850ffb9506fca50afcb519fac62df6d645f2e661f3f484fcffa4\",plasma:\"0d088723069033059742039d5002a25d01a66a00a87801a88405a7900da49c179ea72198b12a90ba3488c33d80cb4779d35171da5a69e16462e76e5bed7953f2834cf68f44fa9a3dfca636fdb32ffec029fcce25f9dc24f5ea27f0f921\",cividis:\"00205100235800265d002961012b65042e670831690d346b11366c16396d1c3c6e213f6e26426e2c456e31476e374a6e3c4d6e42506e47536d4c566d51586e555b6e5a5e6e5e616e62646f66676f6a6a706e6d717270717573727976737c79747f7c75827f758682768985778c8877908b78938e789691789a94789e9778a19b78a59e77a9a177aea575b2a874b6ab73bbaf71c0b26fc5b66dc9b96acebd68d3c065d8c462ddc85fe2cb5ce7cf58ebd355f0d652f3da4ff7de4cfae249fce647\",rainbow:\"6e40aa883eb1a43db3bf3cafd83fa4ee4395fe4b83ff576eff6659ff7847ff8c38f3a130e2b72fcfcc36bee044aff05b8ff4576ff65b52f6673af27828ea8d1ddfa319d0b81cbecb23abd82f96e03d82e14c6edb5a5dd0664dbf6e40aa\",sinebow:\"ff4040fc582af47218e78d0bd5a703bfbf00a7d5038de70b72f41858fc2a40ff402afc5818f4720be78d03d5a700bfbf03a7d50b8de71872f42a58fc4040ff582afc7218f48d0be7a703d5bf00bfd503a7e70b8df41872fc2a58ff4040\",turbo:\"23171b32204a3e2a71453493493eae4b49c54a53d7485ee44569ee4074f53c7ff8378af93295f72e9ff42ba9ef28b3e926bce125c5d925cdcf27d5c629dcbc2de3b232e9a738ee9d3ff39347f68950f9805afc7765fd6e70fe667cfd5e88fc5795fb51a1f84badf545b9f140c5ec3cd0e637dae034e4d931ecd12ef4c92bfac029ffb626ffad24ffa223ff9821ff8d1fff821dff771cfd6c1af76118f05616e84b14df4111d5380fcb2f0dc0260ab61f07ac1805a313029b0f00950c00910b00\",browns:\"eedbbdecca96e9b97ae4a865dc9856d18954c7784cc0673fb85536ad44339f3632\",tealBlues:\"bce4d89dd3d181c3cb65b3c245a2b9368fae347da0306a932c5985\",teals:\"bbdfdfa2d4d58ac9c975bcbb61b0af4da5a43799982b8b8c1e7f7f127273006667\",warmGreys:\"dcd4d0cec5c1c0b8b4b3aaa7a59c9998908c8b827f7e7673726866665c5a59504e\",goldGreen:\"f4d166d5ca60b6c35c98bb597cb25760a6564b9c533f8f4f33834a257740146c36\",goldOrange:\"f4d166f8be5cf8aa4cf5983bf3852aef701be2621fd65322c54923b142239e3a26\",goldRed:\"f4d166f6be59f9aa51fc964ef6834bee734ae56249db5247cf4244c43141b71d3e\",lightGreyRed:\"efe9e6e1dad7d5cbc8c8bdb9bbaea9cd967ddc7b43e15f19df4011dc000b\",lightGreyTeal:\"e4eaead6dcddc8ced2b7c2c7a6b4bc64b0bf22a6c32295c11f85be1876bc\",lightMulti:\"e0f1f2c4e9d0b0de9fd0e181f6e072f6c053f3993ef77440ef4a3c\",lightOrange:\"f2e7daf7d5baf9c499fab184fa9c73f68967ef7860e8645bde515bd43d5b\",lightTealBlue:\"e3e9e0c0dccf9aceca7abfc859afc0389fb9328dad2f7ca0276b95255988\",darkBlue:\"3232322d46681a5c930074af008cbf05a7ce25c0dd38daed50f3faffffff\",darkGold:\"3c3c3c584b37725e348c7631ae8b2bcfa424ecc31ef9de30fff184ffffff\",darkGreen:\"3a3a3a215748006f4d048942489e4276b340a6c63dd2d836ffeb2cffffaa\",darkMulti:\"3737371f5287197d8c29a86995ce3fffe800ffffff\",darkRed:\"3434347036339e3c38cc4037e75d1eec8620eeab29f0ce32ffeb2c\"},(t=>ap(cp(t))));const pp=\"symbol\",gp=\"discrete\",mp=t=>k(t)?t.map((t=>String(t))):String(t),yp=(t,e)=>t[1]-e[1],vp=(t,e)=>e[1]-t[1];function _p(t,e,n){let r;return vt(e)&&(t.bins&&(e=Math.max(e,t.bins.length)),null!=n&&(e=Math.min(e,Math.floor(Dt(t.domain())/n||1)+1))),A(e)&&(r=e.step,e=e.interval),xt(e)&&(e=t.type===Ed?Cr(e):t.type==Dd?Fr(e):s(\"Only time and utc scales accept interval strings.\"),r&&(e=e.every(r))),e}function xp(t,e,n){let r=t.range(),i=r[0],o=F(r),a=yp;if(i>o&&(r=o,o=i,i=r,a=vp),i=Math.floor(i),o=Math.ceil(o),e=e.map((e=>[e,t(e)])).filter((t=>i<=t[1]&&t[1]<=o)).sort(a).map((t=>t[0])),n>0&&e.length>1){const t=[e[0],F(e)];for(;e.length>n&&e.length>=3;)e=e.filter(((t,e)=>!(e%2)));e.length<3&&(e=t)}return e}function bp(t,e){return t.bins?xp(t,t.bins):t.ticks?t.ticks(e):t.domain()}function wp(t,e,n,r,i,o){const a=e.type;let s=mp;if(a===Ed||i===Ed)s=t.timeFormat(r);else if(a===Dd||i===Dd)s=t.utcFormat(r);else if(ep(a)){const i=t.formatFloat(r);if(o||e.bins)s=i;else{const t=kp(e,n,!1);s=e=>t(e)?i(e):\"\"}}else if(e.tickFormat){const i=e.domain();s=t.formatSpan(i[0],i[i.length-1],n,r)}else r&&(s=t.format(r));return s}function kp(t,e,n){const r=bp(t,e),i=t.base(),o=Math.log(i),a=Math.max(1,i*e/r.length),s=t=>{let e=t/Math.pow(i,Math.round(Math.log(t)/o));return e*iAp[t.type]||t.bins;function Cp(t,e,n,r,i,o,a){const s=Mp[e.type]&&o!==Ed&&o!==Dd?function(t,e,n){const r=e[Mp[e.type]](),i=r.length;let o,a=i>1?r[1]-r[0]:r[0];for(o=1;o(e,n,r)=>{const i=Sp(r[n+1],Sp(r.max,1/0)),o=Bp(e,t),a=Bp(i,t);return o&&a?o+\" – \"+a:a?\"< \"+a:\"≥ \"+o},Sp=(t,e)=>null!=t?t:e,$p=t=>(e,n)=>n?t(e):null,Tp=t=>e=>t(e),Bp=(t,e)=>Number.isFinite(t)?e(t):null;function zp(t,e,n,r){const i=r||e.type;return xt(n)&&function(t){return Zd(t,Pd)}(i)&&(n=n.replace(/%a/g,\"%A\").replace(/%b/g,\"%B\")),n||i!==Ed?n||i!==Dd?Cp(t,e,5,null,n,r,!0):t.utcFormat(\"%A, %d %B %Y, %X UTC\"):t.timeFormat(\"%A, %d %B %Y, %X\")}function Np(t,e,n){n=n||{};const r=Math.max(3,n.maxlen||7),i=zp(t,e,n.format,n.formatType);if(tp(e.type)){const t=Ep(e).slice(1).map(i),n=t.length;return`${n} boundar${1===n?\"y\":\"ies\"}: ${t.join(\", \")}`}if(Kd(e.type)){const t=e.domain(),n=t.length;return`${n} value${1===n?\"\":\"s\"}: ${n>r?t.slice(0,r-2).map(i).join(\", \")+\", ending with \"+t.slice(-1).map(i):t.map(i).join(\", \")}`}{const t=e.domain();return`values from ${i(t[0])} to ${i(F(t))}`}}let Op=0;const Rp=\"p_\";function Up(t){return t&&t.gradient}function Lp(t,e,n){const r=t.gradient;let i=t.id,o=\"radial\"===r?Rp:\"\";return i||(i=t.id=\"gradient_\"+Op++,\"radial\"===r?(t.x1=qp(t.x1,.5),t.y1=qp(t.y1,.5),t.r1=qp(t.r1,0),t.x2=qp(t.x2,.5),t.y2=qp(t.y2,.5),t.r2=qp(t.r2,.5),o=Rp):(t.x1=qp(t.x1,0),t.y1=qp(t.y1,0),t.x2=qp(t.x2,1),t.y2=qp(t.y2,0))),e[i]=t,\"url(\"+(n||\"\")+\"#\"+o+i+\")\"}function qp(t,e){return null!=t?t:e}function Pp(t,e){var n,r=[];return n={gradient:\"linear\",x1:t?t[0]:0,y1:t?t[1]:0,x2:e?e[0]:1,y2:e?e[1]:0,stops:r,stop:function(t,e){return r.push({offset:t,color:e}),n}}}const jp={basis:{curve:function(t){return new ec(t)}},\"basis-closed\":{curve:function(t){return new nc(t)}},\"basis-open\":{curve:function(t){return new rc(t)}},bundle:{curve:oc,tension:\"beta\",value:.85},cardinal:{curve:uc,tension:\"tension\",value:0},\"cardinal-open\":{curve:hc,tension:\"tension\",value:0},\"cardinal-closed\":{curve:cc,tension:\"tension\",value:0},\"catmull-rom\":{curve:gc,tension:\"alpha\",value:.5},\"catmull-rom-closed\":{curve:yc,tension:\"alpha\",value:.5},\"catmull-rom-open\":{curve:_c,tension:\"alpha\",value:.5},linear:{curve:Gl},\"linear-closed\":{curve:function(t){return new xc(t)}},monotone:{horizontal:function(t){return new Ec(t)},vertical:function(t){return new Mc(t)}},natural:{curve:function(t){return new Cc(t)}},step:{curve:function(t){return new Sc(t,.5)}},\"step-after\":{curve:function(t){return new Sc(t,1)}},\"step-before\":{curve:function(t){return new Sc(t,0)}}};function Ip(t,e,n){var r=lt(jp,t)&&jp[t],i=null;return r&&(i=r.curve||r[e||\"vertical\"],r.tension&&null!=n&&(i=i[r.tension](n))),i}const Wp={m:2,l:2,h:1,v:1,z:0,c:6,s:4,q:4,t:2,a:7},Hp=/[mlhvzcsqta]([^mlhvzcsqta]+|$)/gi,Yp=/^[+-]?(([0-9]*\\.[0-9]+)|([0-9]+\\.)|([0-9]+))([eE][+-]?[0-9]+)?/,Gp=/^((\\s+,?\\s*)|(,\\s*))/,Vp=/^[01]/;function Xp(t){const e=[];return(t.match(Hp)||[]).forEach((t=>{let n=t[0];const r=n.toLowerCase(),i=Wp[r],o=function(t,e,n){const r=[];for(let i=0;e&&i1&&(g=Math.sqrt(g),n*=g,r*=g);const m=h/n,y=f/n,v=-f/r,_=h/r,x=m*s+y*u,b=v*s+_*u,w=m*t+y*e,k=v*t+_*e;let A=1/((w-x)*(w-x)+(k-b)*(k-b))-.25;A<0&&(A=0);let M=Math.sqrt(A);o==i&&(M=-M);const E=.5*(x+w)-M*(k-b),D=.5*(b+k)+M*(w-x),C=Math.atan2(b-D,x-E);let F=Math.atan2(k-D,w-E)-C;F<0&&1===o?F+=Qp:F>0&&0===o&&(F-=Qp);const S=Math.ceil(Math.abs(F/(Zp+.001))),$=[];for(let t=0;t+t}function vg(t,e,n){return Math.max(e,Math.min(t,n))}function _g(){var t=dg,e=pg,n=gg,r=mg,i=yg(0),o=i,a=i,s=i,u=null;function l(l,c,f){var h,d=null!=c?c:+t.call(this,l),p=null!=f?f:+e.call(this,l),g=+n.call(this,l),m=+r.call(this,l),y=Math.min(g,m)/2,v=vg(+i.call(this,l),0,y),_=vg(+o.call(this,l),0,y),x=vg(+a.call(this,l),0,y),b=vg(+s.call(this,l),0,y);if(u||(u=h=Rl()),v<=0&&_<=0&&x<=0&&b<=0)u.rect(d,p,g,m);else{var w=d+g,k=p+m;u.moveTo(d+v,p),u.lineTo(w-_,p),u.bezierCurveTo(w-hg*_,p,w,p+hg*_,w,p+_),u.lineTo(w,k-b),u.bezierCurveTo(w,k-hg*b,w-hg*b,k,w-b,k),u.lineTo(d+x,k),u.bezierCurveTo(d+hg*x,k,d,k-hg*x,d,k-x),u.lineTo(d,p+v),u.bezierCurveTo(d,p+hg*v,d+hg*v,p,d+v,p),u.closePath()}if(h)return u=null,h+\"\"||null}return l.x=function(e){return arguments.length?(t=yg(e),l):t},l.y=function(t){return arguments.length?(e=yg(t),l):e},l.width=function(t){return arguments.length?(n=yg(t),l):n},l.height=function(t){return arguments.length?(r=yg(t),l):r},l.cornerRadius=function(t,e,n,r){return arguments.length?(i=yg(t),o=null!=e?yg(e):i,s=null!=n?yg(n):i,a=null!=r?yg(r):o,l):i},l.context=function(t){return arguments.length?(u=null==t?null:t,l):u},l}function xg(){var t,e,n,r,i,o,a,s,u=null;function l(t,e,n){const r=n/2;if(i){var l=a-e,c=t-o;if(l||c){var f=Math.hypot(l,c),h=(l/=f)*s,d=(c/=f)*s,p=Math.atan2(c,l);u.moveTo(o-h,a-d),u.lineTo(t-l*r,e-c*r),u.arc(t,e,r,p-Math.PI,p),u.lineTo(o+h,a+d),u.arc(o,a,s,p,p+Math.PI)}else u.arc(t,e,r,0,Qp);u.closePath()}else i=1;o=t,a=e,s=r}function c(o){var a,s,c,f=o.length,h=!1;for(null==u&&(u=c=Rl()),a=0;a<=f;++a)!(at.x||0,kg=t=>t.y||0,Ag=t=>!(!1===t.defined),Mg=function(){var t=Ll,e=ql,n=vl(0),r=null,i=Pl,o=jl,a=Il,s=null,u=Ul(l);function l(){var l,c,f=+t.apply(this,arguments),h=+e.apply(this,arguments),d=i.apply(this,arguments)-Cl,p=o.apply(this,arguments)-Cl,g=_l(p-d),m=p>d;if(s||(s=l=u()),hEl)if(g>Fl-El)s.moveTo(h*bl(d),h*Al(d)),s.arc(0,0,h,d,p,!m),f>El&&(s.moveTo(f*bl(p),f*Al(p)),s.arc(0,0,f,p,d,m));else{var y,v,_=d,x=p,b=d,w=p,k=g,A=g,M=a.apply(this,arguments)/2,E=M>El&&(r?+r.apply(this,arguments):Ml(f*f+h*h)),D=kl(_l(h-f)/2,+n.apply(this,arguments)),C=D,F=D;if(E>El){var S=Sl(E/f*Al(M)),$=Sl(E/h*Al(M));(k-=2*S)>El?(b+=S*=m?1:-1,w-=S):(k=0,b=w=(d+p)/2),(A-=2*$)>El?(_+=$*=m?1:-1,x-=$):(A=0,_=x=(d+p)/2)}var T=h*bl(_),B=h*Al(_),z=f*bl(w),N=f*Al(w);if(D>El){var O,R=h*bl(x),U=h*Al(x),L=f*bl(b),q=f*Al(b);if(g1?0:t<-1?Dl:Math.acos(t)}((P*I+j*W)/(Ml(P*P+j*j)*Ml(I*I+W*W)))/2),Y=Ml(O[0]*O[0]+O[1]*O[1]);C=kl(D,(f-Y)/(H-1)),F=kl(D,(h-Y)/(H+1))}else C=F=0}A>El?F>El?(y=Wl(L,q,T,B,h,F,m),v=Wl(R,U,z,N,h,F,m),s.moveTo(y.cx+y.x01,y.cy+y.y01),FEl&&k>El?C>El?(y=Wl(z,N,R,U,f,-C,m),v=Wl(T,B,L,q,f,-C,m),s.lineTo(y.cx+y.x01,y.cy+y.y01),Ct.startAngle||0)).endAngle((t=>t.endAngle||0)).padAngle((t=>t.padAngle||0)).innerRadius((t=>t.innerRadius||0)).outerRadius((t=>t.outerRadius||0)).cornerRadius((t=>t.cornerRadius||0)),Eg=Zl().x(wg).y1(kg).y0((t=>(t.y||0)+(t.height||0))).defined(Ag),Dg=Zl().y(kg).x1(wg).x0((t=>(t.x||0)+(t.width||0))).defined(Ag),Cg=Jl().x(wg).y(kg).defined(Ag),Fg=_g().x(wg).y(kg).width((t=>t.width||0)).height((t=>t.height||0)).cornerRadius((t=>bg(t.cornerRadiusTopLeft,t.cornerRadius)||0),(t=>bg(t.cornerRadiusTopRight,t.cornerRadius)||0),(t=>bg(t.cornerRadiusBottomRight,t.cornerRadius)||0),(t=>bg(t.cornerRadiusBottomLeft,t.cornerRadius)||0)),Sg=function(t,e){let n=null,r=Ul(i);function i(){let i;if(n||(n=i=r()),t.apply(this,arguments).draw(n,+e.apply(this,arguments)),i)return n=null,i+\"\"||null}return t=\"function\"==typeof t?t:vl(t||Ql),e=\"function\"==typeof e?e:vl(void 0===e?64:+e),i.type=function(e){return arguments.length?(t=\"function\"==typeof e?e:vl(e),i):t},i.size=function(t){return arguments.length?(e=\"function\"==typeof t?t:vl(+t),i):e},i.context=function(t){return arguments.length?(n=null==t?null:t,i):n},i}().type((t=>cg(t.shape||\"circle\"))).size((t=>bg(t.size,64))),$g=xg().x(wg).y(kg).defined(Ag).size((t=>t.size||1));function Tg(t){return t.cornerRadius||t.cornerRadiusTopLeft||t.cornerRadiusTopRight||t.cornerRadiusBottomRight||t.cornerRadiusBottomLeft}function Bg(t,e,n,r){return Fg.context(t)(e,n,r)}var zg=1;function Ng(){zg=1}function Og(t,e,n){var r=e.clip,i=t._defs,o=e.clip_id||(e.clip_id=\"clip\"+zg++),a=i.clipping[o]||(i.clipping[o]={id:o});return J(r)?a.path=r(null):Tg(n)?a.path=Bg(null,n,0,0):(a.width=n.width||0,a.height=n.height||0),\"url(#\"+o+\")\"}function Rg(t){this.clear(),t&&this.union(t)}function Ug(t){this.mark=t,this.bounds=this.bounds||new Rg}function Lg(t){Ug.call(this,t),this.items=this.items||[]}function qg(t){this._pending=0,this._loader=t||fa()}function Pg(t){t._pending+=1}function jg(t){t._pending-=1}function Ig(t,e,n){if(e.stroke&&0!==e.opacity&&0!==e.strokeOpacity){const r=null!=e.strokeWidth?+e.strokeWidth:1;t.expand(r+(n?function(t,e){return t.strokeJoin&&\"miter\"!==t.strokeJoin?0:e}(e,r):0))}return t}Rg.prototype={clone(){return new Rg(this)},clear(){return this.x1=+Number.MAX_VALUE,this.y1=+Number.MAX_VALUE,this.x2=-Number.MAX_VALUE,this.y2=-Number.MAX_VALUE,this},empty(){return this.x1===+Number.MAX_VALUE&&this.y1===+Number.MAX_VALUE&&this.x2===-Number.MAX_VALUE&&this.y2===-Number.MAX_VALUE},equals(t){return this.x1===t.x1&&this.y1===t.y1&&this.x2===t.x2&&this.y2===t.y2},set(t,e,n,r){return nthis.x2&&(this.x2=t),e>this.y2&&(this.y2=e),this},expand(t){return this.x1-=t,this.y1-=t,this.x2+=t,this.y2+=t,this},round(){return this.x1=Math.floor(this.x1),this.y1=Math.floor(this.y1),this.x2=Math.ceil(this.x2),this.y2=Math.ceil(this.y2),this},scale(t){return this.x1*=t,this.y1*=t,this.x2*=t,this.y2*=t,this},translate(t,e){return this.x1+=t,this.x2+=t,this.y1+=e,this.y2+=e,this},rotate(t,e,n){const r=this.rotatedPoints(t,e,n);return this.clear().add(r[0],r[1]).add(r[2],r[3]).add(r[4],r[5]).add(r[6],r[7])},rotatedPoints(t,e,n){var{x1:r,y1:i,x2:o,y2:a}=this,s=Math.cos(t),u=Math.sin(t),l=e-e*s+n*u,c=n-e*u-n*s;return[s*r-u*i+l,u*r+s*i+c,s*r-u*a+l,u*r+s*a+c,s*o-u*i+l,u*o+s*i+c,s*o-u*a+l,u*o+s*a+c]},union(t){return t.x1this.x2&&(this.x2=t.x2),t.y2>this.y2&&(this.y2=t.y2),this},intersect(t){return t.x1>this.x1&&(this.x1=t.x1),t.y1>this.y1&&(this.y1=t.y1),t.x2=t.x2&&this.y1<=t.y1&&this.y2>=t.y2},alignsWith(t){return t&&(this.x1==t.x1||this.x2==t.x2||this.y1==t.y1||this.y2==t.y2)},intersects(t){return t&&!(this.x2t.x2||this.y2t.y2)},contains(t,e){return!(tthis.x2||ethis.y2)},width(){return this.x2-this.x1},height(){return this.y2-this.y1}},dt(Lg,Ug),qg.prototype={pending(){return this._pending},sanitizeURL(t){const e=this;return Pg(e),e._loader.sanitize(t,{context:\"href\"}).then((t=>(jg(e),t))).catch((()=>(jg(e),null)))},loadImage(t){const e=this,n=Tc();return Pg(e),e._loader.sanitize(t,{context:\"image\"}).then((t=>{const r=t.href;if(!r||!n)throw{url:r};const i=new n,o=lt(t,\"crossOrigin\")?t.crossOrigin:\"anonymous\";return null!=o&&(i.crossOrigin=o),i.onload=()=>jg(e),i.onerror=()=>jg(e),i.src=r,i})).catch((t=>(jg(e),{complete:!1,width:0,height:0,src:t&&t.url||\"\"})))},ready(){const t=this;return new Promise((e=>{!function n(r){t.pending()?setTimeout((()=>{n(!0)}),10):e(r)}(!1)}))}};const Wg=Qp-1e-8;let Hg,Yg,Gg,Vg,Xg,Jg,Zg,Qg;const Kg=(t,e)=>Hg.add(t,e),tm=(t,e)=>Kg(Yg=t,Gg=e),em=t=>Kg(t,Hg.y1),nm=t=>Kg(Hg.x1,t),rm=(t,e)=>Xg*t+Zg*e,im=(t,e)=>Jg*t+Qg*e,om=(t,e)=>Kg(rm(t,e),im(t,e)),am=(t,e)=>tm(rm(t,e),im(t,e));function sm(t,e){return Hg=t,e?(Vg=e*Jp,Xg=Qg=Math.cos(Vg),Jg=Math.sin(Vg),Zg=-Jg):(Xg=Qg=1,Vg=Jg=Zg=0),um}const um={beginPath(){},closePath(){},moveTo:am,lineTo:am,rect(t,e,n,r){Vg?(om(t+n,e),om(t+n,e+r),om(t,e+r),am(t,e)):(Kg(t+n,e+r),tm(t,e))},quadraticCurveTo(t,e,n,r){const i=rm(t,e),o=im(t,e),a=rm(n,r),s=im(n,r);lm(Yg,i,a,em),lm(Gg,o,s,nm),tm(a,s)},bezierCurveTo(t,e,n,r,i,o){const a=rm(t,e),s=im(t,e),u=rm(n,r),l=im(n,r),c=rm(i,o),f=im(i,o);cm(Yg,a,u,c,em),cm(Gg,s,l,f,nm),tm(c,f)},arc(t,e,n,r,i,o){if(r+=Vg,i+=Vg,Yg=n*Math.cos(i)+t,Gg=n*Math.sin(i)+e,Math.abs(i-r)>Wg)Kg(t-n,e-n),Kg(t+n,e+n);else{const a=r=>Kg(n*Math.cos(r)+t,n*Math.sin(r)+e);let s,u;if(a(r),a(i),i!==r)if((r%=Qp)<0&&(r+=Qp),(i%=Qp)<0&&(i+=Qp),ii;++u,s-=Zp)a(s);else for(s=r-r%Zp+Zp,u=0;u<4&&s1e-14?(u=a*a+s*o,u>=0&&(u=Math.sqrt(u),l=(-a+u)/o,c=(-a-u)/o)):l=.5*s/a,0m)return!1;d>g&&(g=d)}else if(f>0){if(d0&&(t.globalAlpha=n,t.fillStyle=wm(t,e,e.fill),!0)}var Am=[];function Mm(t,e,n){var r=null!=(r=e.strokeWidth)?r:1;return!(r<=0)&&((n*=null==e.strokeOpacity?1:e.strokeOpacity)>0&&(t.globalAlpha=n,t.strokeStyle=wm(t,e,e.stroke),t.lineWidth=r,t.lineCap=e.strokeCap||\"butt\",t.lineJoin=e.strokeJoin||\"miter\",t.miterLimit=e.strokeMiterLimit||10,t.setLineDash&&(t.setLineDash(e.strokeDash||Am),t.lineDashOffset=e.strokeDashOffset||0),!0))}function Em(t,e){return t.zindex-e.zindex||t.index-e.index}function Dm(t){if(!t.zdirty)return t.zitems;var e,n,r,i=t.items,o=[];for(n=0,r=i.length;n=0;)if(n=e(i[r]))return n;if(i===o)for(r=(i=t.items).length;--r>=0;)if(!i[r].zindex&&(n=e(i[r])))return n;return null}function Sm(t){return function(e,n,r){Cm(n,(n=>{r&&!r.intersects(n.bounds)||Tm(t,e,n,n)}))}}function $m(t){return function(e,n,r){!n.items.length||r&&!r.intersects(n.bounds)||Tm(t,e,n.items[0],n.items)}}function Tm(t,e,n,r){var i=null==n.opacity?1:n.opacity;0!==i&&(t(e,r)||(_m(e,n),n.fill&&km(e,n,i)&&e.fill(),n.stroke&&Mm(e,n,i)&&e.stroke()))}function Bm(t){return t=t||p,function(e,n,r,i,o,a){return r*=e.pixelRatio,i*=e.pixelRatio,Fm(n,(n=>{const s=n.bounds;if((!s||s.contains(o,a))&&s)return t(e,n,r,i,o,a)?n:void 0}))}}function zm(t,e){return function(n,r,i,o){var a,s,u=Array.isArray(r)?r[0]:r,l=null==e?u.fill:e,c=u.stroke&&n.isPointInStroke;return c&&(a=u.strokeWidth,s=u.strokeCap,n.lineWidth=null!=a?a:1,n.lineCap=null!=s?s:\"butt\"),!t(n,r)&&(l&&n.isPointInPath(i,o)||c&&n.isPointInStroke(i,o))}}function Nm(t){return Bm(zm(t))}function Om(t,e){return\"translate(\"+t+\",\"+e+\")\"}function Rm(t){return\"rotate(\"+t+\")\"}function Um(t){return Om(t.x||0,t.y||0)}function Lm(t,e,n){function r(t,n){var r=n.x||0,i=n.y||0,o=n.angle||0;t.translate(r,i),o&&t.rotate(o*=Jp),t.beginPath(),e(t,n),o&&t.rotate(-o),t.translate(-r,-i)}return{type:t,tag:\"path\",nested:!1,attr:function(t,n){t(\"transform\",function(t){return Om(t.x||0,t.y||0)+(t.angle?\" \"+Rm(t.angle):\"\")}(n)),t(\"d\",e(null,n))},bound:function(t,n){return e(sm(t,n.angle),n),Ig(t,n).translate(n.x||0,n.y||0)},draw:Sm(r),pick:Nm(r),isect:n||pm(r)}}var qm=Lm(\"arc\",(function(t,e){return Mg.context(t)(e)}));function Pm(t,e,n){function r(t,n){t.beginPath(),e(t,n)}const i=zm(r);return{type:t,tag:\"path\",nested:!0,attr:function(t,n){var r=n.mark.items;r.length&&t(\"d\",e(null,r))},bound:function(t,n){var r=n.items;return 0===r.length?t:(e(sm(t),r),Ig(t,r[0]))},draw:$m(r),pick:function(t,e,n,r,o,a){var s=e.items,u=e.bounds;return!s||!s.length||u&&!u.contains(o,a)?null:(n*=t.pixelRatio,r*=t.pixelRatio,i(t,s,n,r)?s[0]:null)},isect:gm,tip:n}}var jm=Pm(\"area\",(function(t,e){const n=e[0],r=n.interpolate||\"linear\";return(\"horizontal\"===n.orient?Dg:Eg).curve(Ip(r,n.orient,n.tension)).context(t)(e)}),(function(t,e){for(var n,r,i=\"horizontal\"===t[0].orient?e[1]:e[0],o=\"horizontal\"===t[0].orient?\"y\":\"x\",a=t.length,s=1/0;--a>=0;)!1!==t[a].defined&&(r=Math.abs(t[a][o]-i)).5&&e<1.5?.5-Math.abs(e-1):0}function Hm(t,e){const n=Wm(e);t(\"d\",Bg(null,e,n,n))}function Ym(t,e,n,r){const i=Wm(e);t.beginPath(),Bg(t,e,(n||0)+i,(r||0)+i)}const Gm=zm(Ym),Vm=zm(Ym,!1),Xm=zm(Ym,!0);var Jm={type:\"group\",tag:\"g\",nested:!1,attr:function(t,e){t(\"transform\",Um(e))},bound:function(t,e){if(!e.clip&&e.items){const n=e.items,r=n.length;for(let e=0;e{const i=e.x||0,o=e.y||0,a=e.strokeForeground,s=null==e.opacity?1:e.opacity;(e.stroke||e.fill)&&s&&(Ym(t,e,i,o),_m(t,e),e.fill&&km(t,e,s)&&t.fill(),e.stroke&&!a&&Mm(t,e,s)&&t.stroke()),t.save(),t.translate(i,o),e.clip&&Im(t,e),n&&n.translate(-i,-o),Cm(e,(e=>{(\"group\"===e.marktype||null==r||r.includes(e.marktype))&&this.draw(t,e,n,r)})),n&&n.translate(i,o),t.restore(),a&&e.stroke&&s&&(Ym(t,e,i,o),_m(t,e),Mm(t,e,s)&&t.stroke())}))},pick:function(t,e,n,r,i,o){if(e.bounds&&!e.bounds.contains(i,o)||!e.items)return null;const a=n*t.pixelRatio,s=r*t.pixelRatio;return Fm(e,(u=>{let l,c,f;const h=u.bounds;if(h&&!h.contains(i,o))return;c=u.x||0,f=u.y||0;const d=c+(u.width||0),p=f+(u.height||0),g=u.clip;if(g&&(id||op))return;if(t.save(),t.translate(c,f),c=i-c,f=o-f,g&&Tg(u)&&!Xm(t,u,a,s))return t.restore(),null;const m=u.strokeForeground,y=!1!==e.interactive;return y&&m&&u.stroke&&Vm(t,u,a,s)?(t.restore(),u):(l=Fm(u,(t=>function(t,e,n){return(!1!==t.interactive||\"group\"===t.marktype)&&t.bounds&&t.bounds.contains(e,n)}(t,c,f)?this.pick(t,n,r,c,f):null)),!l&&y&&(u.fill||!m&&u.stroke)&&Gm(t,u,a,s)&&(l=u),t.restore(),l||null)}))},isect:mm,content:function(t,e,n){t(\"clip-path\",e.clip?Og(n,e,e):null)},background:function(t,e){t(\"class\",\"background\"),t(\"aria-hidden\",!0),Hm(t,e)},foreground:function(t,e){t(\"class\",\"foreground\"),t(\"aria-hidden\",!0),e.strokeForeground?Hm(t,e):t(\"d\",\"\")}},Zm={xmlns:\"http://www.w3.org/2000/svg\",\"xmlns:xlink\":\"http://www.w3.org/1999/xlink\",version:\"1.1\"};function Qm(t,e){var n=t.image;return(!n||t.url&&t.url!==n.url)&&(n={complete:!1,width:0,height:0},e.loadImage(t.url).then((e=>{t.image=e,t.image.url=t.url}))),n}function Km(t,e){return null!=t.width?t.width:e&&e.width?!1!==t.aspect&&t.height?t.height*e.width/e.height:e.width:0}function ty(t,e){return null!=t.height?t.height:e&&e.height?!1!==t.aspect&&t.width?t.width*e.height/e.width:e.height:0}function ey(t,e){return\"center\"===t?e/2:\"right\"===t?e:0}function ny(t,e){return\"middle\"===t?e/2:\"bottom\"===t?e:0}var ry={type:\"image\",tag:\"image\",nested:!1,attr:function(t,e,n){const r=Qm(e,n),i=Km(e,r),o=ty(e,r),a=(e.x||0)-ey(e.align,i),s=(e.y||0)-ny(e.baseline,o);t(\"href\",!r.src&&r.toDataURL?r.toDataURL():r.src||\"\",Zm[\"xmlns:xlink\"],\"xlink:href\"),t(\"transform\",Om(a,s)),t(\"width\",i),t(\"height\",o),t(\"preserveAspectRatio\",!1===e.aspect?\"none\":\"xMidYMid\")},bound:function(t,e){const n=e.image,r=Km(e,n),i=ty(e,n),o=(e.x||0)-ey(e.align,r),a=(e.y||0)-ny(e.baseline,i);return t.set(o,a,o+r,a+i)},draw:function(t,e,n){Cm(e,(e=>{if(n&&!n.intersects(e.bounds))return;const r=Qm(e,this);let i=Km(e,r),o=ty(e,r);if(0===i||0===o)return;let a,s,u,l,c=(e.x||0)-ey(e.align,i),f=(e.y||0)-ny(e.baseline,o);!1!==e.aspect&&(s=r.width/r.height,u=e.width/e.height,s==s&&u==u&&s!==u&&(u=0;)if(!1!==t[o].defined&&(n=t[o].x-e[0])*n+(r=t[o].y-e[1])*r{if(!n||n.intersects(e.bounds)){var r=null==e.opacity?1:e.opacity;r&&ly(t,e,r)&&(_m(t,e),t.stroke())}}))},pick:Bm((function(t,e,n,r){return!!t.isPointInStroke&&(ly(t,e,1)&&t.isPointInStroke(n,r))})),isect:ym},fy=Lm(\"shape\",(function(t,e){return(e.mark.shape||e.shape).context(t)(e)})),hy=Lm(\"symbol\",(function(t,e){return Sg.context(t)(e)}),gm);const dy=kt();var py={height:xy,measureWidth:vy,estimateWidth:my,width:my,canvas:gy};function gy(t){py.width=t&&hm?vy:my}function my(t,e){return yy(Ay(t,e),xy(t))}function yy(t,e){return~~(.8*t.length*e)}function vy(t,e){return xy(t)<=0||!(e=Ay(t,e))?0:_y(e,Ey(t))}function _y(t,e){const n=`(${e}) ${t}`;let r=dy.get(n);return void 0===r&&(hm.font=e,r=hm.measureText(t).width,dy.set(n,r)),r}function xy(t){return null!=t.fontSize?+t.fontSize||0:11}function by(t){return null!=t.lineHeight?t.lineHeight:xy(t)+2}function wy(t){return e=t.lineBreak&&t.text&&!k(t.text)?t.text.split(t.lineBreak):t.text,k(e)?e.length>1?e:e[0]:e;var e}function ky(t){const e=wy(t);return(k(e)?e.length-1:0)*by(t)}function Ay(t,e){const n=null==e?\"\":(e+\"\").trim();return t.limit>0&&n.length?function(t,e){var n=+t.limit,r=function(t){if(py.width===vy){const e=Ey(t);return t=>_y(t,e)}if(py.width===my){const e=xy(t);return t=>yy(t,e)}return e=>py.width(t,e)}(t);if(r(e)>>1,r(e.slice(i))>n?s=i+1:u=i;return o+e.slice(s)}for(;s>>1),r(e.slice(0,i))Math.max(t,py.width(e,n))),0)):r=py.width(e,f),\"center\"===o?l-=r/2:\"right\"===o&&(l-=r),t.set(l+=s,c+=u,l+r,c+i),e.angle&&!n)t.rotate(e.angle*Jp,s,u);else if(2===n)return t.rotatedPoints(e.angle*Jp,s,u);return t}var Ty={type:\"text\",tag:\"text\",nested:!1,attr:function(t,e){var n,r=e.dx||0,i=(e.dy||0)+Dy(e),o=Sy(e),a=o.x1,s=o.y1,u=e.angle||0;t(\"text-anchor\",Cy[e.align]||\"start\"),u?(n=Om(a,s)+\" \"+Rm(u),(r||i)&&(n+=\" \"+Om(r,i))):n=Om(a+r,s+i),t(\"transform\",n)},bound:$y,draw:function(t,e,n){Cm(e,(e=>{var r,i,o,a,s,u,l,c=null==e.opacity?1:e.opacity;if(!(n&&!n.intersects(e.bounds)||0===c||e.fontSize<=0||null==e.text||0===e.text.length)){if(t.font=Ey(e),t.textAlign=e.align||\"left\",i=(r=Sy(e)).x1,o=r.y1,e.angle&&(t.save(),t.translate(i,o),t.rotate(e.angle*Jp),i=o=0),i+=e.dx||0,o+=(e.dy||0)+Dy(e),u=wy(e),_m(t,e),k(u))for(s=by(e),a=0;a=0;)if(!1!==t[i].defined&&(n=t[i].x-e[0])*n+(r=t[i].y-e[1])*r<(n=t[i].size||1)*n)return t[i];return null})),zy={arc:qm,area:jm,group:Jm,image:ry,line:iy,path:ay,rect:uy,rule:cy,shape:fy,symbol:hy,text:Ty,trail:By};function Ny(t,e,n){var r=zy[t.mark.marktype],i=e||r.bound;return r.nested&&(t=t.mark),i(t.bounds||(t.bounds=new Rg),t,n)}var Oy={mark:null};function Ry(t,e,n){var r,i,o,a,s=zy[t.marktype],u=s.bound,l=t.items,c=l&&l.length;if(s.nested)return c?o=l[0]:(Oy.mark=t,o=Oy),a=Ny(o,u,n),e=e&&e.union(a)||a;if(e=e||t.bounds&&t.bounds.clear()||new Rg,c)for(r=0,i=l.length;re;)t.removeChild(n[--r]);return t}function Vy(t){return\"mark-\"+t.marktype+(t.role?\" role-\"+t.role:\"\")+(t.name?\" \"+t.name:\"\")}function Xy(t,e){const n=e.getBoundingClientRect();return[t.clientX-n.left-(e.clientLeft||0),t.clientY-n.top-(e.clientTop||0)]}function Jy(t,e){this._active=null,this._handlers={},this._loader=t||fa(),this._tooltip=e||Zy}function Zy(t,e,n,r){t.element().setAttribute(\"title\",r||\"\")}function Qy(t){this._el=null,this._bgcolor=null,this._loader=new qg(t)}jy.prototype={toJSON(t){return Ly(this.root,t||0)},mark(t,e,n){const r=Iy(t,e=e||this.root.items[0]);return e.items[n]=r,r.zindex&&(r.group.zdirty=!0),r}},Jy.prototype={initialize(t,e,n){return this._el=t,this._obj=n||null,this.origin(e)},element(){return this._el},canvas(){return this._el&&this._el.firstChild},origin(t){return arguments.length?(this._origin=t||[0,0],this):this._origin.slice()},scene(t){return arguments.length?(this._scene=t,this):this._scene},on(){},off(){},_handlerIndex(t,e,n){for(let r=t?t.length:0;--r>=0;)if(t[r].type===e&&(!n||t[r].handler===n))return r;return-1},handlers(t){const e=this._handlers,n=[];if(t)n.push(...e[this.eventName(t)]);else for(const t in e)n.push(...e[t]);return n},eventName(t){const e=t.indexOf(\".\");return e<0?t:t.slice(0,e)},handleHref(t,e,n){this._loader.sanitize(n,{context:\"href\"}).then((e=>{const n=new MouseEvent(t.type,t),r=Wy(null,\"a\");for(const t in e)r.setAttribute(t,e[t]);r.dispatchEvent(n)})).catch((()=>{}))},handleTooltip(t,e,n){if(e&&null!=e.tooltip){e=function(t,e,n,r){var i,o,a=t&&t.mark;if(a&&(i=zy[a.marktype]).tip){for((o=Xy(e,n))[0]-=r[0],o[1]-=r[1];t=t.mark.group;)o[0]-=t.x||0,o[1]-=t.y||0;t=i.tip(a.items,o)}return t}(e,t,this.canvas(),this._origin);const r=n&&e&&e.tooltip||null;this._tooltip.call(this._obj,this,t,e,r)}},getItemBoundingClientRect(t){const e=this.canvas();if(!e)return;const n=e.getBoundingClientRect(),r=this._origin,i=t.bounds,o=i.width(),a=i.height();let s=i.x1+r[0]+n.left,u=i.y1+r[1]+n.top;for(;t.mark&&(t=t.mark.group);)s+=t.x||0,u+=t.y||0;return{x:s,y:u,width:o,height:a,left:s,top:u,right:s+o,bottom:u+a}}},Qy.prototype={initialize(t,e,n,r,i){return this._el=t,this.resize(e,n,r,i)},element(){return this._el},canvas(){return this._el&&this._el.firstChild},background(t){return 0===arguments.length?this._bgcolor:(this._bgcolor=t,this)},resize(t,e,n,r){return this._width=t,this._height=e,this._origin=n||[0,0],this._scale=r||1,this},dirty(){},render(t,e){const n=this;return n._call=function(){n._render(t,e)},n._call(),n._call=null,n},_render(){},renderAsync(t,e){const n=this.render(t,e);return this._ready?this._ready.then((()=>n)):Promise.resolve(n)},_load(t,e){var n=this,r=n._loader[t](e);if(!n._ready){const t=n._call;n._ready=n._loader.ready().then((e=>{e&&t(),n._ready=null}))}return r},sanitizeURL(t){return this._load(\"sanitizeURL\",t)},loadImage(t){return this._load(\"loadImage\",t)}};const Ky=\"dragenter\",tv=\"dragleave\",ev=\"dragover\",nv=\"pointerdown\",rv=\"pointermove\",iv=\"pointerout\",ov=\"pointerover\",av=\"mousedown\",sv=\"mousemove\",uv=\"mouseout\",lv=\"mouseover\",cv=\"click\",fv=\"mousewheel\",hv=\"touchstart\",dv=\"touchmove\",pv=\"touchend\",gv=rv,mv=iv,yv=cv;function vv(t,e){Jy.call(this,t,e),this._down=null,this._touch=null,this._first=!0,this._events={}}function _v(t,e){(t=>t===hv||t===dv||t===pv?[hv,dv,pv]:[t])(e).forEach((e=>function(t,e){const n=t.canvas();n&&!t._events[e]&&(t._events[e]=1,n.addEventListener(e,t[e]?n=>t[e](n):n=>t.fire(e,n)))}(t,e)))}function xv(t,e,n){e.forEach((e=>t.fire(e,n)))}function bv(t,e,n){return function(r){const i=this._active,o=this.pickEvent(r);o===i||(i&&i.exit||xv(this,n,r),this._active=o,xv(this,e,r)),xv(this,t,r)}}function wv(t){return function(e){xv(this,t,e),this._active=null}}function kv(t,e,n,r,i,o){const a=\"undefined\"!=typeof HTMLElement&&t instanceof HTMLElement&&null!=t.parentNode,s=t.getContext(\"2d\"),u=a?\"undefined\"!=typeof window&&window.devicePixelRatio||1:i;t.width=e*u,t.height=n*u;for(const t in o)s[t]=o[t];return a&&1!==u&&(t.style.width=e+\"px\",t.style.height=n+\"px\"),s.pixelRatio=u,s.setTransform(u,0,0,u,u*r[0],u*r[1]),t}function Av(t){Qy.call(this,t),this._options={},this._redraw=!1,this._dirty=new Rg,this._tempb=new Rg}dt(vv,Jy,{initialize(t,e,n){return this._canvas=t&&Hy(t,\"canvas\"),[cv,av,nv,rv,iv,tv].forEach((t=>_v(this,t))),Jy.prototype.initialize.call(this,t,e,n)},canvas(){return this._canvas},context(){return this._canvas.getContext(\"2d\")},events:[\"keydown\",\"keypress\",\"keyup\",Ky,tv,ev,nv,\"pointerup\",rv,iv,ov,av,\"mouseup\",sv,uv,lv,cv,\"dblclick\",\"wheel\",fv,hv,dv,pv],DOMMouseScroll(t){this.fire(fv,t)},pointermove:bv([rv,sv],[ov,lv],[iv,uv]),dragover:bv([ev],[Ky],[tv]),pointerout:wv([iv,uv]),dragleave:wv([tv]),pointerdown(t){this._down=this._active,this.fire(nv,t)},mousedown(t){this._down=this._active,this.fire(av,t)},click(t){this._down===this._active&&(this.fire(cv,t),this._down=null)},touchstart(t){this._touch=this.pickEvent(t.changedTouches[0]),this._first&&(this._active=this._touch,this._first=!1),this.fire(hv,t,!0)},touchmove(t){this.fire(dv,t,!0)},touchend(t){this.fire(pv,t,!0),this._touch=null},fire(t,e,n){const r=n?this._touch:this._active,i=this._handlers[t];if(e.vegaType=t,t===yv&&r&&r.href?this.handleHref(e,r,r.href):t!==gv&&t!==mv||this.handleTooltip(e,r,t!==mv),i)for(let t=0,n=i.length;t=0&&r.splice(i,1),this},pickEvent(t){const e=Xy(t,this._canvas),n=this._origin;return this.pick(this._scene,e[0],e[1],e[0]-n[0],e[1]-n[1])},pick(t,e,n,r,i){const o=this.context();return zy[t.marktype].pick.call(this,o,t,e,n,r,i)}});const Mv=Qy.prototype;function Ev(t,e){Jy.call(this,t,e);const n=this;n._hrefHandler=Dv(n,((t,e)=>{e&&e.href&&n.handleHref(t,e,e.href)})),n._tooltipHandler=Dv(n,((t,e)=>{n.handleTooltip(t,e,t.type!==mv)}))}dt(Av,Qy,{initialize(t,e,n,r,i,o){return this._options=o||{},this._canvas=this._options.externalContext?null:$c(1,1,this._options.type),t&&this._canvas&&(Gy(t,0).appendChild(this._canvas),this._canvas.setAttribute(\"class\",\"marks\")),Mv.initialize.call(this,t,e,n,r,i)},resize(t,e,n,r){if(Mv.resize.call(this,t,e,n,r),this._canvas)kv(this._canvas,this._width,this._height,this._origin,this._scale,this._options.context);else{const t=this._options.externalContext;t||s(\"CanvasRenderer is missing a valid canvas or context\"),t.scale(this._scale,this._scale),t.translate(this._origin[0],this._origin[1])}return this._redraw=!0,this},canvas(){return this._canvas},context(){return this._options.externalContext||(this._canvas?this._canvas.getContext(\"2d\"):null)},dirty(t){const e=this._tempb.clear().union(t.bounds);let n=t.mark.group;for(;n;)e.translate(n.x||0,n.y||0),n=n.mark.group;this._dirty.union(e)},_render(t,e){const n=this.context(),r=this._origin,i=this._width,o=this._height,a=this._dirty,s=((t,e,n)=>(new Rg).set(0,0,e,n).translate(-t[0],-t[1]))(r,i,o);n.save();const u=this._redraw||a.empty()?(this._redraw=!1,s.expand(1)):function(t,e,n){return e.expand(1).round(),t.pixelRatio%1&&e.scale(t.pixelRatio).round().scale(1/t.pixelRatio),e.translate(-n[0]%1,-n[1]%1),t.beginPath(),t.rect(e.x1,e.y1,e.width(),e.height()),t.clip(),e}(n,s.intersect(a),r);return this.clear(-r[0],-r[1],i,o),this.draw(n,t,u,e),n.restore(),a.clear(),this},draw(t,e,n,r){if(\"group\"!==e.marktype&&null!=r&&!r.includes(e.marktype))return;const i=zy[e.marktype];e.clip&&function(t,e){var n=e.clip;t.save(),J(n)?(t.beginPath(),n(t),t.clip()):Im(t,e.group)}(t,e),i.draw.call(this,t,e,n,r),e.clip&&t.restore()},clear(t,e,n,r){const i=this._options,o=this.context();\"pdf\"===i.type||i.externalContext||o.clearRect(t,e,n,r),null!=this._bgcolor&&(o.fillStyle=this._bgcolor,o.fillRect(t,e,n,r))}});const Dv=(t,e)=>n=>{let r=n.target.__data__;r=Array.isArray(r)?r[0]:r,n.vegaType=n.type,e.call(t._obj,n,r)};dt(Ev,Jy,{initialize(t,e,n){let r=this._svg;return r&&(r.removeEventListener(yv,this._hrefHandler),r.removeEventListener(gv,this._tooltipHandler),r.removeEventListener(mv,this._tooltipHandler)),this._svg=r=t&&Hy(t,\"svg\"),r&&(r.addEventListener(yv,this._hrefHandler),r.addEventListener(gv,this._tooltipHandler),r.addEventListener(mv,this._tooltipHandler)),Jy.prototype.initialize.call(this,t,e,n)},canvas(){return this._svg},on(t,e){const n=this.eventName(t),r=this._handlers;if(this._handlerIndex(r[n],t,e)<0){const i={type:t,handler:e,listener:Dv(this,e)};(r[n]||(r[n]=[])).push(i),this._svg&&this._svg.addEventListener(n,i.listener)}return this},off(t,e){const n=this.eventName(t),r=this._handlers[n],i=this._handlerIndex(r,t,e);return i>=0&&(this._svg&&this._svg.removeEventListener(n,r[i].listener),r.splice(i,1)),this}});const Cv=\"aria-hidden\",Fv=\"aria-label\",Sv=\"role\",$v=\"aria-roledescription\",Tv=\"graphics-object\",Bv=\"graphics-symbol\",zv=(t,e,n)=>({[Sv]:t,[$v]:e,[Fv]:n||void 0}),Nv=Bt([\"axis-domain\",\"axis-grid\",\"axis-label\",\"axis-tick\",\"axis-title\",\"legend-band\",\"legend-entry\",\"legend-gradient\",\"legend-label\",\"legend-title\",\"legend-symbol\",\"title\"]),Ov={axis:{desc:\"axis\",caption:function(t){const e=t.datum,n=t.orient,r=e.title?Pv(t):null,i=t.context,o=i.scales[e.scale].value,a=i.dataflow.locale(),s=o.type;return(\"left\"===n||\"right\"===n?\"Y\":\"X\")+\"-axis\"+(r?` titled '${r}'`:\"\")+` for a ${Kd(s)?\"discrete\":s} scale`+` with ${Np(a,o,t)}`}},legend:{desc:\"legend\",caption:function(t){const e=t.datum,n=e.title?Pv(t):null,r=`${e.type||\"\"} legend`.trim(),i=e.scales,o=Object.keys(i),a=t.context,s=a.scales[i[o[0]]].value,u=a.dataflow.locale();return l=r,(l.length?l[0].toUpperCase()+l.slice(1):l)+(n?` titled '${n}'`:\"\")+` for ${function(t){return t=t.map((t=>t+(\"fill\"===t||\"stroke\"===t?\" color\":\"\"))),t.length<2?t[0]:t.slice(0,-1).join(\", \")+\" and \"+F(t)}(o)}`+` with ${Np(u,s,t)}`;var l}},\"title-text\":{desc:\"title\",caption:t=>`Title text '${qv(t)}'`},\"title-subtitle\":{desc:\"subtitle\",caption:t=>`Subtitle text '${qv(t)}'`}},Rv={ariaRole:Sv,ariaRoleDescription:$v,description:Fv};function Uv(t,e){const n=!1===e.aria;if(t(Cv,n||void 0),n||null==e.description)for(const e in Rv)t(Rv[e],void 0);else{const n=e.mark.marktype;t(Fv,e.description),t(Sv,e.ariaRole||(\"group\"===n?Tv:Bv)),t($v,e.ariaRoleDescription||`${n} mark`)}}function Lv(t){return!1===t.aria?{[Cv]:!0}:Nv[t.role]?null:Ov[t.role]?function(t,e){try{const n=t.items[0],r=e.caption||(()=>\"\");return zv(e.role||Bv,e.desc,n.description||r(n))}catch(t){return null}}(t,Ov[t.role]):function(t){const e=t.marktype,n=\"group\"===e||\"text\"===e||t.items.some((t=>null!=t.description&&!1!==t.aria));return zv(n?Tv:Bv,`${e} mark container`,t.description)}(t)}function qv(t){return V(t.text).join(\" \")}function Pv(t){try{return V(F(t.items).items[0].text).join(\" \")}catch(t){return null}}const jv=t=>(t+\"\").replace(/&/g,\"&\").replace(//g,\">\");function Iv(){let t=\"\",e=\"\",n=\"\";const r=[],i=()=>e=n=\"\",o=(t,n)=>{var r;return null!=n&&(e+=` ${t}=\"${r=n,jv(r).replace(/\"/g,\""\").replace(/\\t/g,\" \").replace(/\\n/g,\" \").replace(/\\r/g,\" \")}\"`),a},a={open(s){(o=>{e&&(t+=`${e}>${n}`,i()),r.push(o)})(s),e=\"<\"+s;for(var u=arguments.length,l=new Array(u>1?u-1:0),c=1;c${n}`:\"/>\"):``,i(),a},attr:o,text:t=>(n+=jv(t),a),toString:()=>t};return a}const Wv=t=>Hv(Iv(),t)+\"\";function Hv(t,e){if(t.open(e.tagName),e.hasAttributes()){const n=e.attributes,r=n.length;for(let e=0;e1&&t.previousSibling!=e}(a,n))&&e.insertBefore(a,n?n.nextSibling:e.firstChild),a}dt(Zv,Qy,{initialize(t,e,n,r,i){return this._defs={},this._clearDefs(),t&&(this._svg=Yy(t,0,\"svg\",Jv),this._svg.setAttributeNS(Xv,\"xmlns\",Jv),this._svg.setAttributeNS(Xv,\"xmlns:xlink\",Zm[\"xmlns:xlink\"]),this._svg.setAttribute(\"version\",Zm.version),this._svg.setAttribute(\"class\",\"marks\"),Gy(t,1),this._root=Yy(this._svg,0,\"g\",Jv),u_(this._root,Vv),Gy(this._svg,1)),this.background(this._bgcolor),Qv.initialize.call(this,t,e,n,r,i)},background(t){return arguments.length&&this._svg&&this._svg.style.setProperty(\"background-color\",t),Qv.background.apply(this,arguments)},resize(t,e,n,r){return Qv.resize.call(this,t,e,n,r),this._svg&&(u_(this._svg,{width:this._width*this._scale,height:this._height*this._scale,viewBox:`0 0 ${this._width} ${this._height}`}),this._root.setAttribute(\"transform\",`translate(${this._origin})`)),this._dirty=[],this},canvas(){return this._svg},svg(){const t=this._svg,e=this._bgcolor;if(!t)return null;let n;e&&(t.removeAttribute(\"style\"),n=Yy(t,0,\"rect\",Jv),u_(n,{width:this._width,height:this._height,fill:e}));const r=Wv(t);return e&&(t.removeChild(n),this._svg.style.setProperty(\"background-color\",e)),r},_render(t,e){return this._dirtyCheck()&&(this._dirtyAll&&this._clearDefs(),this.mark(this._root,t,void 0,e),Gy(this._root,1)),this.defs(),this._dirty=[],++this._dirtyID,this},dirty(t){t.dirty!==this._dirtyID&&(t.dirty=this._dirtyID,this._dirty.push(t))},isDirty(t){return this._dirtyAll||!t._svg||!t._svg.ownerSVGElement||t.dirty===this._dirtyID},_dirtyCheck(){this._dirtyAll=!0;const t=this._dirty;if(!t.length||!this._dirtyID)return!0;const e=++this._dirtyID;let n,r,i,o,a,s,u;for(a=0,s=t.length;a{t.dirty=e}))),r.zdirty||(n.exit?(o.nested&&r.items.length?(u=r.items[0],u._svg&&this._update(o,u._svg,u)):n._svg&&(u=n._svg.parentNode,u&&u.removeChild(n._svg)),n._svg=null):(n=o.nested?r.items[0]:n,n._update!==e&&(n._svg&&n._svg.ownerSVGElement?this._update(o,n._svg,n):(this._dirtyAll=!1,Kv(n,e)),n._update=e)));return!this._dirtyAll},mark(t,e,n,r){if(!this.isDirty(e))return e._svg;const i=this._svg,o=e.marktype,a=zy[o],s=!1===e.interactive?\"none\":null,u=\"g\"===a.tag,l=n_(e,t,n,\"g\",i);if(\"group\"!==o&&null!=r&&!r.includes(o))return Gy(l,0),e._svg;l.setAttribute(\"class\",Vy(e));const c=Lv(e);for(const t in c)l_(l,t,c[t]);u||l_(l,\"pointer-events\",s),l_(l,\"clip-path\",e.clip?Og(this,e,e.group):null);let f=null,h=0;const d=t=>{const e=this.isDirty(t),n=n_(t,l,f,a.tag,i);e&&(this._update(a,n,t),u&&function(t,e,n,r){e=e.lastChild.previousSibling;let i,o=0;Cm(n,(n=>{i=t.mark(e,n,i,r),++o})),Gy(e,1+o)}(this,n,t,r)),f=n,++h};return a.nested?e.items.length&&d(e.items[0]):Cm(e,d),Gy(l,h),l},_update(t,e,n){r_=e,i_=e.__values__,Uv(a_,n),t.attr(a_,n,this);const r=o_[t.type];r&&r.call(this,t,e,n),r_&&this.style(r_,n)},style(t,e){if(null!=e){for(const n in Yv){let r=\"font\"===n?My(e):e[n];if(r===i_[n])continue;const i=Yv[n];null==r?t.removeAttribute(i):(Up(r)&&(r=Lp(r,this._defs.gradient,c_())),t.setAttribute(i,r+\"\")),i_[n]=r}for(const n in Gv)s_(t,Gv[n],e[n])}},defs(){const t=this._svg,e=this._defs;let n=e.el,r=0;for(const i in e.gradient)n||(e.el=n=Yy(t,1,\"defs\",Jv)),r=t_(n,e.gradient[i],r);for(const i in e.clipping)n||(e.el=n=Yy(t,1,\"defs\",Jv)),r=e_(n,e.clipping[i],r);n&&(0===r?(t.removeChild(n),e.el=null):Gy(n,r))},_clearDefs(){const t=this._defs;t.gradient={},t.clipping={}}});let r_=null,i_=null;const o_={group(t,e,n){const r=r_=e.childNodes[2];i_=r.__values__,t.foreground(a_,n,this),i_=e.__values__,r_=e.childNodes[1],t.content(a_,n,this);const i=r_=e.childNodes[0];t.background(a_,n,this);const o=!1===n.mark.interactive?\"none\":null;if(o!==i_.events&&(l_(r,\"pointer-events\",o),l_(i,\"pointer-events\",o),i_.events=o),n.strokeForeground&&n.stroke){const t=n.fill;l_(r,\"display\",null),this.style(i,n),l_(i,\"stroke\",null),t&&(n.fill=null),i_=r.__values__,this.style(r,n),t&&(n.fill=t),r_=null}else l_(r,\"display\",\"none\")},image(t,e,n){!1===n.smooth?(s_(e,\"image-rendering\",\"optimizeSpeed\"),s_(e,\"image-rendering\",\"pixelated\")):s_(e,\"image-rendering\",null)},text(t,e,n){const r=wy(n);let i,o,a,s;k(r)?(o=r.map((t=>Ay(n,t))),i=o.join(\"\\n\"),i!==i_.text&&(Gy(e,0),a=e.ownerDocument,s=by(n),o.forEach(((t,r)=>{const i=Wy(a,\"tspan\",Jv);i.__data__=n,i.textContent=t,r&&(i.setAttribute(\"x\",0),i.setAttribute(\"dy\",s)),e.appendChild(i)})),i_.text=i)):(o=Ay(n,r),o!==i_.text&&(e.textContent=o,i_.text=o)),l_(e,\"font-family\",My(n)),l_(e,\"font-size\",xy(n)+\"px\"),l_(e,\"font-style\",n.fontStyle),l_(e,\"font-variant\",n.fontVariant),l_(e,\"font-weight\",n.fontWeight)}};function a_(t,e,n){e!==i_[t]&&(n?function(t,e,n,r){null!=n?t.setAttributeNS(r,e,n):t.removeAttributeNS(r,e)}(r_,t,e,n):l_(r_,t,e),i_[t]=e)}function s_(t,e,n){n!==i_[e]&&(null==n?t.style.removeProperty(e):t.style.setProperty(e,n+\"\"),i_[e]=n)}function u_(t,e){for(const n in e)l_(t,n,e[n])}function l_(t,e,n){null!=n?t.setAttribute(e,n):t.removeAttribute(e)}function c_(){let t;return\"undefined\"==typeof window?\"\":(t=window.location).hash?t.href.slice(0,-t.hash.length):t.href}function f_(t){Qy.call(this,t),this._text=null,this._defs={gradient:{},clipping:{}}}dt(f_,Qy,{svg(){return this._text},_render(t){const e=Iv();e.open(\"svg\",ot({},Zm,{class:\"marks\",width:this._width*this._scale,height:this._height*this._scale,viewBox:`0 0 ${this._width} ${this._height}`}));const n=this._bgcolor;return n&&\"transparent\"!==n&&\"none\"!==n&&e.open(\"rect\",{width:this._width,height:this._height,fill:n}).close(),e.open(\"g\",Vv,{transform:\"translate(\"+this._origin+\")\"}),this.mark(e,t),e.close(),this.defs(e),this._text=e.close()+\"\",this},mark(t,e){const n=zy[e.marktype],r=n.tag,i=[Uv,n.attr];t.open(\"g\",{class:Vy(e),\"clip-path\":e.clip?Og(this,e,e.group):null},Lv(e),{\"pointer-events\":\"g\"!==r&&!1===e.interactive?\"none\":null});const o=o=>{const a=this.href(o);if(a&&t.open(\"a\",a),t.open(r,this.attr(e,o,i,\"g\"!==r?r:null)),\"text\"===r){const e=wy(o);if(k(e)){const n={x:0,dy:by(o)};for(let r=0;rthis.mark(t,e))),t.close(),r&&a?(i&&(o.fill=null),o.stroke=a,t.open(\"path\",this.attr(e,o,n.foreground,\"bgrect\")).close(),i&&(o.fill=i)):t.open(\"path\",this.attr(e,o,n.foreground,\"bgfore\")).close()}t.close(),a&&t.close()};return n.nested?e.items&&e.items.length&&o(e.items[0]):Cm(e,o),t.close()},href(t){const e=t.href;let n;if(e){if(n=this._hrefs&&this._hrefs[e])return n;this.sanitizeURL(e).then((t=>{t[\"xlink:href\"]=t.href,t.href=null,(this._hrefs||(this._hrefs={}))[e]=t}))}return null},attr(t,e,n,r){const i={},o=(t,e,n,r)=>{i[r||t]=e};return Array.isArray(n)?n.forEach((t=>t(o,e,this))):n(o,e,this),r&&function(t,e,n,r,i){let o;if(null==e)return t;\"bgrect\"===r&&!1===n.interactive&&(t[\"pointer-events\"]=\"none\");if(\"bgfore\"===r&&(!1===n.interactive&&(t[\"pointer-events\"]=\"none\"),t.display=\"none\",null!==e.fill))return t;\"image\"===r&&!1===e.smooth&&(o=[\"image-rendering: optimizeSpeed;\",\"image-rendering: pixelated;\"]);\"text\"===r&&(t[\"font-family\"]=My(e),t[\"font-size\"]=xy(e)+\"px\",t[\"font-style\"]=e.fontStyle,t[\"font-variant\"]=e.fontVariant,t[\"font-weight\"]=e.fontWeight);for(const n in Yv){let r=e[n];const o=Yv[n];(\"transparent\"!==r||\"fill\"!==o&&\"stroke\"!==o)&&null!=r&&(Up(r)&&(r=Lp(r,i.gradient,\"\")),t[o]=r)}for(const t in Gv){const n=e[t];null!=n&&(o=o||[],o.push(`${Gv[t]}: ${n};`))}o&&(t.style=o.join(\" \"))}(i,e,t,r,this._defs),i},defs(t){const e=this._defs.gradient,n=this._defs.clipping;if(0!==Object.keys(e).length+Object.keys(n).length){t.open(\"defs\");for(const n in e){const r=e[n],i=r.stops;\"radial\"===r.gradient?(t.open(\"pattern\",{id:Rp+n,viewBox:\"0,0,1,1\",width:\"100%\",height:\"100%\",preserveAspectRatio:\"xMidYMid slice\"}),t.open(\"rect\",{width:\"1\",height:\"1\",fill:\"url(#\"+n+\")\"}).close(),t.close(),t.open(\"radialGradient\",{id:n,fx:r.x1,fy:r.y1,fr:r.r1,cx:r.x2,cy:r.y2,r:r.r2})):t.open(\"linearGradient\",{id:n,x1:r.x1,x2:r.x2,y1:r.y1,y2:r.y2});for(let e=0;e!h_.svgMarkTypes.includes(t)));this._svgRenderer.render(t,h_.svgMarkTypes),this._canvasRenderer.render(t,n)},resize(t,e,n,r){return p_.resize.call(this,t,e,n,r),this._svgRenderer.resize(t,e,n,r),this._canvasRenderer.resize(t,e,n,r),this},background(t){return h_.svgOnTop?this._canvasRenderer.background(t):this._svgRenderer.background(t),this}}),dt(g_,vv,{initialize(t,e,n){const r=Yy(Yy(t,0,\"div\"),h_.svgOnTop?0:1,\"div\");return vv.prototype.initialize.call(this,r,e,n)}});const m_=\"canvas\",y_=\"hybrid\",v_=\"none\",__={Canvas:m_,PNG:\"png\",SVG:\"svg\",Hybrid:y_,None:v_},x_={};function b_(t,e){return t=String(t||\"\").toLowerCase(),arguments.length>1?(x_[t]=e,this):x_[t]}function w_(t,e,n){const r=[],i=(new Rg).union(e),o=t.marktype;return o?k_(t,i,n,r):\"group\"===o?A_(t,i,n,r):s(\"Intersect scene must be mark node or group item.\")}function k_(t,e,n,r){if(function(t,e,n){return t.bounds&&e.intersects(t.bounds)&&(\"group\"===t.marktype||!1!==t.interactive&&(!n||n(t)))}(t,e,n)){const i=t.items,o=t.marktype,a=i.length;let s=0;if(\"group\"===o)for(;s=0;r--)if(i[r]!=o[r])return!1;for(r=i.length-1;r>=0;r--)if(!F_(t[n=i[r]],e[n],n))return!1;return typeof t==typeof e}(t,e):t==e)}function S_(t,e){return F_(Xp(t),Xp(e))}const $_=\"top\",T_=\"left\",B_=\"right\",z_=\"bottom\",N_=\"top-left\",O_=\"top-right\",R_=\"bottom-left\",U_=\"bottom-right\",L_=\"start\",q_=\"middle\",P_=\"end\",j_=\"x\",I_=\"y\",W_=\"group\",H_=\"axis\",Y_=\"title\",G_=\"frame\",V_=\"scope\",X_=\"legend\",J_=\"row-header\",Z_=\"row-footer\",Q_=\"row-title\",K_=\"column-header\",tx=\"column-footer\",ex=\"column-title\",nx=\"padding\",rx=\"symbol\",ix=\"fit\",ox=\"fit-x\",ax=\"fit-y\",sx=\"pad\",ux=\"none\",lx=\"all\",cx=\"each\",fx=\"flush\",hx=\"column\",dx=\"row\";function px(t){Ja.call(this,null,t)}function gx(t,e,n){return e(t.bounds.clear(),t,n)}dt(px,Ja,{transform(t,e){const n=e.dataflow,r=t.mark,i=r.marktype,o=zy[i],a=o.bound;let s,u=r.bounds;if(o.nested)r.items.length&&n.dirty(r.items[0]),u=gx(r,a),r.items.forEach((t=>{t.bounds.clear().union(u)}));else if(i===W_||t.modified())switch(e.visit(e.MOD,(t=>n.dirty(t))),u.clear(),r.items.forEach((t=>u.union(gx(t,a)))),r.role){case H_:case X_:case Y_:e.reflow()}else s=e.changed(e.REM),e.visit(e.ADD,(t=>{u.union(gx(t,a))})),e.visit(e.MOD,(t=>{s=s||u.alignsWith(t.bounds),n.dirty(t),u.union(gx(t,a))})),s&&(u.clear(),r.items.forEach((t=>u.union(t.bounds))));return D_(r),e.modifies(\"bounds\")}});const mx=\":vega_identifier:\";function yx(t){Ja.call(this,0,t)}function vx(t){Ja.call(this,null,t)}function _x(t){Ja.call(this,null,t)}yx.Definition={type:\"Identifier\",metadata:{modifies:!0},params:[{name:\"as\",type:\"string\",required:!0}]},dt(yx,Ja,{transform(t,e){const n=(i=e.dataflow)._signals[mx]||(i._signals[mx]=i.add(0)),r=t.as;var i;let o=n.value;return e.visit(e.ADD,(t=>t[r]=t[r]||++o)),n.set(this.value=o),e}}),dt(vx,Ja,{transform(t,e){let n=this.value;n||(n=e.dataflow.scenegraph().mark(t.markdef,function(t){const e=t.groups,n=t.parent;return e&&1===e.size?e.get(Object.keys(e.object)[0]):e&&n?e.lookup(n):null}(t),t.index),n.group.context=t.context,t.context.group||(t.context.group=n.group),n.source=this.source,n.clip=t.clip,n.interactive=t.interactive,this.value=n);const r=n.marktype===W_?Lg:Ug;return e.visit(e.ADD,(t=>r.call(t,n))),(t.modified(\"clip\")||t.modified(\"interactive\"))&&(n.clip=t.clip,n.interactive=!!t.interactive,n.zdirty=!0,e.reflow()),n.items=e.source,e}});const xx={parity:t=>t.filter(((t,e)=>e%2?t.opacity=0:1)),greedy:(t,e)=>{let n;return t.filter(((t,r)=>r&&bx(n.bounds,t.bounds,e)?t.opacity=0:(n=t,1)))}},bx=(t,e,n)=>n>Math.max(e.x1-t.x2,t.x1-e.x2,e.y1-t.y2,t.y1-e.y2),wx=(t,e)=>{for(var n,r=1,i=t.length,o=t[0].bounds;r{const e=t.bounds;return e.width()>1&&e.height()>1},Ax=t=>(t.forEach((t=>t.opacity=1)),t),Mx=(t,e)=>t.reflow(e.modified()).modifies(\"opacity\");function Ex(t){Ja.call(this,null,t)}dt(_x,Ja,{transform(t,e){const n=xx[t.method]||xx.parity,r=t.separation||0;let i,o,a=e.materialize(e.SOURCE).source;if(!a||!a.length)return;if(!t.method)return t.modified(\"method\")&&(Ax(a),e=Mx(e,t)),e;if(a=a.filter(kx),!a.length)return;if(t.sort&&(a=a.slice().sort(t.sort)),i=Ax(a),e=Mx(e,t),i.length>=3&&wx(i,r)){do{i=n(i,r)}while(i.length>=3&&wx(i,r));i.length<3&&!F(a).opacity&&(i.length>1&&(F(i).opacity=0),F(a).opacity=1)}t.boundScale&&t.boundTolerance>=0&&(o=((t,e,n)=>{var r=t.range(),i=new Rg;return e===$_||e===z_?i.set(r[0],-1/0,r[1],1/0):i.set(-1/0,r[0],1/0,r[1]),i.expand(n||1),t=>i.encloses(t.bounds)})(t.boundScale,t.boundOrient,+t.boundTolerance),a.forEach((t=>{o(t)||(t.opacity=0)})));const s=i[0].mark.bounds.clear();return a.forEach((t=>{t.opacity&&s.union(t.bounds)})),e}}),dt(Ex,Ja,{transform(t,e){const n=e.dataflow;if(e.visit(e.ALL,(t=>n.dirty(t))),e.fields&&e.fields.zindex){const t=e.source&&e.source[0];t&&(t.mark.zdirty=!0)}}});const Dx=new Rg;function Cx(t,e,n){return t[e]===n?0:(t[e]=n,1)}function Fx(t){var e=t.items[0].orient;return e===T_||e===B_}function Sx(t,e,n,r){var i,o,a=e.items[0],s=a.datum,u=null!=a.translate?a.translate:.5,l=a.orient,c=function(t){let e=+t.grid;return[t.ticks?e++:-1,t.labels?e++:-1,e+ +t.domain]}(s),f=a.range,h=a.offset,d=a.position,p=a.minExtent,g=a.maxExtent,m=s.title&&a.items[c[2]].items[0],y=a.titlePadding,v=a.bounds,_=m&&ky(m),x=0,b=0;switch(Dx.clear().union(v),v.clear(),(i=c[0])>-1&&v.union(a.items[i].bounds),(i=c[1])>-1&&v.union(a.items[i].bounds),l){case $_:x=d||0,b=-h,o=Math.max(p,Math.min(g,-v.y1)),v.add(0,-o).add(f,0),m&&$x(t,m,o,y,_,0,-1,v);break;case T_:x=-h,b=d||0,o=Math.max(p,Math.min(g,-v.x1)),v.add(-o,0).add(0,f),m&&$x(t,m,o,y,_,1,-1,v);break;case B_:x=n+h,b=d||0,o=Math.max(p,Math.min(g,v.x2)),v.add(0,0).add(o,f),m&&$x(t,m,o,y,_,1,1,v);break;case z_:x=d||0,b=r+h,o=Math.max(p,Math.min(g,v.y2)),v.add(0,0).add(f,o),m&&$x(t,m,o,y,0,0,1,v);break;default:x=a.x,b=a.y}return Ig(v.translate(x,b),a),Cx(a,\"x\",x+u)|Cx(a,\"y\",b+u)&&(a.bounds=Dx,t.dirty(a),a.bounds=v,t.dirty(a)),a.mark.bounds.clear().union(v)}function $x(t,e,n,r,i,o,a,s){const u=e.bounds;if(e.auto){const s=a*(n+i+r);let l=0,c=0;t.dirty(e),o?l=(e.x||0)-(e.x=s):c=(e.y||0)-(e.y=s),e.mark.bounds.clear().union(u.translate(-l,-c)),t.dirty(e)}s.union(u)}const Tx=(t,e)=>Math.floor(Math.min(t,e)),Bx=(t,e)=>Math.ceil(Math.max(t,e));function zx(t){return(new Rg).set(0,0,t.width||0,t.height||0)}function Nx(t){const e=t.bounds.clone();return e.empty()?e.set(0,0,0,0):e.translate(-(t.x||0),-(t.y||0))}function Ox(t,e,n){const r=A(t)?t[e]:t;return null!=r?r:void 0!==n?n:0}function Rx(t){return t<0?Math.ceil(-t):0}function Ux(t,e,n){var r,i,o,a,s,u,l,c,f,h,d,p=!n.nodirty,g=n.bounds===fx?zx:Nx,m=Dx.set(0,0,0,0),y=Ox(n.align,hx),v=Ox(n.align,dx),_=Ox(n.padding,hx),x=Ox(n.padding,dx),b=n.columns||e.length,w=b<=0?1:Math.ceil(e.length/b),k=e.length,A=Array(k),M=Array(b),E=0,D=Array(k),C=Array(w),F=0,S=Array(k),$=Array(k),T=Array(k);for(i=0;i1)for(i=0;i0&&(S[i]+=f/2);if(v&&Ox(n.center,dx)&&1!==b)for(i=0;i0&&($[i]+=h/2);for(i=0;ii&&(t.warn(\"Grid headers exceed limit: \"+i),e=e.slice(0,i)),A+=o,g=0,y=e.length;g=0&&null==(x=n[m]);m-=h);s?(b=null==d?x.x:Math.round(x.bounds.x1+d*x.bounds.width()),w=A):(b=A,w=null==d?x.y:Math.round(x.bounds.y1+d*x.bounds.height())),v.union(_.bounds.translate(b-(_.x||0),w-(_.y||0))),_.x=b,_.y=w,t.dirty(_),M=a(M,v[l])}return M}function Ix(t,e,n,r,i,o){if(e){t.dirty(e);var a=n,s=n;r?a=Math.round(i.x1+o*i.width()):s=Math.round(i.y1+o*i.height()),e.bounds.translate(a-(e.x||0),s-(e.y||0)),e.mark.bounds.clear().union(e.bounds),e.x=a,e.y=s,t.dirty(e)}}function Wx(t,e,n,r,i,o,a){const s=function(t,e){const n=t[e]||{};return(e,r)=>null!=n[e]?n[e]:null!=t[e]?t[e]:r}(n,e),u=function(t,e){let n=-1/0;return t.forEach((t=>{null!=t.offset&&(n=Math.max(n,t.offset))})),n>-1/0?n:e}(t,s(\"offset\",0)),l=s(\"anchor\",L_),c=l===P_?1:l===q_?.5:0,f={align:cx,bounds:s(\"bounds\",fx),columns:\"vertical\"===s(\"direction\")?1:t.length,padding:s(\"margin\",8),center:s(\"center\"),nodirty:!0};switch(e){case T_:f.anchor={x:Math.floor(r.x1)-u,column:P_,y:c*(a||r.height()+2*r.y1),row:l};break;case B_:f.anchor={x:Math.ceil(r.x2)+u,y:c*(a||r.height()+2*r.y1),row:l};break;case $_:f.anchor={y:Math.floor(i.y1)-u,row:P_,x:c*(o||i.width()+2*i.x1),column:l};break;case z_:f.anchor={y:Math.ceil(i.y2)+u,x:c*(o||i.width()+2*i.x1),column:l};break;case N_:f.anchor={x:u,y:u};break;case O_:f.anchor={x:o-u,y:u,column:P_};break;case R_:f.anchor={x:u,y:a-u,row:P_};break;case U_:f.anchor={x:o-u,y:a-u,column:P_,row:P_}}return f}function Hx(t,e){var n,r,i=e.items[0],o=i.datum,a=i.orient,s=i.bounds,u=i.x,l=i.y;return i._bounds?i._bounds.clear().union(s):i._bounds=s.clone(),s.clear(),function(t,e,n){var r=e.padding,i=r-n.x,o=r-n.y;if(e.datum.title){var a=e.items[1].items[0],s=a.anchor,u=e.titlePadding||0,l=r-a.x,c=r-a.y;switch(a.orient){case T_:i+=Math.ceil(a.bounds.width())+u;break;case B_:case z_:break;default:o+=a.bounds.height()+u}switch((i||o)&&Gx(t,n,i,o),a.orient){case T_:c+=Yx(e,n,a,s,1,1);break;case B_:l+=Yx(e,n,a,P_,0,0)+u,c+=Yx(e,n,a,s,1,1);break;case z_:l+=Yx(e,n,a,s,0,0),c+=Yx(e,n,a,P_,-1,0,1)+u;break;default:l+=Yx(e,n,a,s,0,0)}(l||c)&&Gx(t,a,l,c),(l=Math.round(a.bounds.x1-r))<0&&(Gx(t,n,-l,0),Gx(t,a,-l,0))}else(i||o)&&Gx(t,n,i,o)}(t,i,i.items[0].items[0]),s=function(t,e){return t.items.forEach((t=>e.union(t.bounds))),e.x1=t.padding,e.y1=t.padding,e}(i,s),n=2*i.padding,r=2*i.padding,s.empty()||(n=Math.ceil(s.width()+n),r=Math.ceil(s.height()+r)),o.type===rx&&function(t){const e=t.reduce(((t,e)=>(t[e.column]=Math.max(e.bounds.x2-e.x,t[e.column]||0),t)),{});t.forEach((t=>{t.width=e[t.column],t.height=t.bounds.y2-t.y}))}(i.items[0].items[0].items[0].items),a!==ux&&(i.x=u=0,i.y=l=0),i.width=n,i.height=r,Ig(s.set(u,l,u+n,l+r),i),i.mark.bounds.clear().union(s),i}function Yx(t,e,n,r,i,o,a){const s=\"symbol\"!==t.datum.type,u=n.datum.vgrad,l=(!s||!o&&u||a?e:e.items[0]).bounds[i?\"y2\":\"x2\"]-t.padding,c=u&&o?l:0,f=u&&o?0:l,h=i<=0?0:ky(n);return Math.round(r===L_?c:r===P_?f-h:.5*(l-h))}function Gx(t,e,n,r){e.x+=n,e.y+=r,e.bounds.translate(n,r),e.mark.bounds.translate(n,r),t.dirty(e)}function Vx(t){Ja.call(this,null,t)}dt(Vx,Ja,{transform(t,e){const n=e.dataflow;return t.mark.items.forEach((e=>{t.layout&&Lx(n,e,t.layout),function(t,e,n){var r,i,o,a,s,u=e.items,l=Math.max(0,e.width||0),c=Math.max(0,e.height||0),f=(new Rg).set(0,0,l,c),h=f.clone(),d=f.clone(),p=[];for(a=0,s=u.length;a{(o=t.orient||B_)!==ux&&(e[o]||(e[o]=[])).push(t)}));for(const r in e){const i=e[r];Ux(t,i,Wx(i,r,n.legends,h,d,l,c))}p.forEach((e=>{const r=e.bounds;if(r.equals(e._bounds)||(e.bounds=e._bounds,t.dirty(e),e.bounds=r,t.dirty(e)),!n.autosize||n.autosize.type!==ix&&n.autosize.type!==ox&&n.autosize.type!==ax)f.union(r);else switch(e.orient){case T_:case B_:f.add(r.x1,0).add(r.x2,0);break;case $_:case z_:f.add(0,r.y1).add(0,r.y2)}}))}f.union(h).union(d),r&&f.union(function(t,e,n,r,i){var o,a=e.items[0],s=a.frame,u=a.orient,l=a.anchor,c=a.offset,f=a.padding,h=a.items[0].items[0],d=a.items[1]&&a.items[1].items[0],p=u===T_||u===B_?r:n,g=0,m=0,y=0,v=0,_=0;if(s!==W_?u===T_?(g=i.y2,p=i.y1):u===B_?(g=i.y1,p=i.y2):(g=i.x1,p=i.x2):u===T_&&(g=r,p=0),o=l===L_?g:l===P_?p:(g+p)/2,d&&d.text){switch(u){case $_:case z_:_=h.bounds.height()+f;break;case T_:v=h.bounds.width()+f;break;case B_:v=-h.bounds.width()-f}Dx.clear().union(d.bounds),Dx.translate(v-(d.x||0),_-(d.y||0)),Cx(d,\"x\",v)|Cx(d,\"y\",_)&&(t.dirty(d),d.bounds.clear().union(Dx),d.mark.bounds.clear().union(Dx),t.dirty(d)),Dx.clear().union(d.bounds)}else Dx.clear();switch(Dx.union(h.bounds),u){case $_:m=o,y=i.y1-Dx.height()-c;break;case T_:m=i.x1-Dx.width()-c,y=o;break;case B_:m=i.x2+Dx.width()+c,y=o;break;case z_:m=o,y=i.y2+c;break;default:m=a.x,y=a.y}return Cx(a,\"x\",m)|Cx(a,\"y\",y)&&(Dx.translate(m,y),t.dirty(a),a.bounds.clear().union(Dx),e.bounds.clear().union(Dx),t.dirty(a)),a.bounds}(t,r,l,c,f));e.clip&&f.set(0,0,e.width||0,e.height||0);!function(t,e,n,r){const i=r.autosize||{},o=i.type;if(t._autosize<1||!o)return;let a=t._width,s=t._height,u=Math.max(0,e.width||0),l=Math.max(0,Math.ceil(-n.x1)),c=Math.max(0,e.height||0),f=Math.max(0,Math.ceil(-n.y1));const h=Math.max(0,Math.ceil(n.x2-u)),d=Math.max(0,Math.ceil(n.y2-c));if(i.contains===nx){const e=t.padding();a-=e.left+e.right,s-=e.top+e.bottom}o===ux?(l=0,f=0,u=a,c=s):o===ix?(u=Math.max(0,a-l-h),c=Math.max(0,s-f-d)):o===ox?(u=Math.max(0,a-l-h),s=c+f+d):o===ax?(a=u+l+h,c=Math.max(0,s-f-d)):o===sx&&(a=u+l+h,s=c+f+d);t._resizeView(a,s,u,c,[l,f],i.resize)}(t,e,f,n)}(n,e,t)})),function(t){return t&&\"legend-entry\"!==t.mark.role}(t.mark.group)?e.reflow():e}});var Xx=Object.freeze({__proto__:null,bound:px,identifier:yx,mark:vx,overlap:_x,render:Ex,viewlayout:Vx});function Jx(t){Ja.call(this,null,t)}function Zx(t){Ja.call(this,null,t)}function Qx(){return _a({})}function Kx(t){Ja.call(this,null,t)}function tb(t){Ja.call(this,[],t)}dt(Jx,Ja,{transform(t,e){if(this.value&&!t.modified())return e.StopPropagation;var n=e.dataflow.locale(),r=e.fork(e.NO_SOURCE|e.NO_FIELDS),i=this.value,o=t.scale,a=_p(o,null==t.count?t.values?t.values.length:10:t.count,t.minstep),s=t.format||wp(n,o,a,t.formatSpecifier,t.formatType,!!t.values),u=t.values?xp(o,t.values,a):bp(o,a);return i&&(r.rem=i),i=u.map(((t,e)=>_a({index:e/(u.length-1||1),value:t,label:s(t)}))),t.extra&&i.length&&i.push(_a({index:-1,extra:{value:i[0].value},label:\"\"})),r.source=i,r.add=i,this.value=i,r}}),dt(Zx,Ja,{transform(t,e){var n=e.dataflow,r=e.fork(e.NO_SOURCE|e.NO_FIELDS),i=t.item||Qx,o=t.key||ya,a=this.value;return k(r.encode)&&(r.encode=null),a&&(t.modified(\"key\")||e.modified(o))&&s(\"DataJoin does not support modified key function or fields.\"),a||(e=e.addAll(),this.value=a=function(t){const e=ft().test((t=>t.exit));return e.lookup=n=>e.get(t(n)),e}(o)),e.visit(e.ADD,(t=>{const e=o(t);let n=a.get(e);n?n.exit?(a.empty--,r.add.push(n)):r.mod.push(n):(n=i(t),a.set(e,n),r.add.push(n)),n.datum=t,n.exit=!1})),e.visit(e.MOD,(t=>{const e=o(t),n=a.get(e);n&&(n.datum=t,r.mod.push(n))})),e.visit(e.REM,(t=>{const e=o(t),n=a.get(e);t!==n.datum||n.exit||(r.rem.push(n),n.exit=!0,++a.empty)})),e.changed(e.ADD_MOD)&&r.modifies(\"datum\"),(e.clean()||t.clean&&a.empty>n.cleanThreshold)&&n.runAfter(a.clean),r}}),dt(Kx,Ja,{transform(t,e){var n=e.fork(e.ADD_REM),r=t.mod||!1,i=t.encoders,o=e.encode;if(k(o)){if(!n.changed()&&!o.every((t=>i[t])))return e.StopPropagation;o=o[0],n.encode=null}var a=\"enter\"===o,s=i.update||g,u=i.enter||g,l=i.exit||g,c=(o&&!a?i[o]:s)||g;if(e.changed(e.ADD)&&(e.visit(e.ADD,(e=>{u(e,t),s(e,t)})),n.modifies(u.output),n.modifies(s.output),c!==g&&c!==s&&(e.visit(e.ADD,(e=>{c(e,t)})),n.modifies(c.output))),e.changed(e.REM)&&l!==g&&(e.visit(e.REM,(e=>{l(e,t)})),n.modifies(l.output)),a||c!==g){const i=e.MOD|(t.modified()?e.REFLOW:0);a?(e.visit(i,(e=>{const i=u(e,t)||r;(c(e,t)||i)&&n.mod.push(e)})),n.mod.length&&n.modifies(u.output)):e.visit(i,(e=>{(c(e,t)||r)&&n.mod.push(e)})),n.mod.length&&n.modifies(c.output)}return n.changed()?n:e.StopPropagation}}),dt(tb,Ja,{transform(t,e){if(null!=this.value&&!t.modified())return e.StopPropagation;var n,r,i,o,a,s=e.dataflow.locale(),u=e.fork(e.NO_SOURCE|e.NO_FIELDS),l=this.value,c=t.type||pp,f=t.scale,h=+t.limit,d=_p(f,null==t.count?5:t.count,t.minstep),p=!!t.values||c===pp,g=t.format||Cp(s,f,d,c,t.formatSpecifier,t.formatType,p),m=t.values||Ep(f,d);return l&&(u.rem=l),c===pp?(h&&m.length>h?(e.dataflow.warn(\"Symbol legend count exceeds limit, filtering items.\"),l=m.slice(0,h-1),a=!0):l=m,J(i=t.size)?(t.values||0!==f(l[0])||(l=l.slice(1)),o=l.reduce(((e,n)=>Math.max(e,i(n,t))),0)):i=rt(o=i||8),l=l.map(((e,n)=>_a({index:n,label:g(e,n,l),value:e,offset:o,size:i(e,t)}))),a&&(a=m[l.length],l.push(_a({index:l.length,label:`…${m.length-l.length} entries`,value:a,offset:o,size:i(a,t)})))):\"gradient\"===c?(n=f.domain(),r=up(f,n[0],F(n)),m.length<3&&!t.values&&n[0]!==F(n)&&(m=[n[0],F(n)]),l=m.map(((t,e)=>_a({index:e,label:g(t,e,m),value:t,perc:r(t)})))):(i=m.length-1,r=function(t){const e=t.domain(),n=e.length-1;let r=+e[0],i=+F(e),o=i-r;if(t.type===Td){const t=n?o/n:.1;r-=t,i+=t,o=i-r}return t=>(t-r)/o}(f),l=m.map(((t,e)=>_a({index:e,label:g(t,e,m),value:t,perc:e?r(t):0,perc2:e===i?1:r(m[e+1])})))),u.source=l,u.add=l,this.value=l,u}});const eb=t=>t.source.x,nb=t=>t.source.y,rb=t=>t.target.x,ib=t=>t.target.y;function ob(t){Ja.call(this,{},t)}ob.Definition={type:\"LinkPath\",metadata:{modifies:!0},params:[{name:\"sourceX\",type:\"field\",default:\"source.x\"},{name:\"sourceY\",type:\"field\",default:\"source.y\"},{name:\"targetX\",type:\"field\",default:\"target.x\"},{name:\"targetY\",type:\"field\",default:\"target.y\"},{name:\"orient\",type:\"enum\",default:\"vertical\",values:[\"horizontal\",\"vertical\",\"radial\"]},{name:\"shape\",type:\"enum\",default:\"line\",values:[\"line\",\"arc\",\"curve\",\"diagonal\",\"orthogonal\"]},{name:\"require\",type:\"signal\"},{name:\"as\",type:\"string\",default:\"path\"}]},dt(ob,Ja,{transform(t,e){var n=t.sourceX||eb,r=t.sourceY||nb,i=t.targetX||rb,o=t.targetY||ib,a=t.as||\"path\",u=t.orient||\"vertical\",l=t.shape||\"line\",c=lb.get(l+\"-\"+u)||lb.get(l);return c||s(\"LinkPath unsupported type: \"+t.shape+(t.orient?\"-\"+t.orient:\"\")),e.visit(e.SOURCE,(t=>{t[a]=c(n(t),r(t),i(t),o(t))})),e.reflow(t.modified()).modifies(a)}});const ab=(t,e,n,r)=>\"M\"+t+\",\"+e+\"L\"+n+\",\"+r,sb=(t,e,n,r)=>{var i=n-t,o=r-e,a=Math.hypot(i,o)/2;return\"M\"+t+\",\"+e+\"A\"+a+\",\"+a+\" \"+180*Math.atan2(o,i)/Math.PI+\" 0 1 \"+n+\",\"+r},ub=(t,e,n,r)=>{const i=n-t,o=r-e,a=.2*(i+o),s=.2*(o-i);return\"M\"+t+\",\"+e+\"C\"+(t+a)+\",\"+(e+s)+\" \"+(n+s)+\",\"+(r-a)+\" \"+n+\",\"+r},lb=ft({line:ab,\"line-radial\":(t,e,n,r)=>ab(e*Math.cos(t),e*Math.sin(t),r*Math.cos(n),r*Math.sin(n)),arc:sb,\"arc-radial\":(t,e,n,r)=>sb(e*Math.cos(t),e*Math.sin(t),r*Math.cos(n),r*Math.sin(n)),curve:ub,\"curve-radial\":(t,e,n,r)=>ub(e*Math.cos(t),e*Math.sin(t),r*Math.cos(n),r*Math.sin(n)),\"orthogonal-horizontal\":(t,e,n,r)=>\"M\"+t+\",\"+e+\"V\"+r+\"H\"+n,\"orthogonal-vertical\":(t,e,n,r)=>\"M\"+t+\",\"+e+\"H\"+n+\"V\"+r,\"orthogonal-radial\":(t,e,n,r)=>{const i=Math.cos(t),o=Math.sin(t),a=Math.cos(n),s=Math.sin(n);return\"M\"+e*i+\",\"+e*o+\"A\"+e+\",\"+e+\" 0 0,\"+((Math.abs(n-t)>Math.PI?n<=t:n>t)?1:0)+\" \"+e*a+\",\"+e*s+\"L\"+r*a+\",\"+r*s},\"diagonal-horizontal\":(t,e,n,r)=>{const i=(t+n)/2;return\"M\"+t+\",\"+e+\"C\"+i+\",\"+e+\" \"+i+\",\"+r+\" \"+n+\",\"+r},\"diagonal-vertical\":(t,e,n,r)=>{const i=(e+r)/2;return\"M\"+t+\",\"+e+\"C\"+t+\",\"+i+\" \"+n+\",\"+i+\" \"+n+\",\"+r},\"diagonal-radial\":(t,e,n,r)=>{const i=Math.cos(t),o=Math.sin(t),a=Math.cos(n),s=Math.sin(n),u=(e+r)/2;return\"M\"+e*i+\",\"+e*o+\"C\"+u*i+\",\"+u*o+\" \"+u*a+\",\"+u*s+\" \"+r*a+\",\"+r*s}});function cb(t){Ja.call(this,null,t)}cb.Definition={type:\"Pie\",metadata:{modifies:!0},params:[{name:\"field\",type:\"field\"},{name:\"startAngle\",type:\"number\",default:0},{name:\"endAngle\",type:\"number\",default:6.283185307179586},{name:\"sort\",type:\"boolean\",default:!1},{name:\"as\",type:\"string\",array:!0,length:2,default:[\"startAngle\",\"endAngle\"]}]},dt(cb,Ja,{transform(t,e){var n,r,i,o=t.as||[\"startAngle\",\"endAngle\"],a=o[0],s=o[1],u=t.field||d,l=t.startAngle||0,c=null!=t.endAngle?t.endAngle:2*Math.PI,f=e.source,h=f.map(u),p=h.length,g=l,m=(c-l)/$e(h),y=Se(p);for(t.sort&&y.sort(((t,e)=>h[t]-h[e])),n=0;nt+(e<0?-1:e>0?1:0)),0))!==e.length&&n.warn(\"Log scale domain includes zero: \"+Ct(e)));return e}function mb(t,e,n){return J(t)&&(e||n)?op(t,yb(e||[0,1],n)):t}function yb(t,e){return e?t.slice().reverse():t}function vb(t){Ja.call(this,null,t)}dt(pb,Ja,{transform(t,e){var n=e.dataflow,r=this.value,i=function(t){var e,n=t.type,r=\"\";if(n===Cd)return Cd+\"-\"+bd;(function(t){const e=t.type;return Qd(e)&&e!==Ed&&e!==Dd&&(t.scheme||t.range&&t.range.length&&t.range.every(xt))})(t)&&(r=2===(e=t.rawDomain?t.rawDomain.length:t.domain?t.domain.length+ +(null!=t.domainMid):0)?Cd+\"-\":3===e?Fd+\"-\":\"\");return(r+n||bd).toLowerCase()}(t);for(i in r&&i===r.type||(this.value=r=Xd(i)()),t)if(!db[i]){if(\"padding\"===i&&hb(r.type))continue;J(r[i])?r[i](t[i]):n.warn(\"Unsupported scale property: \"+i)}return function(t,e,n){var r=t.type,i=e.round||!1,o=e.range;if(null!=e.rangeStep)o=function(t,e,n){t!==Nd&&t!==zd&&s(\"Only band and point scales support rangeStep.\");var r=(null!=e.paddingOuter?e.paddingOuter:e.padding)||0,i=t===zd?1:(null!=e.paddingInner?e.paddingInner:e.padding)||0;return[0,e.rangeStep*xd(n,i,r)]}(r,e,n);else if(e.scheme&&(o=function(t,e,n){var r,i=e.schemeExtent;k(e.scheme)?r=ap(e.scheme,e.interpolate,e.interpolateGamma):(r=dp(e.scheme.toLowerCase()))||s(`Unrecognized scheme name: ${e.scheme}`);return n=t===Td?n+1:t===Od?n-1:t===Sd||t===$d?+e.schemeCount||fb:n,np(t)?mb(r,i,e.reverse):J(r)?sp(mb(r,i),n):t===Bd?r:r.slice(0,n)}(r,e,n),J(o))){if(t.interpolator)return t.interpolator(o);s(`Scale type ${r} does not support interpolating color schemes.`)}if(o&&np(r))return t.interpolator(ap(yb(o,e.reverse),e.interpolate,e.interpolateGamma));o&&e.interpolate&&t.interpolate?t.interpolate(lp(e.interpolate,e.interpolateGamma)):J(t.round)?t.round(i):J(t.rangeRound)&&t.interpolate(i?yh:mh);o&&t.range(yb(o,e.reverse))}(r,t,function(t,e,n){let r=e.bins;if(r&&!k(r)){const e=t.domain(),n=e[0],i=F(e),o=r.step;let a=null==r.start?n:r.start,u=null==r.stop?i:r.stop;o||s(\"Scale bins parameter missing step property.\"),ai&&(u=o*Math.floor(i/o)),r=Se(a,u+o/2,o)}r?t.bins=r:t.bins&&delete t.bins;t.type===Od&&(r?e.domain||e.domainRaw||(t.domain(r),n=r.length):t.bins=t.domain());return n}(r,t,function(t,e,n){const r=function(t,e,n){return e?(t.domain(gb(t.type,e,n)),e.length):-1}(t,e.domainRaw,n);if(r>-1)return r;var i,o,a=e.domain,s=t.type,u=e.zero||void 0===e.zero&&function(t){const e=t.type;return!t.bins&&(e===bd||e===kd||e===Ad)}(t);if(!a)return 0;hb(s)&&e.padding&&a[0]!==F(a)&&(a=function(t,e,n,r,i,o){var a=Math.abs(F(n)-n[0]),s=a/(a-2*r),u=t===wd?I(e,null,s):t===Ad?W(e,null,s,.5):t===kd?W(e,null,s,i||1):t===Md?H(e,null,s,o||1):j(e,null,s);return e=e.slice(),e[0]=u[0],e[e.length-1]=u[1],e}(s,a,e.range,e.padding,e.exponent,e.constant));if((u||null!=e.domainMin||null!=e.domainMax||null!=e.domainMid)&&(i=(a=a.slice()).length-1||1,u&&(a[0]>0&&(a[0]=0),a[i]<0&&(a[i]=0)),null!=e.domainMin&&(a[0]=e.domainMin),null!=e.domainMax&&(a[i]=e.domainMax),null!=e.domainMid)){const t=(o=e.domainMid)>a[i]?i+1:ot(u);if(null==e)d.push(t.slice());else for(i={},o=0,a=t.length;oh&&(h=f),n&&c.sort(n)}return d.max=h,d}(e.source,t.groupby,l,c),r=0,i=n.length,o=n.max;r0?1:t<0?-1:0},Gb=Math.sqrt,Vb=Math.tan;function Xb(t){return t>1?0:t<-1?Sb:Math.acos(t)}function Jb(t){return t>1?$b:t<-1?-$b:Math.asin(t)}function Zb(){}function Qb(t,e){t&&tw.hasOwnProperty(t.type)&&tw[t.type](t,e)}var Kb={Feature:function(t,e){Qb(t.geometry,e)},FeatureCollection:function(t,e){for(var n=t.features,r=-1,i=n.length;++r=0?1:-1,i=r*n,o=Lb(e=(e*=Nb)/2+Tb),a=Hb(e),s=uw*a,u=sw*o+s*Lb(i),l=s*r*Hb(i);xw.add(Ub(l,u)),aw=t,sw=o,uw=a}function Dw(t){return[Ub(t[1],t[0]),Jb(t[2])]}function Cw(t){var e=t[0],n=t[1],r=Lb(n);return[r*Lb(e),r*Hb(e),Hb(n)]}function Fw(t,e){return t[0]*e[0]+t[1]*e[1]+t[2]*e[2]}function Sw(t,e){return[t[1]*e[2]-t[2]*e[1],t[2]*e[0]-t[0]*e[2],t[0]*e[1]-t[1]*e[0]]}function $w(t,e){t[0]+=e[0],t[1]+=e[1],t[2]+=e[2]}function Tw(t,e){return[t[0]*e,t[1]*e,t[2]*e]}function Bw(t){var e=Gb(t[0]*t[0]+t[1]*t[1]+t[2]*t[2]);t[0]/=e,t[1]/=e,t[2]/=e}var zw,Nw,Ow,Rw,Uw,Lw,qw,Pw,jw,Iw,Ww,Hw,Yw,Gw,Vw,Xw,Jw={point:Zw,lineStart:Kw,lineEnd:tk,polygonStart:function(){Jw.point=ek,Jw.lineStart=nk,Jw.lineEnd=rk,yw=new se,ww.polygonStart()},polygonEnd:function(){ww.polygonEnd(),Jw.point=Zw,Jw.lineStart=Kw,Jw.lineEnd=tk,xw<0?(lw=-(fw=180),cw=-(hw=90)):yw>Cb?hw=90:yw<-Cb&&(cw=-90),_w[0]=lw,_w[1]=fw},sphere:function(){lw=-(fw=180),cw=-(hw=90)}};function Zw(t,e){vw.push(_w=[lw=t,fw=t]),ehw&&(hw=e)}function Qw(t,e){var n=Cw([t*Nb,e*Nb]);if(mw){var r=Sw(mw,n),i=Sw([r[1],-r[0],0],r);Bw(i),i=Dw(i);var o,a=t-dw,s=a>0?1:-1,u=i[0]*zb*s,l=Ob(a)>180;l^(s*dwhw&&(hw=o):l^(s*dw<(u=(u+360)%360-180)&&uhw&&(hw=e)),l?tik(lw,fw)&&(fw=t):ik(t,fw)>ik(lw,fw)&&(lw=t):fw>=lw?(tfw&&(fw=t)):t>dw?ik(lw,t)>ik(lw,fw)&&(fw=t):ik(t,fw)>ik(lw,fw)&&(lw=t)}else vw.push(_w=[lw=t,fw=t]);ehw&&(hw=e),mw=n,dw=t}function Kw(){Jw.point=Qw}function tk(){_w[0]=lw,_w[1]=fw,Jw.point=Zw,mw=null}function ek(t,e){if(mw){var n=t-dw;yw.add(Ob(n)>180?n+(n>0?360:-360):n)}else pw=t,gw=e;ww.point(t,e),Qw(t,e)}function nk(){ww.lineStart()}function rk(){ek(pw,gw),ww.lineEnd(),Ob(yw)>Cb&&(lw=-(fw=180)),_w[0]=lw,_w[1]=fw,mw=null}function ik(t,e){return(e-=t)<0?e+360:e}function ok(t,e){return t[0]-e[0]}function ak(t,e){return t[0]<=t[1]?t[0]<=e&&e<=t[1]:eSb&&(t-=Math.round(t/Bb)*Bb),[t,e]}function xk(t,e,n){return(t%=Bb)?e||n?vk(wk(t),kk(e,n)):wk(t):e||n?kk(e,n):_k}function bk(t){return function(e,n){return Ob(e+=t)>Sb&&(e-=Math.round(e/Bb)*Bb),[e,n]}}function wk(t){var e=bk(t);return e.invert=bk(-t),e}function kk(t,e){var n=Lb(t),r=Hb(t),i=Lb(e),o=Hb(e);function a(t,e){var a=Lb(e),s=Lb(t)*a,u=Hb(t)*a,l=Hb(e),c=l*n+s*r;return[Ub(u*i-c*o,s*n-l*r),Jb(c*i+u*o)]}return a.invert=function(t,e){var a=Lb(e),s=Lb(t)*a,u=Hb(t)*a,l=Hb(e),c=l*i-u*o;return[Ub(u*i+l*o,s*n+c*r),Jb(c*n-s*r)]},a}function Ak(t,e){(e=Cw(e))[0]-=t,Bw(e);var n=Xb(-e[1]);return((-e[2]<0?-n:n)+Bb-Cb)%Bb}function Mk(){var t,e=[];return{point:function(e,n,r){t.push([e,n,r])},lineStart:function(){e.push(t=[])},lineEnd:Zb,rejoin:function(){e.length>1&&e.push(e.pop().concat(e.shift()))},result:function(){var n=e;return e=[],t=null,n}}}function Ek(t,e){return Ob(t[0]-e[0])=0;--o)i.point((c=l[o])[0],c[1]);else r(h.x,h.p.x,-1,i);h=h.p}l=(h=h.o).z,d=!d}while(!h.v);i.lineEnd()}}}function Fk(t){if(e=t.length){for(var e,n,r=0,i=t[0];++r=0?1:-1,E=M*A,D=E>Sb,C=m*w;if(u.add(Ub(C*M*Hb(E),y*k+C*Lb(E))),a+=D?A+M*Bb:A,D^p>=n^x>=n){var F=Sw(Cw(d),Cw(_));Bw(F);var S=Sw(o,F);Bw(S);var $=(D^A>=0?-1:1)*Jb(S[2]);(r>$||r===$&&(F[0]||F[1]))&&(s+=D^A>=0?1:-1)}}return(a<-Cb||a0){for(f||(i.polygonStart(),f=!0),i.lineStart(),t=0;t1&&2&u&&h.push(h.pop().concat(h.shift())),a.push(h.filter(Tk))}return h}}function Tk(t){return t.length>1}function Bk(t,e){return((t=t.x)[0]<0?t[1]-$b-Cb:$b-t[1])-((e=e.x)[0]<0?e[1]-$b-Cb:$b-e[1])}_k.invert=_k;var zk=$k((function(){return!0}),(function(t){var e,n=NaN,r=NaN,i=NaN;return{lineStart:function(){t.lineStart(),e=1},point:function(o,a){var s=o>0?Sb:-Sb,u=Ob(o-n);Ob(u-Sb)0?$b:-$b),t.point(i,r),t.lineEnd(),t.lineStart(),t.point(s,r),t.point(o,r),e=0):i!==s&&u>=Sb&&(Ob(n-i)Cb?Rb((Hb(e)*(o=Lb(r))*Hb(n)-Hb(r)*(i=Lb(e))*Hb(t))/(i*o*a)):(e+r)/2}(n,r,o,a),t.point(i,r),t.lineEnd(),t.lineStart(),t.point(s,r),e=0),t.point(n=o,r=a),i=s},lineEnd:function(){t.lineEnd(),n=r=NaN},clean:function(){return 2-e}}}),(function(t,e,n,r){var i;if(null==t)i=n*$b,r.point(-Sb,i),r.point(0,i),r.point(Sb,i),r.point(Sb,0),r.point(Sb,-i),r.point(0,-i),r.point(-Sb,-i),r.point(-Sb,0),r.point(-Sb,i);else if(Ob(t[0]-e[0])>Cb){var o=t[0]0,i=Ob(e)>Cb;function o(t,n){return Lb(t)*Lb(n)>e}function a(t,n,r){var i=[1,0,0],o=Sw(Cw(t),Cw(n)),a=Fw(o,o),s=o[0],u=a-s*s;if(!u)return!r&&t;var l=e*a/u,c=-e*s/u,f=Sw(i,o),h=Tw(i,l);$w(h,Tw(o,c));var d=f,p=Fw(h,d),g=Fw(d,d),m=p*p-g*(Fw(h,h)-1);if(!(m<0)){var y=Gb(m),v=Tw(d,(-p-y)/g);if($w(v,h),v=Dw(v),!r)return v;var _,x=t[0],b=n[0],w=t[1],k=n[1];b0^v[1]<(Ob(v[0]-x)Sb^(x<=v[0]&&v[0]<=b)){var E=Tw(d,(-p+y)/g);return $w(E,h),[v,Dw(E)]}}}function s(e,n){var i=r?t:Sb-t,o=0;return e<-i?o|=1:e>i&&(o|=2),n<-i?o|=4:n>i&&(o|=8),o}return $k(o,(function(t){var e,n,u,l,c;return{lineStart:function(){l=u=!1,c=1},point:function(f,h){var d,p=[f,h],g=o(f,h),m=r?g?0:s(f,h):g?s(f+(f<0?Sb:-Sb),h):0;if(!e&&(l=u=g)&&t.lineStart(),g!==u&&(!(d=a(e,p))||Ek(e,d)||Ek(p,d))&&(p[2]=1),g!==u)c=0,g?(t.lineStart(),d=a(p,e),t.point(d[0],d[1])):(d=a(e,p),t.point(d[0],d[1],2),t.lineEnd()),e=d;else if(i&&e&&r^g){var y;m&n||!(y=a(p,e,!0))||(c=0,r?(t.lineStart(),t.point(y[0][0],y[0][1]),t.point(y[1][0],y[1][1]),t.lineEnd()):(t.point(y[1][0],y[1][1]),t.lineEnd(),t.lineStart(),t.point(y[0][0],y[0][1],3)))}!g||e&&Ek(e,p)||t.point(p[0],p[1]),e=p,u=g,n=m},lineEnd:function(){u&&t.lineEnd(),e=null},clean:function(){return c|(l&&u)<<1}}}),(function(e,r,i,o){!function(t,e,n,r,i,o){if(n){var a=Lb(e),s=Hb(e),u=r*n;null==i?(i=e+r*Bb,o=e-u/2):(i=Ak(a,i),o=Ak(a,o),(r>0?io)&&(i+=r*Bb));for(var l,c=i;r>0?c>o:c0)do{l.point(0===c||3===c?t:n,c>1?r:e)}while((c=(c+s+4)%4)!==f);else l.point(o[0],o[1])}function a(r,i){return Ob(r[0]-t)0?0:3:Ob(r[0]-n)0?2:1:Ob(r[1]-e)0?1:0:i>0?3:2}function s(t,e){return u(t.x,e.x)}function u(t,e){var n=a(t,1),r=a(e,1);return n!==r?n-r:0===n?e[1]-t[1]:1===n?t[0]-e[0]:2===n?t[1]-e[1]:e[0]-t[0]}return function(a){var u,l,c,f,h,d,p,g,m,y,v,_=a,x=Mk(),b={point:w,lineStart:function(){b.point=k,l&&l.push(c=[]);y=!0,m=!1,p=g=NaN},lineEnd:function(){u&&(k(f,h),d&&m&&x.rejoin(),u.push(x.result()));b.point=w,m&&_.lineEnd()},polygonStart:function(){_=x,u=[],l=[],v=!0},polygonEnd:function(){var e=function(){for(var e=0,n=0,i=l.length;nr&&(h-o)*(r-a)>(d-a)*(t-o)&&++e:d<=r&&(h-o)*(r-a)<(d-a)*(t-o)&&--e;return e}(),n=v&&e,i=(u=Fe(u)).length;(n||i)&&(a.polygonStart(),n&&(a.lineStart(),o(null,null,1,a),a.lineEnd()),i&&Ck(u,s,e,o,a),a.polygonEnd());_=a,u=l=c=null}};function w(t,e){i(t,e)&&_.point(t,e)}function k(o,a){var s=i(o,a);if(l&&c.push([o,a]),y)f=o,h=a,d=s,y=!1,s&&(_.lineStart(),_.point(o,a));else if(s&&m)_.point(o,a);else{var u=[p=Math.max(Rk,Math.min(Ok,p)),g=Math.max(Rk,Math.min(Ok,g))],x=[o=Math.max(Rk,Math.min(Ok,o)),a=Math.max(Rk,Math.min(Ok,a))];!function(t,e,n,r,i,o){var a,s=t[0],u=t[1],l=0,c=1,f=e[0]-s,h=e[1]-u;if(a=n-s,f||!(a>0)){if(a/=f,f<0){if(a0){if(a>c)return;a>l&&(l=a)}if(a=i-s,f||!(a<0)){if(a/=f,f<0){if(a>c)return;a>l&&(l=a)}else if(f>0){if(a0)){if(a/=h,h<0){if(a0){if(a>c)return;a>l&&(l=a)}if(a=o-u,h||!(a<0)){if(a/=h,h<0){if(a>c)return;a>l&&(l=a)}else if(h>0){if(a0&&(t[0]=s+l*f,t[1]=u+l*h),c<1&&(e[0]=s+c*f,e[1]=u+c*h),!0}}}}}(u,x,t,e,n,r)?s&&(_.lineStart(),_.point(o,a),v=!1):(m||(_.lineStart(),_.point(u[0],u[1])),_.point(x[0],x[1]),s||_.lineEnd(),v=!1)}p=o,g=a,m=s}return b}}function Lk(t,e,n){var r=Se(t,e-Cb,n).concat(e);return function(t){return r.map((function(e){return[t,e]}))}}function qk(t,e,n){var r=Se(t,e-Cb,n).concat(e);return function(t){return r.map((function(e){return[e,t]}))}}var Pk,jk,Ik,Wk,Hk=t=>t,Yk=new se,Gk=new se,Vk={point:Zb,lineStart:Zb,lineEnd:Zb,polygonStart:function(){Vk.lineStart=Xk,Vk.lineEnd=Qk},polygonEnd:function(){Vk.lineStart=Vk.lineEnd=Vk.point=Zb,Yk.add(Ob(Gk)),Gk=new se},result:function(){var t=Yk/2;return Yk=new se,t}};function Xk(){Vk.point=Jk}function Jk(t,e){Vk.point=Zk,Pk=Ik=t,jk=Wk=e}function Zk(t,e){Gk.add(Wk*t-Ik*e),Ik=t,Wk=e}function Qk(){Zk(Pk,jk)}var Kk=1/0,tA=Kk,eA=-Kk,nA=eA,rA={point:function(t,e){teA&&(eA=t);enA&&(nA=e)},lineStart:Zb,lineEnd:Zb,polygonStart:Zb,polygonEnd:Zb,result:function(){var t=[[Kk,tA],[eA,nA]];return eA=nA=-(tA=Kk=1/0),t}};var iA,oA,aA,sA,uA=0,lA=0,cA=0,fA=0,hA=0,dA=0,pA=0,gA=0,mA=0,yA={point:vA,lineStart:_A,lineEnd:wA,polygonStart:function(){yA.lineStart=kA,yA.lineEnd=AA},polygonEnd:function(){yA.point=vA,yA.lineStart=_A,yA.lineEnd=wA},result:function(){var t=mA?[pA/mA,gA/mA]:dA?[fA/dA,hA/dA]:cA?[uA/cA,lA/cA]:[NaN,NaN];return uA=lA=cA=fA=hA=dA=pA=gA=mA=0,t}};function vA(t,e){uA+=t,lA+=e,++cA}function _A(){yA.point=xA}function xA(t,e){yA.point=bA,vA(aA=t,sA=e)}function bA(t,e){var n=t-aA,r=e-sA,i=Gb(n*n+r*r);fA+=i*(aA+t)/2,hA+=i*(sA+e)/2,dA+=i,vA(aA=t,sA=e)}function wA(){yA.point=vA}function kA(){yA.point=MA}function AA(){EA(iA,oA)}function MA(t,e){yA.point=EA,vA(iA=aA=t,oA=sA=e)}function EA(t,e){var n=t-aA,r=e-sA,i=Gb(n*n+r*r);fA+=i*(aA+t)/2,hA+=i*(sA+e)/2,dA+=i,pA+=(i=sA*t-aA*e)*(aA+t),gA+=i*(sA+e),mA+=3*i,vA(aA=t,sA=e)}function DA(t){this._context=t}DA.prototype={_radius:4.5,pointRadius:function(t){return this._radius=t,this},polygonStart:function(){this._line=0},polygonEnd:function(){this._line=NaN},lineStart:function(){this._point=0},lineEnd:function(){0===this._line&&this._context.closePath(),this._point=NaN},point:function(t,e){switch(this._point){case 0:this._context.moveTo(t,e),this._point=1;break;case 1:this._context.lineTo(t,e);break;default:this._context.moveTo(t+this._radius,e),this._context.arc(t,e,this._radius,0,Bb)}},result:Zb};var CA,FA,SA,$A,TA,BA=new se,zA={point:Zb,lineStart:function(){zA.point=NA},lineEnd:function(){CA&&OA(FA,SA),zA.point=Zb},polygonStart:function(){CA=!0},polygonEnd:function(){CA=null},result:function(){var t=+BA;return BA=new se,t}};function NA(t,e){zA.point=OA,FA=$A=t,SA=TA=e}function OA(t,e){$A-=t,TA-=e,BA.add(Gb($A*$A+TA*TA)),$A=t,TA=e}let RA,UA,LA,qA;class PA{constructor(t){this._append=null==t?jA:function(t){const e=Math.floor(t);if(!(e>=0))throw new RangeError(`invalid digits: ${t}`);if(e>15)return jA;if(e!==RA){const t=10**e;RA=e,UA=function(e){let n=1;this._+=e[0];for(const r=e.length;n=0))throw new RangeError(`invalid digits: ${t}`);i=e}return null===e&&(r=new PA(i)),a},a.projection(t).digits(i).context(e)}function WA(t){return function(e){var n=new HA;for(var r in t)n[r]=t[r];return n.stream=e,n}}function HA(){}function YA(t,e,n){var r=t.clipExtent&&t.clipExtent();return t.scale(150).translate([0,0]),null!=r&&t.clipExtent(null),rw(n,t.stream(rA)),e(rA.result()),null!=r&&t.clipExtent(r),t}function GA(t,e,n){return YA(t,(function(n){var r=e[1][0]-e[0][0],i=e[1][1]-e[0][1],o=Math.min(r/(n[1][0]-n[0][0]),i/(n[1][1]-n[0][1])),a=+e[0][0]+(r-o*(n[1][0]+n[0][0]))/2,s=+e[0][1]+(i-o*(n[1][1]+n[0][1]))/2;t.scale(150*o).translate([a,s])}),n)}function VA(t,e,n){return GA(t,[[0,0],e],n)}function XA(t,e,n){return YA(t,(function(n){var r=+e,i=r/(n[1][0]-n[0][0]),o=(r-i*(n[1][0]+n[0][0]))/2,a=-i*n[0][1];t.scale(150*i).translate([o,a])}),n)}function JA(t,e,n){return YA(t,(function(n){var r=+e,i=r/(n[1][1]-n[0][1]),o=-i*n[0][0],a=(r-i*(n[1][1]+n[0][1]))/2;t.scale(150*i).translate([o,a])}),n)}HA.prototype={constructor:HA,point:function(t,e){this.stream.point(t,e)},sphere:function(){this.stream.sphere()},lineStart:function(){this.stream.lineStart()},lineEnd:function(){this.stream.lineEnd()},polygonStart:function(){this.stream.polygonStart()},polygonEnd:function(){this.stream.polygonEnd()}};var ZA=16,QA=Lb(30*Nb);function KA(t,e){return+e?function(t,e){function n(r,i,o,a,s,u,l,c,f,h,d,p,g,m){var y=l-r,v=c-i,_=y*y+v*v;if(_>4*e&&g--){var x=a+h,b=s+d,w=u+p,k=Gb(x*x+b*b+w*w),A=Jb(w/=k),M=Ob(Ob(w)-1)e||Ob((y*F+v*S)/_-.5)>.3||a*h+s*d+u*p2?t[2]%360*Nb:0,F()):[m*zb,y*zb,v*zb]},D.angle=function(t){return arguments.length?(_=t%360*Nb,F()):_*zb},D.reflectX=function(t){return arguments.length?(x=t?-1:1,F()):x<0},D.reflectY=function(t){return arguments.length?(b=t?-1:1,F()):b<0},D.precision=function(t){return arguments.length?(a=KA(s,E=t*t),S()):Gb(E)},D.fitExtent=function(t,e){return GA(D,t,e)},D.fitSize=function(t,e){return VA(D,t,e)},D.fitWidth=function(t,e){return XA(D,t,e)},D.fitHeight=function(t,e){return JA(D,t,e)},function(){return e=t.apply(this,arguments),D.invert=e.invert&&C,F()}}function iM(t){var e=0,n=Sb/3,r=rM(t),i=r(e,n);return i.parallels=function(t){return arguments.length?r(e=t[0]*Nb,n=t[1]*Nb):[e*zb,n*zb]},i}function oM(t,e){var n=Hb(t),r=(n+Hb(e))/2;if(Ob(r)2?t[2]*Nb:0),e.invert=function(e){return(e=t.invert(e[0]*Nb,e[1]*Nb))[0]*=zb,e[1]*=zb,e},e}(i.rotate()).invert([0,0]));return u(null==l?[[s[0]-o,s[1]-o],[s[0]+o,s[1]+o]]:t===hM?[[Math.max(s[0]-o,l),e],[Math.min(s[0]+o,n),r]]:[[l,Math.max(s[1]-o,e)],[n,Math.min(s[1]+o,r)]])}return i.scale=function(t){return arguments.length?(a(t),c()):a()},i.translate=function(t){return arguments.length?(s(t),c()):s()},i.center=function(t){return arguments.length?(o(t),c()):o()},i.clipExtent=function(t){return arguments.length?(null==t?l=e=n=r=null:(l=+t[0][0],e=+t[0][1],n=+t[1][0],r=+t[1][1]),c()):null==l?null:[[l,e],[n,r]]},c()}function pM(t){return Vb(($b+t)/2)}function gM(t,e){var n=Lb(t),r=t===e?Hb(t):Ib(n/Lb(e))/Ib(pM(e)/pM(t)),i=n*Wb(pM(t),r)/r;if(!r)return hM;function o(t,e){i>0?e<-$b+Cb&&(e=-$b+Cb):e>$b-Cb&&(e=$b-Cb);var n=i/Wb(pM(e),r);return[n*Hb(r*t),i-n*Lb(r*t)]}return o.invert=function(t,e){var n=i-e,o=Yb(r)*Gb(t*t+n*n),a=Ub(t,Ob(n))*Yb(n);return n*r<0&&(a-=Sb*Yb(t)*Yb(n)),[a/r,2*Rb(Wb(i/o,1/r))-$b]},o}function mM(t,e){return[t,e]}function yM(t,e){var n=Lb(t),r=t===e?Hb(t):(n-Lb(e))/(e-t),i=n/r+t;if(Ob(r)Cb&&--i>0);return[t/(.8707+(o=r*r)*(o*(o*o*o*(.003971-.001529*o)-.013791)-.131979)),r]},EM.invert=lM(Jb),DM.invert=lM((function(t){return 2*Rb(t)})),CM.invert=function(t,e){return[-e,2*Rb(Pb(t))-$b]};var FM=Math.abs,SM=Math.cos,$M=Math.sin,TM=1e-6,BM=Math.PI,zM=BM/2,NM=function(t){return t>0?Math.sqrt(t):0}(2);function OM(t){return t>1?zM:t<-1?-zM:Math.asin(t)}function RM(t,e){var n,r=t*$M(e),i=30;do{e-=n=(e+$M(e)-r)/(1+SM(e))}while(FM(n)>TM&&--i>0);return e/2}var UM=function(t,e,n){function r(r,i){return[t*r*SM(i=RM(n,i)),e*$M(i)]}return r.invert=function(r,i){return i=OM(i/e),[r/(t*SM(i)),OM((2*i+$M(2*i))/n)]},r}(NM/zM,NM,BM);const LM=IA(),qM=[\"clipAngle\",\"clipExtent\",\"scale\",\"translate\",\"center\",\"rotate\",\"parallels\",\"precision\",\"reflectX\",\"reflectY\",\"coefficient\",\"distance\",\"fraction\",\"lobes\",\"parallel\",\"radius\",\"ratio\",\"spacing\",\"tilt\"];function PM(t,e){if(!t||\"string\"!=typeof t)throw new Error(\"Projection type must be a name string.\");return t=t.toLowerCase(),arguments.length>1?(IM[t]=function(t,e){return function n(){const r=e();return r.type=t,r.path=IA().projection(r),r.copy=r.copy||function(){const t=n();return qM.forEach((e=>{r[e]&&t[e](r[e]())})),t.path.pointRadius(r.path.pointRadius()),t},Vd(r)}}(t,e),this):IM[t]||null}function jM(t){return t&&t.path||LM}const IM={albers:sM,albersusa:function(){var t,e,n,r,i,o,a=sM(),s=aM().rotate([154,0]).center([-2,58.5]).parallels([55,65]),u=aM().rotate([157,0]).center([-3,19.9]).parallels([8,18]),l={point:function(t,e){o=[t,e]}};function c(t){var e=t[0],a=t[1];return o=null,n.point(e,a),o||(r.point(e,a),o)||(i.point(e,a),o)}function f(){return t=e=null,c}return c.invert=function(t){var e=a.scale(),n=a.translate(),r=(t[0]-n[0])/e,i=(t[1]-n[1])/e;return(i>=.12&&i<.234&&r>=-.425&&r<-.214?s:i>=.166&&i<.234&&r>=-.214&&r<-.115?u:a).invert(t)},c.stream=function(n){return t&&e===n?t:(r=[a.stream(e=n),s.stream(n),u.stream(n)],i=r.length,t={point:function(t,e){for(var n=-1;++n2?t[2]+90:90]):[(t=n())[0],t[1],t[2]-90]},n([0,0,90]).scale(159.155)}};for(const t in IM)PM(t,IM[t]);function WM(){}const HM=[[],[[[1,1.5],[.5,1]]],[[[1.5,1],[1,1.5]]],[[[1.5,1],[.5,1]]],[[[1,.5],[1.5,1]]],[[[1,1.5],[.5,1]],[[1,.5],[1.5,1]]],[[[1,.5],[1,1.5]]],[[[1,.5],[.5,1]]],[[[.5,1],[1,.5]]],[[[1,1.5],[1,.5]]],[[[.5,1],[1,.5]],[[1.5,1],[1,1.5]]],[[[1.5,1],[1,.5]]],[[[.5,1],[1.5,1]]],[[[1,1.5],[1.5,1]]],[[[.5,1],[1,1.5]]],[]];function YM(){var t=1,e=1,n=a;function r(t,e){return e.map((e=>i(t,e)))}function i(r,i){var a=[],s=[];return function(n,r,i){var a,s,u,l,c,f,h=new Array,d=new Array;a=s=-1,l=n[0]>=r,HM[l<<1].forEach(p);for(;++a=r,HM[u|l<<1].forEach(p);HM[l<<0].forEach(p);for(;++s=r,c=n[s*t]>=r,HM[l<<1|c<<2].forEach(p);++a=r,f=c,c=n[s*t+a+1]>=r,HM[u|l<<1|c<<2|f<<3].forEach(p);HM[l|c<<3].forEach(p)}a=-1,c=n[s*t]>=r,HM[c<<2].forEach(p);for(;++a=r,HM[c<<2|f<<3].forEach(p);function p(t){var e,n,r=[t[0][0]+a,t[0][1]+s],u=[t[1][0]+a,t[1][1]+s],l=o(r),c=o(u);(e=d[l])?(n=h[c])?(delete d[e.end],delete h[n.start],e===n?(e.ring.push(u),i(e.ring)):h[e.start]=d[n.end]={start:e.start,end:n.end,ring:e.ring.concat(n.ring)}):(delete d[e.end],e.ring.push(u),d[e.end=c]=e):(e=h[c])?(n=d[l])?(delete h[e.start],delete d[n.end],e===n?(e.ring.push(u),i(e.ring)):h[n.start]=d[e.end]={start:n.start,end:e.end,ring:n.ring.concat(e.ring)}):(delete h[e.start],e.ring.unshift(r),h[e.start=l]=e):h[l]=d[c]={start:l,end:c,ring:[r,u]}}HM[c<<3].forEach(p)}(r,i,(t=>{n(t,r,i),function(t){var e=0,n=t.length,r=t[n-1][1]*t[0][0]-t[n-1][0]*t[0][1];for(;++e0?a.push([t]):s.push(t)})),s.forEach((t=>{for(var e,n=0,r=a.length;n{var o,a=n[0],s=n[1],u=0|a,l=0|s,c=r[l*t+u];a>0&&a0&&s=0&&o>=0||s(\"invalid size\"),t=i,e=o,r},r.smooth=function(t){return arguments.length?(n=t?a:WM,r):n===a},r}function GM(t,e){for(var n,r=-1,i=e.length;++rr!=d>r&&n<(h-l)*(r-c)/(d-c)+l&&(i=-i)}return i}function XM(t,e,n){var r,i,o,a;return function(t,e,n){return(e[0]-t[0])*(n[1]-t[1])==(n[0]-t[0])*(e[1]-t[1])}(t,e,n)&&(i=t[r=+(t[0]===e[0])],o=n[r],a=e[r],i<=o&&o<=a||a<=o&&o<=i)}function JM(t,e,n){return function(r){var i=at(r),o=n?Math.min(i[0],0):i[0],a=i[1],s=a-o,u=e?be(o,a,t):s/(t+1);return Se(o+u,a,u)}}function ZM(t){Ja.call(this,null,t)}function QM(t,e,n,r,i){const o=t.x1||0,a=t.y1||0,s=e*n<0;function u(t){t.forEach(l)}function l(t){s&&t.reverse(),t.forEach(c)}function c(t){t[0]=(t[0]-o)*e+r,t[1]=(t[1]-a)*n+i}return function(t){return t.coordinates.forEach(u),t}}function KM(t,e,n){const r=t>=0?t:rs(e,n);return Math.round((Math.sqrt(4*r*r+1)-1)/2)}function tE(t){return J(t)?t:rt(+t)}function eE(){var t=t=>t[0],e=t=>t[1],n=d,r=[-1,-1],i=960,o=500,a=2;function u(s,u){const l=KM(r[0],s,t)>>a,c=KM(r[1],s,e)>>a,f=l?l+2:0,h=c?c+2:0,d=2*f+(i>>a),p=2*h+(o>>a),g=new Float32Array(d*p),m=new Float32Array(d*p);let y=g;s.forEach((r=>{const i=f+(+t(r)>>a),o=h+(+e(r)>>a);i>=0&&i=0&&o0&&c>0?(nE(d,p,g,m,l),rE(d,p,m,g,c),nE(d,p,g,m,l),rE(d,p,m,g,c),nE(d,p,g,m,l),rE(d,p,m,g,c)):l>0?(nE(d,p,g,m,l),nE(d,p,m,g,l),nE(d,p,g,m,l),y=m):c>0&&(rE(d,p,g,m,c),rE(d,p,m,g,c),rE(d,p,g,m,c),y=m);const v=u?Math.pow(2,-2*a):1/$e(y);for(let t=0,e=d*p;t>a),y2:h+(o>>a)}}return u.x=function(e){return arguments.length?(t=tE(e),u):t},u.y=function(t){return arguments.length?(e=tE(t),u):e},u.weight=function(t){return arguments.length?(n=tE(t),u):n},u.size=function(t){if(!arguments.length)return[i,o];var e=+t[0],n=+t[1];return e>=0&&n>=0||s(\"invalid size\"),i=e,o=n,u},u.cellSize=function(t){return arguments.length?((t=+t)>=1||s(\"invalid cell size\"),a=Math.floor(Math.log(t)/Math.LN2),u):1<=i&&(e>=o&&(s-=n[e-o+a*t]),r[e-i+a*t]=s/Math.min(e+1,t-1+o-e,o))}function rE(t,e,n,r,i){const o=1+(i<<1);for(let a=0;a=i&&(s>=o&&(u-=n[a+(s-o)*t]),r[a+(s-i)*t]=u/Math.min(s+1,e-1+o-s,o))}function iE(t){Ja.call(this,null,t)}ZM.Definition={type:\"Isocontour\",metadata:{generates:!0},params:[{name:\"field\",type:\"field\"},{name:\"thresholds\",type:\"number\",array:!0},{name:\"levels\",type:\"number\"},{name:\"nice\",type:\"boolean\",default:!1},{name:\"resolve\",type:\"enum\",values:[\"shared\",\"independent\"],default:\"independent\"},{name:\"zero\",type:\"boolean\",default:!0},{name:\"smooth\",type:\"boolean\",default:!0},{name:\"scale\",type:\"number\",expr:!0},{name:\"translate\",type:\"number\",array:!0,expr:!0},{name:\"as\",type:\"string\",null:!0,default:\"contour\"}]},dt(ZM,Ja,{transform(t,e){if(this.value&&!e.changed()&&!t.modified())return e.StopPropagation;var n=e.fork(e.NO_SOURCE|e.NO_FIELDS),r=e.materialize(e.SOURCE).source,i=t.field||f,o=YM().smooth(!1!==t.smooth),a=t.thresholds||function(t,e,n){const r=JM(n.levels||10,n.nice,!1!==n.zero);return\"shared\"!==n.resolve?r:r(t.map((t=>we(e(t).values))))}(r,i,t),s=null===t.as?null:t.as||\"contour\",u=[];return r.forEach((e=>{const n=i(e),r=o.size([n.width,n.height])(n.values,k(a)?a:a(n.values));!function(t,e,n,r){let i=r.scale||e.scale,o=r.translate||e.translate;J(i)&&(i=i(n,r));J(o)&&(o=o(n,r));if((1===i||null==i)&&!o)return;const a=(vt(i)?i:i[0])||1,s=(vt(i)?i:i[1])||1,u=o&&o[0]||0,l=o&&o[1]||0;t.forEach(QM(e,a,s,u,l))}(r,n,e,t),r.forEach((t=>{u.push(ba(e,_a(null!=s?{[s]:t}:t)))}))})),this.value&&(n.rem=this.value),this.value=n.source=n.add=u,n}}),iE.Definition={type:\"KDE2D\",metadata:{generates:!0},params:[{name:\"size\",type:\"number\",array:!0,length:2,required:!0},{name:\"x\",type:\"field\",required:!0},{name:\"y\",type:\"field\",required:!0},{name:\"weight\",type:\"field\"},{name:\"groupby\",type:\"field\",array:!0},{name:\"cellSize\",type:\"number\"},{name:\"bandwidth\",type:\"number\",array:!0,length:2},{name:\"counts\",type:\"boolean\",default:!1},{name:\"as\",type:\"string\",default:\"grid\"}]};const oE=[\"x\",\"y\",\"weight\",\"size\",\"cellSize\",\"bandwidth\"];function aE(t,e){return oE.forEach((n=>null!=e[n]?t[n](e[n]):0)),t}function sE(t){Ja.call(this,null,t)}dt(iE,Ja,{transform(t,e){if(this.value&&!e.changed()&&!t.modified())return e.StopPropagation;var r,i=e.fork(e.NO_SOURCE|e.NO_FIELDS),o=function(t,e){var n,r,i,o,a,s,u=[],l=t=>t(o);if(null==e)u.push(t);else for(n={},r=0,i=t.length;r_a(function(t,e){for(let n=0;nCb})).map(u)).concat(Se(qb(o/d)*d,i,d).filter((function(t){return Ob(t%g)>Cb})).map(l))}return y.lines=function(){return v().map((function(t){return{type:\"LineString\",coordinates:t}}))},y.outline=function(){return{type:\"Polygon\",coordinates:[c(r).concat(f(a).slice(1),c(n).reverse().slice(1),f(s).reverse().slice(1))]}},y.extent=function(t){return arguments.length?y.extentMajor(t).extentMinor(t):y.extentMinor()},y.extentMajor=function(t){return arguments.length?(r=+t[0][0],n=+t[1][0],s=+t[0][1],a=+t[1][1],r>n&&(t=r,r=n,n=t),s>a&&(t=s,s=a,a=t),y.precision(m)):[[r,s],[n,a]]},y.extentMinor=function(n){return arguments.length?(e=+n[0][0],t=+n[1][0],o=+n[0][1],i=+n[1][1],e>t&&(n=e,e=t,t=n),o>i&&(n=o,o=i,i=n),y.precision(m)):[[e,o],[t,i]]},y.step=function(t){return arguments.length?y.stepMajor(t).stepMinor(t):y.stepMinor()},y.stepMajor=function(t){return arguments.length?(p=+t[0],g=+t[1],y):[p,g]},y.stepMinor=function(t){return arguments.length?(h=+t[0],d=+t[1],y):[h,d]},y.precision=function(h){return arguments.length?(m=+h,u=Lk(o,i,90),l=qk(e,t,m),c=Lk(s,a,90),f=qk(r,n,m),y):m},y.extentMajor([[-180,-90+Cb],[180,90-Cb]]).extentMinor([[-180,-80-Cb],[180,80+Cb]])}()}function gE(t){Ja.call(this,null,t)}function mE(t){if(!J(t))return!1;const e=Bt(r(t));return e.$x||e.$y||e.$value||e.$max}function yE(t){Ja.call(this,null,t),this.modified(!0)}function vE(t,e,n){J(t[e])&&t[e](n)}cE.Definition={type:\"GeoJSON\",metadata:{},params:[{name:\"fields\",type:\"field\",array:!0,length:2},{name:\"geojson\",type:\"field\"}]},dt(cE,Ja,{transform(t,e){var n,i=this._features,o=this._points,a=t.fields,s=a&&a[0],u=a&&a[1],l=t.geojson||!a&&f,c=e.ADD;n=t.modified()||e.changed(e.REM)||e.modified(r(l))||s&&e.modified(r(s))||u&&e.modified(r(u)),this.value&&!n||(c=e.SOURCE,this._features=i=[],this._points=o=[]),l&&e.visit(c,(t=>i.push(l(t)))),s&&u&&(e.visit(c,(t=>{var e=s(t),n=u(t);null!=e&&null!=n&&(e=+e)===e&&(n=+n)===n&&o.push([e,n])})),i=i.concat({type:uE,geometry:{type:\"MultiPoint\",coordinates:o}})),this.value={type:lE,features:i}}}),fE.Definition={type:\"GeoPath\",metadata:{modifies:!0},params:[{name:\"projection\",type:\"projection\"},{name:\"field\",type:\"field\"},{name:\"pointRadius\",type:\"number\",expr:!0},{name:\"as\",type:\"string\",default:\"path\"}]},dt(fE,Ja,{transform(t,e){var n=e.fork(e.ALL),r=this.value,i=t.field||f,o=t.as||\"path\",a=n.SOURCE;!r||t.modified()?(this.value=r=jM(t.projection),n.materialize().reflow()):a=i===f||e.modified(i.fields)?n.ADD_MOD:n.ADD;const s=function(t,e){const n=t.pointRadius();t.context(null),null!=e&&t.pointRadius(e);return n}(r,t.pointRadius);return n.visit(a,(t=>t[o]=r(i(t)))),r.pointRadius(s),n.modifies(o)}}),hE.Definition={type:\"GeoPoint\",metadata:{modifies:!0},params:[{name:\"projection\",type:\"projection\",required:!0},{name:\"fields\",type:\"field\",array:!0,required:!0,length:2},{name:\"as\",type:\"string\",array:!0,length:2,default:[\"x\",\"y\"]}]},dt(hE,Ja,{transform(t,e){var n,r=t.projection,i=t.fields[0],o=t.fields[1],a=t.as||[\"x\",\"y\"],s=a[0],u=a[1];function l(t){const e=r([i(t),o(t)]);e?(t[s]=e[0],t[u]=e[1]):(t[s]=void 0,t[u]=void 0)}return t.modified()?e=e.materialize().reflow(!0).visit(e.SOURCE,l):(n=e.modified(i.fields)||e.modified(o.fields),e.visit(n?e.ADD_MOD:e.ADD,l)),e.modifies(a)}}),dE.Definition={type:\"GeoShape\",metadata:{modifies:!0,nomod:!0},params:[{name:\"projection\",type:\"projection\"},{name:\"field\",type:\"field\",default:\"datum\"},{name:\"pointRadius\",type:\"number\",expr:!0},{name:\"as\",type:\"string\",default:\"shape\"}]},dt(dE,Ja,{transform(t,e){var n=e.fork(e.ALL),r=this.value,i=t.as||\"shape\",o=n.ADD;return r&&!t.modified()||(this.value=r=function(t,e,n){const r=null==n?n=>t(e(n)):r=>{var i=t.pointRadius(),o=t.pointRadius(n)(e(r));return t.pointRadius(i),o};return r.context=e=>(t.context(e),r),r}(jM(t.projection),t.field||l(\"datum\"),t.pointRadius),n.materialize().reflow(),o=n.SOURCE),n.visit(o,(t=>t[i]=r)),n.modifies(i)}}),pE.Definition={type:\"Graticule\",metadata:{changes:!0,generates:!0},params:[{name:\"extent\",type:\"array\",array:!0,length:2,content:{type:\"number\",array:!0,length:2}},{name:\"extentMajor\",type:\"array\",array:!0,length:2,content:{type:\"number\",array:!0,length:2}},{name:\"extentMinor\",type:\"array\",array:!0,length:2,content:{type:\"number\",array:!0,length:2}},{name:\"step\",type:\"number\",array:!0,length:2},{name:\"stepMajor\",type:\"number\",array:!0,length:2,default:[90,360]},{name:\"stepMinor\",type:\"number\",array:!0,length:2,default:[10,10]},{name:\"precision\",type:\"number\",default:2.5}]},dt(pE,Ja,{transform(t,e){var n,r=this.value,i=this.generator;if(!r.length||t.modified())for(const e in t)J(i[e])&&i[e](t[e]);return n=i(),r.length?e.mod.push(wa(r[0],n)):e.add.push(_a(n)),r[0]=n,e}}),gE.Definition={type:\"heatmap\",metadata:{modifies:!0},params:[{name:\"field\",type:\"field\"},{name:\"color\",type:\"string\",expr:!0},{name:\"opacity\",type:\"number\",expr:!0},{name:\"resolve\",type:\"enum\",values:[\"shared\",\"independent\"],default:\"independent\"},{name:\"as\",type:\"string\",default:\"image\"}]},dt(gE,Ja,{transform(t,e){if(!e.changed()&&!t.modified())return e.StopPropagation;var n=e.materialize(e.SOURCE).source,r=\"shared\"===t.resolve,i=t.field||f,o=function(t,e){let n;J(t)?(n=n=>t(n,e),n.dep=mE(t)):t?n=rt(t):(n=t=>t.$value/t.$max||0,n.dep=!0);return n}(t.opacity,t),a=function(t,e){let n;J(t)?(n=n=>af(t(n,e)),n.dep=mE(t)):n=rt(af(t||\"#888\"));return n}(t.color,t),s=t.as||\"image\",u={$x:0,$y:0,$value:0,$max:r?we(n.map((t=>we(i(t).values)))):0};return n.forEach((t=>{const e=i(t),n=ot({},t,u);r||(n.$max=we(e.values||[])),t[s]=function(t,e,n,r){const i=t.width,o=t.height,a=t.x1||0,s=t.y1||0,u=t.x2||i,l=t.y2||o,c=t.values,f=c?t=>c[t]:h,d=$c(u-a,l-s),p=d.getContext(\"2d\"),g=p.getImageData(0,0,u-a,l-s),m=g.data;for(let t=s,o=0;t{null!=t[e]&&vE(n,e,t[e])}))):qM.forEach((e=>{t.modified(e)&&vE(n,e,t[e])})),null!=t.pointRadius&&n.path.pointRadius(t.pointRadius),t.fit&&function(t,e){const n=function(t){return t=V(t),1===t.length?t[0]:{type:lE,features:t.reduce(((t,e)=>t.concat(function(t){return t.type===lE?t.features:V(t).filter((t=>null!=t)).map((t=>t.type===uE?t:{type:uE,geometry:t}))}(e))),[])}}(e.fit);e.extent?t.fitExtent(e.extent,n):e.size&&t.fitSize(e.size,n)}(n,t),e.fork(e.NO_SOURCE|e.NO_FIELDS)}});var _E=Object.freeze({__proto__:null,contour:sE,geojson:cE,geopath:fE,geopoint:hE,geoshape:dE,graticule:pE,heatmap:gE,isocontour:ZM,kde2d:iE,projection:yE});function xE(t,e,n,r){if(isNaN(e)||isNaN(n))return t;var i,o,a,s,u,l,c,f,h,d=t._root,p={data:r},g=t._x0,m=t._y0,y=t._x1,v=t._y1;if(!d)return t._root=p,t;for(;d.length;)if((l=e>=(o=(g+y)/2))?g=o:y=o,(c=n>=(a=(m+v)/2))?m=a:v=a,i=d,!(d=d[f=c<<1|l]))return i[f]=p,t;if(s=+t._x.call(null,d.data),u=+t._y.call(null,d.data),e===s&&n===u)return p.next=d,i?i[f]=p:t._root=p,t;do{i=i?i[f]=new Array(4):t._root=new Array(4),(l=e>=(o=(g+y)/2))?g=o:y=o,(c=n>=(a=(m+v)/2))?m=a:v=a}while((f=c<<1|l)==(h=(u>=a)<<1|s>=o));return i[h]=d,i[f]=p,t}function bE(t,e,n,r,i){this.node=t,this.x0=e,this.y0=n,this.x1=r,this.y1=i}function wE(t){return t[0]}function kE(t){return t[1]}function AE(t,e,n){var r=new ME(null==e?wE:e,null==n?kE:n,NaN,NaN,NaN,NaN);return null==t?r:r.addAll(t)}function ME(t,e,n,r,i,o){this._x=t,this._y=e,this._x0=n,this._y0=r,this._x1=i,this._y1=o,this._root=void 0}function EE(t){for(var e={data:t.data},n=e;t=t.next;)n=n.next={data:t.data};return e}var DE=AE.prototype=ME.prototype;function CE(t){return function(){return t}}function FE(t){return 1e-6*(t()-.5)}function SE(t){return t.x+t.vx}function $E(t){return t.y+t.vy}function TE(t){return t.index}function BE(t,e){var n=t.get(e);if(!n)throw new Error(\"node not found: \"+e);return n}DE.copy=function(){var t,e,n=new ME(this._x,this._y,this._x0,this._y0,this._x1,this._y1),r=this._root;if(!r)return n;if(!r.length)return n._root=EE(r),n;for(t=[{source:r,target:n._root=new Array(4)}];r=t.pop();)for(var i=0;i<4;++i)(e=r.source[i])&&(e.length?t.push({source:e,target:r.target[i]=new Array(4)}):r.target[i]=EE(e));return n},DE.add=function(t){const e=+this._x.call(null,t),n=+this._y.call(null,t);return xE(this.cover(e,n),e,n,t)},DE.addAll=function(t){var e,n,r,i,o=t.length,a=new Array(o),s=new Array(o),u=1/0,l=1/0,c=-1/0,f=-1/0;for(n=0;nc&&(c=r),if&&(f=i));if(u>c||l>f)return this;for(this.cover(u,l).cover(c,f),n=0;nt||t>=i||r>e||e>=o;)switch(s=(eh||(o=u.y0)>d||(a=u.x1)=y)<<1|t>=m)&&(u=p[p.length-1],p[p.length-1]=p[p.length-1-l],p[p.length-1-l]=u)}else{var v=t-+this._x.call(null,g.data),_=e-+this._y.call(null,g.data),x=v*v+_*_;if(x=(s=(p+m)/2))?p=s:m=s,(c=a>=(u=(g+y)/2))?g=u:y=u,e=d,!(d=d[f=c<<1|l]))return this;if(!d.length)break;(e[f+1&3]||e[f+2&3]||e[f+3&3])&&(n=e,h=f)}for(;d.data!==t;)if(r=d,!(d=d.next))return this;return(i=d.next)&&delete d.next,r?(i?r.next=i:delete r.next,this):e?(i?e[f]=i:delete e[f],(d=e[0]||e[1]||e[2]||e[3])&&d===(e[3]||e[2]||e[1]||e[0])&&!d.length&&(n?n[h]=d:this._root=d),this):(this._root=i,this)},DE.removeAll=function(t){for(var e=0,n=t.length;e{}};function NE(){for(var t,e=0,n=arguments.length,r={};e=0&&(e=t.slice(n+1),t=t.slice(0,n)),t&&!r.hasOwnProperty(t))throw new Error(\"unknown type: \"+t);return{type:t,name:e}}))),a=-1,s=o.length;if(!(arguments.length<2)){if(null!=e&&\"function\"!=typeof e)throw new Error(\"invalid callback: \"+e);for(;++a0)for(var n,r,i=new Array(n),o=0;o=0&&e._call.call(void 0,t),e=e._next;--PE}()}finally{PE=0,function(){var t,e,n=LE,r=1/0;for(;n;)n._call?(r>n._time&&(r=n._time),t=n,n=n._next):(e=n._next,n._next=null,n=t?t._next=e:LE=e);qE=t,nD(r)}(),YE=0}}function eD(){var t=VE.now(),e=t-HE;e>WE&&(GE-=e,HE=t)}function nD(t){PE||(jE&&(jE=clearTimeout(jE)),t-YE>24?(t<1/0&&(jE=setTimeout(tD,t-VE.now()-GE)),IE&&(IE=clearInterval(IE))):(IE||(HE=VE.now(),IE=setInterval(eD,WE)),PE=1,XE(tD)))}QE.prototype=KE.prototype={constructor:QE,restart:function(t,e,n){if(\"function\"!=typeof t)throw new TypeError(\"callback is not a function\");n=(null==n?JE():+n)+(null==e?0:+e),this._next||qE===this||(qE?qE._next=this:LE=this,qE=this),this._call=t,this._time=n,nD()},stop:function(){this._call&&(this._call=null,this._time=1/0,nD())}};const rD=1664525,iD=1013904223,oD=4294967296;function aD(t){return t.x}function sD(t){return t.y}var uD=10,lD=Math.PI*(3-Math.sqrt(5));function cD(t){var e,n=1,r=.001,i=1-Math.pow(r,1/300),o=0,a=.6,s=new Map,u=KE(f),l=NE(\"tick\",\"end\"),c=function(){let t=1;return()=>(t=(rD*t+iD)%oD)/oD}();function f(){h(),l.call(\"tick\",e),n1?(null==n?s.delete(t):s.set(t,p(n)),e):s.get(t)},find:function(e,n,r){var i,o,a,s,u,l=0,c=t.length;for(null==r?r=1/0:r*=r,l=0;l1?(l.on(t,n),e):l.on(t)}}}const fD={center:function(t,e){var n,r=1;function i(){var i,o,a=n.length,s=0,u=0;for(i=0;il+p||oc+p||au.index){var g=l-s.x-s.vx,m=c-s.y-s.vy,y=g*g+m*m;yt.r&&(t.r=t[e].r)}function u(){if(e){var r,i,o=e.length;for(n=new Array(o),r=0;r=s)){(t.data!==e||t.next)&&(0===f&&(p+=(f=FE(n))*f),0===h&&(p+=(h=FE(n))*h),p[s(t,e,r),t])));for(a=0,i=new Array(l);ae(t,n):e)}mD.Definition={type:\"Force\",metadata:{modifies:!0},params:[{name:\"static\",type:\"boolean\",default:!1},{name:\"restart\",type:\"boolean\",default:!1},{name:\"iterations\",type:\"number\",default:300},{name:\"alpha\",type:\"number\",default:1},{name:\"alphaMin\",type:\"number\",default:.001},{name:\"alphaTarget\",type:\"number\",default:0},{name:\"velocityDecay\",type:\"number\",default:.4},{name:\"forces\",type:\"param\",array:!0,params:[{key:{force:\"center\"},params:[{name:\"x\",type:\"number\",default:0},{name:\"y\",type:\"number\",default:0}]},{key:{force:\"collide\"},params:[{name:\"radius\",type:\"number\",expr:!0},{name:\"strength\",type:\"number\",default:.7},{name:\"iterations\",type:\"number\",default:1}]},{key:{force:\"nbody\"},params:[{name:\"strength\",type:\"number\",default:-30,expr:!0},{name:\"theta\",type:\"number\",default:.9},{name:\"distanceMin\",type:\"number\",default:1},{name:\"distanceMax\",type:\"number\"}]},{key:{force:\"link\"},params:[{name:\"links\",type:\"data\"},{name:\"id\",type:\"field\"},{name:\"distance\",type:\"number\",default:30,expr:!0},{name:\"strength\",type:\"number\",expr:!0},{name:\"iterations\",type:\"number\",default:1}]},{key:{force:\"x\"},params:[{name:\"strength\",type:\"number\",default:.1},{name:\"x\",type:\"field\"}]},{key:{force:\"y\"},params:[{name:\"strength\",type:\"number\",default:.1},{name:\"y\",type:\"field\"}]}]},{name:\"as\",type:\"string\",array:!0,modify:!1,default:gD}]},dt(mD,Ja,{transform(t,e){var n,r,i=this.value,o=e.changed(e.ADD_REM),a=t.modified(dD),s=t.iterations||300;if(i?(o&&(e.modifies(\"index\"),i.nodes(e.source)),(a||e.changed(e.MOD))&&yD(i,t,0,e)):(this.value=i=function(t,e){const n=cD(t),r=n.stop,i=n.restart;let o=!1;return n.stopped=()=>o,n.restart=()=>(o=!1,i()),n.stop=()=>(o=!0,r()),yD(n,e,!0).on(\"end\",(()=>o=!0))}(e.source,t),i.on(\"tick\",(n=e.dataflow,r=this,()=>n.touch(r).run())),t.static||(o=!0,i.tick()),e.modifies(\"index\")),a||o||t.modified(pD)||e.changed()&&t.restart)if(i.alpha(Math.max(i.alpha(),t.alpha||1)).alphaDecay(1-Math.pow(i.alphaMin(),1/s)),t.static)for(i.stop();--s>=0;)i.tick();else if(i.stopped()&&i.restart(),!o)return e.StopPropagation;return this.finish(t,e)},finish(t,e){const n=e.dataflow;for(let t,e=this._argops,s=0,u=e.length;s=0;)e+=n[r].value;else e=1;t.value=e}function ED(t,e){t instanceof Map?(t=[void 0,t],void 0===e&&(e=CD)):void 0===e&&(e=DD);for(var n,r,i,o,a,s=new $D(t),u=[s];n=u.pop();)if((i=e(n.data))&&(a=(i=Array.from(i)).length))for(n.children=i,o=a-1;o>=0;--o)u.push(r=i[o]=new $D(i[o])),r.parent=n,r.depth=n.depth+1;return s.eachBefore(SD)}function DD(t){return t.children}function CD(t){return Array.isArray(t)?t[1]:null}function FD(t){void 0!==t.data.value&&(t.value=t.data.value),t.data=t.data.data}function SD(t){var e=0;do{t.height=e}while((t=t.parent)&&t.height<++e)}function $D(t){this.data=t,this.depth=this.height=0,this.parent=null}function TD(t){return null==t?null:BD(t)}function BD(t){if(\"function\"!=typeof t)throw new Error;return t}function zD(){return 0}function ND(t){return function(){return t}}$D.prototype=ED.prototype={constructor:$D,count:function(){return this.eachAfter(MD)},each:function(t,e){let n=-1;for(const r of this)t.call(e,r,++n,this);return this},eachAfter:function(t,e){for(var n,r,i,o=this,a=[o],s=[],u=-1;o=a.pop();)if(s.push(o),n=o.children)for(r=0,i=n.length;r=0;--r)o.push(n[r]);return this},find:function(t,e){let n=-1;for(const r of this)if(t.call(e,r,++n,this))return r},sum:function(t){return this.eachAfter((function(e){for(var n=+t(e.data)||0,r=e.children,i=r&&r.length;--i>=0;)n+=r[i].value;e.value=n}))},sort:function(t){return this.eachBefore((function(e){e.children&&e.children.sort(t)}))},path:function(t){for(var e=this,n=function(t,e){if(t===e)return t;var n=t.ancestors(),r=e.ancestors(),i=null;t=n.pop(),e=r.pop();for(;t===e;)i=t,t=n.pop(),e=r.pop();return i}(e,t),r=[e];e!==n;)e=e.parent,r.push(e);for(var i=r.length;t!==n;)r.splice(i,0,t),t=t.parent;return r},ancestors:function(){for(var t=this,e=[t];t=t.parent;)e.push(t);return e},descendants:function(){return Array.from(this)},leaves:function(){var t=[];return this.eachBefore((function(e){e.children||t.push(e)})),t},links:function(){var t=this,e=[];return t.each((function(n){n!==t&&e.push({source:n.parent,target:n})})),e},copy:function(){return ED(this).eachBefore(FD)},[Symbol.iterator]:function*(){var t,e,n,r,i=this,o=[i];do{for(t=o.reverse(),o=[];i=t.pop();)if(yield i,e=i.children)for(n=0,r=e.length;n0&&n*n>r*r+i*i}function jD(t,e){for(var n=0;n1e-6?(D+Math.sqrt(D*D-4*E*C))/(2*E):C/D);return{x:r+w+k*F,y:i+A+M*F,r:F}}function YD(t,e,n){var r,i,o,a,s=t.x-e.x,u=t.y-e.y,l=s*s+u*u;l?(i=e.r+n.r,i*=i,a=t.r+n.r,i>(a*=a)?(r=(l+a-i)/(2*l),o=Math.sqrt(Math.max(0,a/l-r*r)),n.x=t.x-r*s-o*u,n.y=t.y-r*u+o*s):(r=(l+i-a)/(2*l),o=Math.sqrt(Math.max(0,i/l-r*r)),n.x=e.x+r*s-o*u,n.y=e.y+r*u+o*s)):(n.x=e.x+n.r,n.y=e.y)}function GD(t,e){var n=t.r+e.r-1e-6,r=e.x-t.x,i=e.y-t.y;return n>0&&n*n>r*r+i*i}function VD(t){var e=t._,n=t.next._,r=e.r+n.r,i=(e.x*n.r+n.x*e.r)/r,o=(e.y*n.r+n.y*e.r)/r;return i*i+o*o}function XD(t){this._=t,this.next=null,this.previous=null}function JD(t,e){if(!(o=(t=function(t){return\"object\"==typeof t&&\"length\"in t?t:Array.from(t)}(t)).length))return 0;var n,r,i,o,a,s,u,l,c,f,h;if((n=t[0]).x=0,n.y=0,!(o>1))return n.r;if(r=t[1],n.x=-r.r,r.x=n.r,r.y=0,!(o>2))return n.r+r.r;YD(r,n,i=t[2]),n=new XD(n),r=new XD(r),i=new XD(i),n.next=i.previous=r,r.next=n.previous=i,i.next=r.previous=n;t:for(u=3;ufunction(t){t=`${t}`;let e=t.length;cC(t,e-1)&&!cC(t,e-2)&&(t=t.slice(0,-1));return\"/\"===t[0]?t:`/${t}`}(t(e,n,r)))),n=e.map(lC),i=new Set(e).add(\"\");for(const t of n)i.has(t)||(i.add(t),e.push(t),n.push(lC(t)),h.push(oC));d=(t,n)=>e[n],p=(t,e)=>n[e]}for(a=0,i=h.length;a=0&&(l=h[t]).data===oC;--t)l.data=null}if(s.parent=rC,s.eachBefore((function(t){t.depth=t.parent.depth+1,--i})).eachBefore(SD),s.parent=null,i>0)throw new Error(\"cycle\");return s}return r.id=function(t){return arguments.length?(e=TD(t),r):e},r.parentId=function(t){return arguments.length?(n=TD(t),r):n},r.path=function(e){return arguments.length?(t=TD(e),r):t},r}function lC(t){let e=t.length;if(e<2)return\"\";for(;--e>1&&!cC(t,e););return t.slice(0,e)}function cC(t,e){if(\"/\"===t[e]){let n=0;for(;e>0&&\"\\\\\"===t[--e];)++n;if(0==(1&n))return!0}return!1}function fC(t,e){return t.parent===e.parent?1:2}function hC(t){var e=t.children;return e?e[0]:t.t}function dC(t){var e=t.children;return e?e[e.length-1]:t.t}function pC(t,e,n){var r=n/(e.i-t.i);e.c-=r,e.s+=n,t.c+=r,e.z+=n,e.m+=n}function gC(t,e,n){return t.a.parent===e.parent?t.a:n}function mC(t,e){this._=t,this.parent=null,this.children=null,this.A=null,this.a=this,this.z=0,this.m=0,this.c=0,this.s=0,this.t=null,this.i=e}function yC(t,e,n,r,i){for(var o,a=t.children,s=-1,u=a.length,l=t.value&&(i-n)/t.value;++sh&&(h=s),m=c*c*g,(d=Math.max(h/m,m/f))>p){c-=s;break}p=d}y.push(a={value:c,dice:u1?e:1)},n}(vC);var bC=function t(e){function n(t,n,r,i,o){if((a=t._squarify)&&a.ratio===e)for(var a,s,u,l,c,f=-1,h=a.length,d=t.value;++f1?e:1)},n}(vC);function wC(t,e,n){const r={};return t.each((t=>{const i=t.data;n(i)&&(r[e(i)]=t)})),t.lookup=r,t}function kC(t){Ja.call(this,null,t)}kC.Definition={type:\"Nest\",metadata:{treesource:!0,changes:!0},params:[{name:\"keys\",type:\"field\",array:!0},{name:\"generate\",type:\"boolean\"}]};const AC=t=>t.values;function MC(){const t=[],e={entries:t=>r(n(t,0),0),key:n=>(t.push(n),e)};function n(e,r){if(r>=t.length)return e;const i=e.length,o=t[r++],a={},s={};let u,l,c,f=-1;for(;++ft.length)return e;const i=[];for(const t in e)i.push({key:t,values:r(e[t],n)});return i}return e}function EC(t){Ja.call(this,null,t)}dt(kC,Ja,{transform(t,e){e.source||s(\"Nest transform requires an upstream data source.\");var n=t.generate,r=t.modified(),i=e.clone(),o=this.value;return(!o||r||e.changed())&&(o&&o.each((t=>{t.children&&ma(t.data)&&i.rem.push(t.data)})),this.value=o=ED({values:V(t.keys).reduce(((t,e)=>(t.key(e),t)),MC()).entries(i.source)},AC),n&&o.each((t=>{t.children&&(t=_a(t.data),i.add.push(t),i.source.push(t))})),wC(o,ya,ya)),i.source.root=o,i}});const DC=(t,e)=>t.parent===e.parent?1:2;dt(EC,Ja,{transform(t,e){e.source&&e.source.root||s(this.constructor.name+\" transform requires a backing tree data source.\");const n=this.layout(t.method),r=this.fields,i=e.source.root,o=t.as||r;t.field?i.sum(t.field):i.count(),t.sort&&i.sort(ka(t.sort,(t=>t.data))),function(t,e,n){for(let r,i=0,o=e.length;ifunction(t,e,n){const r=t.data,i=e.length-1;for(let o=0;o(t=(OD*t+RD)%UD)/UD}();return i.x=e/2,i.y=n/2,t?i.eachBefore(QD(t)).eachAfter(KD(r,.5,o)).eachBefore(tC(1)):i.eachBefore(QD(ZD)).eachAfter(KD(zD,1,o)).eachAfter(KD(r,i.r/Math.min(e,n),o)).eachBefore(tC(Math.min(e,n)/(2*i.r))),i}return i.radius=function(e){return arguments.length?(t=TD(e),i):t},i.size=function(t){return arguments.length?(e=+t[0],n=+t[1],i):[e,n]},i.padding=function(t){return arguments.length?(r=\"function\"==typeof t?t:ND(+t),i):r},i},params:[\"radius\",\"size\",\"padding\"],fields:CC});const SC=[\"x0\",\"y0\",\"x1\",\"y1\",\"depth\",\"children\"];function $C(t){EC.call(this,t)}function TC(t){Ja.call(this,null,t)}$C.Definition={type:\"Partition\",metadata:{tree:!0,modifies:!0},params:[{name:\"field\",type:\"field\"},{name:\"sort\",type:\"compare\"},{name:\"padding\",type:\"number\",default:0},{name:\"round\",type:\"boolean\",default:!1},{name:\"size\",type:\"number\",array:!0,length:2},{name:\"as\",type:\"string\",array:!0,length:SC.length,default:SC}]},dt($C,EC,{layout:function(){var t=1,e=1,n=0,r=!1;function i(i){var o=i.height+1;return i.x0=i.y0=n,i.x1=t,i.y1=e/o,i.eachBefore(function(t,e){return function(r){r.children&&nC(r,r.x0,t*(r.depth+1)/e,r.x1,t*(r.depth+2)/e);var i=r.x0,o=r.y0,a=r.x1-n,s=r.y1-n;a=0;--i)s.push(n=e.children[i]=new mC(r[i],i)),n.parent=e;return(a.parent=new mC(null,0)).children=[a],a}(i);if(u.eachAfter(o),u.parent.m=-u.z,u.eachBefore(a),r)i.eachBefore(s);else{var l=i,c=i,f=i;i.eachBefore((function(t){t.xc.x&&(c=t),t.depth>f.depth&&(f=t)}));var h=l===c?1:t(l,c)/2,d=h-l.x,p=e/(c.x+h+d),g=n/(f.depth||1);i.eachBefore((function(t){t.x=(t.x+d)*p,t.y=t.depth*g}))}return i}function o(e){var n=e.children,r=e.parent.children,i=e.i?r[e.i-1]:null;if(n){!function(t){for(var e,n=0,r=0,i=t.children,o=i.length;--o>=0;)(e=i[o]).z+=n,e.m+=n,n+=e.s+(r+=e.c)}(e);var o=(n[0].z+n[n.length-1].z)/2;i?(e.z=i.z+t(e._,i._),e.m=e.z-o):e.z=o}else i&&(e.z=i.z+t(e._,i._));e.parent.A=function(e,n,r){if(n){for(var i,o=e,a=e,s=n,u=o.parent.children[0],l=o.m,c=a.m,f=s.m,h=u.m;s=dC(s),o=hC(o),s&&o;)u=hC(u),(a=dC(a)).a=e,(i=s.z+f-o.z-l+t(s._,o._))>0&&(pC(gC(s,e,r),e,i),l+=i,c+=i),f+=s.m,l+=o.m,h+=u.m,c+=a.m;s&&!dC(a)&&(a.t=s,a.m+=f-c),o&&!hC(u)&&(u.t=o,u.m+=l-h,r=e)}return r}(e,i,e.parent.A||r[0])}function a(t){t._.x=t.z+t.parent.m,t.m+=t.parent.m}function s(t){t.x*=e,t.y=t.depth*n}return i.separation=function(e){return arguments.length?(t=e,i):t},i.size=function(t){return arguments.length?(r=!1,e=+t[0],n=+t[1],i):r?null:[e,n]},i.nodeSize=function(t){return arguments.length?(r=!0,e=+t[0],n=+t[1],i):r?[e,n]:null},i},cluster:function(){var t=wD,e=1,n=1,r=!1;function i(i){var o,a=0;i.eachAfter((function(e){var n=e.children;n?(e.x=function(t){return t.reduce(kD,0)/t.length}(n),e.y=function(t){return 1+t.reduce(AD,0)}(n)):(e.x=o?a+=t(e,o):0,e.y=0,o=e)}));var s=function(t){for(var e;e=t.children;)t=e[0];return t}(i),u=function(t){for(var e;e=t.children;)t=e[e.length-1];return t}(i),l=s.x-t(s,u)/2,c=u.x+t(u,s)/2;return i.eachAfter(r?function(t){t.x=(t.x-i.x)*e,t.y=(i.y-t.y)*n}:function(t){t.x=(t.x-l)/(c-l)*e,t.y=(1-(i.y?t.y/i.y:1))*n})}return i.separation=function(e){return arguments.length?(t=e,i):t},i.size=function(t){return arguments.length?(r=!1,e=+t[0],n=+t[1],i):r?null:[e,n]},i.nodeSize=function(t){return arguments.length?(r=!0,e=+t[0],n=+t[1],i):r?[e,n]:null},i}},zC=[\"x\",\"y\",\"depth\",\"children\"];function NC(t){EC.call(this,t)}function OC(t){Ja.call(this,[],t)}NC.Definition={type:\"Tree\",metadata:{tree:!0,modifies:!0},params:[{name:\"field\",type:\"field\"},{name:\"sort\",type:\"compare\"},{name:\"method\",type:\"enum\",default:\"tidy\",values:[\"tidy\",\"cluster\"]},{name:\"size\",type:\"number\",array:!0,length:2},{name:\"nodeSize\",type:\"number\",array:!0,length:2},{name:\"separation\",type:\"boolean\",default:!0},{name:\"as\",type:\"string\",array:!0,length:zC.length,default:zC}]},dt(NC,EC,{layout(t){const e=t||\"tidy\";if(lt(BC,e))return BC[e]();s(\"Unrecognized Tree layout method: \"+e)},params:[\"size\",\"nodeSize\"],fields:zC}),OC.Definition={type:\"TreeLinks\",metadata:{tree:!0,generates:!0,changes:!0},params:[]},dt(OC,Ja,{transform(t,e){const n=this.value,r=e.source&&e.source.root,i=e.fork(e.NO_SOURCE),o={};return r||s(\"TreeLinks transform requires a tree data source.\"),e.changed(e.ADD_REM)?(i.rem=n,e.visit(e.SOURCE,(t=>o[ya(t)]=1)),r.each((t=>{const e=t.data,n=t.parent&&t.parent.data;n&&o[ya(e)]&&o[ya(n)]&&i.add.push(_a({source:n,target:e}))})),this.value=i.add):e.changed(e.MOD)&&(e.visit(e.MOD,(t=>o[ya(t)]=1)),n.forEach((t=>{(o[ya(t.source)]||o[ya(t.target)])&&i.mod.push(t)}))),i}});const RC={binary:function(t,e,n,r,i){var o,a,s=t.children,u=s.length,l=new Array(u+1);for(l[0]=a=o=0;o=n-1){var c=s[e];return c.x0=i,c.y0=o,c.x1=a,void(c.y1=u)}var f=l[e],h=r/2+f,d=e+1,p=n-1;for(;d>>1;l[g]u-o){var v=r?(i*y+a*m)/r:a;t(e,d,m,i,o,v,u),t(d,n,y,v,o,a,u)}else{var _=r?(o*y+u*m)/r:u;t(e,d,m,i,o,a,_),t(d,n,y,i,_,a,u)}}(0,u,t.value,e,n,r,i)},dice:nC,slice:yC,slicedice:function(t,e,n,r,i){(1&t.depth?yC:nC)(t,e,n,r,i)},squarify:xC,resquarify:bC},UC=[\"x0\",\"y0\",\"x1\",\"y1\",\"depth\",\"children\"];function LC(t){EC.call(this,t)}LC.Definition={type:\"Treemap\",metadata:{tree:!0,modifies:!0},params:[{name:\"field\",type:\"field\"},{name:\"sort\",type:\"compare\"},{name:\"method\",type:\"enum\",default:\"squarify\",values:[\"squarify\",\"resquarify\",\"binary\",\"dice\",\"slice\",\"slicedice\"]},{name:\"padding\",type:\"number\",default:0},{name:\"paddingInner\",type:\"number\",default:0},{name:\"paddingOuter\",type:\"number\",default:0},{name:\"paddingTop\",type:\"number\",default:0},{name:\"paddingRight\",type:\"number\",default:0},{name:\"paddingBottom\",type:\"number\",default:0},{name:\"paddingLeft\",type:\"number\",default:0},{name:\"ratio\",type:\"number\",default:1.618033988749895},{name:\"round\",type:\"boolean\",default:!1},{name:\"size\",type:\"number\",array:!0,length:2},{name:\"as\",type:\"string\",array:!0,length:UC.length,default:UC}]},dt(LC,EC,{layout(){const t=function(){var t=xC,e=!1,n=1,r=1,i=[0],o=zD,a=zD,s=zD,u=zD,l=zD;function c(t){return t.x0=t.y0=0,t.x1=n,t.y1=r,t.eachBefore(f),i=[0],e&&t.eachBefore(eC),t}function f(e){var n=i[e.depth],r=e.x0+n,c=e.y0+n,f=e.x1-n,h=e.y1-n;f{const n=t.tile();n.ratio&&t.tile(n.ratio(e))},t.method=e=>{lt(RC,e)?t.tile(RC[e]):s(\"Unrecognized Treemap layout method: \"+e)},t},params:[\"method\",\"ratio\",\"size\",\"round\",\"padding\",\"paddingInner\",\"paddingOuter\",\"paddingTop\",\"paddingRight\",\"paddingBottom\",\"paddingLeft\"],fields:UC});var qC=Object.freeze({__proto__:null,nest:kC,pack:FC,partition:$C,stratify:TC,tree:NC,treelinks:OC,treemap:LC});const PC=4278190080;function jC(t,e,n){return new Uint32Array(t.getImageData(0,0,e,n).data.buffer)}function IC(t,e,n){if(!e.length)return;const r=e[0].mark.marktype;\"group\"===r?e.forEach((e=>{e.items.forEach((e=>IC(t,e.items,n)))})):zy[r].draw(t,{items:n?e.map(WC):e})}function WC(t){const e=ba(t,{});return e.stroke&&0!==e.strokeOpacity||e.fill&&0!==e.fillOpacity?{...e,strokeOpacity:1,stroke:\"#000\",fillOpacity:0}:e}const HC=5,YC=31,GC=32,VC=new Uint32Array(GC+1),XC=new Uint32Array(GC+1);XC[0]=0,VC[0]=~XC[0];for(let t=1;t<=GC;++t)XC[t]=XC[t-1]<<1|1,VC[t]=~XC[t];function JC(t,e,n){const r=Math.max(1,Math.sqrt(t*e/1e6)),i=~~((t+2*n+r)/r),o=~~((e+2*n+r)/r),a=t=>~~((t+n)/r);return a.invert=t=>t*r-n,a.bitmap=()=>function(t,e){const n=new Uint32Array(~~((t*e+GC)/GC));function r(t,e){n[t]|=e}function i(t,e){n[t]&=e}return{array:n,get:(e,r)=>{const i=r*t+e;return n[i>>>HC]&1<<(i&YC)},set:(e,n)=>{const i=n*t+e;r(i>>>HC,1<<(i&YC))},clear:(e,n)=>{const r=n*t+e;i(r>>>HC,~(1<<(r&YC)))},getRange:(e,r,i,o)=>{let a,s,u,l,c=o;for(;c>=r;--c)if(a=c*t+e,s=c*t+i,u=a>>>HC,l=s>>>HC,u===l){if(n[u]&VC[a&YC]&XC[1+(s&YC)])return!0}else{if(n[u]&VC[a&YC])return!0;if(n[l]&XC[1+(s&YC)])return!0;for(let t=u+1;t{let a,s,u,l,c;for(;n<=o;++n)if(a=n*t+e,s=n*t+i,u=a>>>HC,l=s>>>HC,u===l)r(u,VC[a&YC]&XC[1+(s&YC)]);else for(r(u,VC[a&YC]),r(l,XC[1+(s&YC)]),c=u+1;c{let a,s,u,l,c;for(;n<=o;++n)if(a=n*t+e,s=n*t+r,u=a>>>HC,l=s>>>HC,u===l)i(u,XC[a&YC]|VC[1+(s&YC)]);else for(i(u,XC[a&YC]),i(l,VC[1+(s&YC)]),c=u+1;cn<0||r<0||o>=e||i>=t}}(i,o),a.ratio=r,a.padding=n,a.width=t,a.height=e,a}function ZC(t,e,n,r,i,o){let a=n/2;return t-a<0||t+a>i||e-(a=r/2)<0||e+a>o}function QC(t,e,n,r,i,o,a,s){const u=i*o/(2*r),l=t(e-u),c=t(e+u),f=t(n-(o/=2)),h=t(n+o);return a.outOfBounds(l,f,c,h)||a.getRange(l,f,c,h)||s&&s.getRange(l,f,c,h)}const KC=[-1,-1,1,1],tF=[-1,1,-1,1];const eF=[\"right\",\"center\",\"left\"],nF=[\"bottom\",\"middle\",\"top\"];function rF(t,e,n,r,i,o,a,s,u,l,c,f){return!(i.outOfBounds(t,n,e,r)||(f&&o||i).getRange(t,n,e,r))}const iF={\"top-left\":0,top:1,\"top-right\":2,left:4,middle:5,right:6,\"bottom-left\":8,bottom:9,\"bottom-right\":10},oF={naive:function(t,e,n,r){const i=t.width,o=t.height;return function(t){const e=t.datum.datum.items[r].items,n=e.length,a=t.datum.fontSize,s=py.width(t.datum,t.datum.text);let u,l,c,f,h,d,p,g=0;for(let r=0;r=g&&(g=p,t.x=h,t.y=d);return h=s/2,d=a/2,u=t.x-h,l=t.x+h,c=t.y-d,f=t.y+d,t.align=\"center\",u<0&&l<=i?t.align=\"left\":0<=u&&i=1;)h=(d+p)/2,QC(t,c,f,l,u,h,a,s)?p=h:d=h;if(d>r)return[c,f,d,!0]}}return function(e){const s=e.datum.datum.items[r].items,l=s.length,c=e.datum.fontSize,f=py.width(e.datum,e.datum.text);let h,d,p,g,m,y,v,_,x,b,w,k,A,M,E,D,C,F=n?c:0,S=!1,$=!1,T=0;for(let r=0;rd&&(C=h,h=d,d=C),p>g&&(C=p,p=g,g=C),x=t(h),w=t(d),b=~~((x+w)/2),k=t(p),M=t(g),A=~~((k+M)/2),v=b;v>=x;--v)for(_=A;_>=k;--_)D=u(v,_,F,f,c),D&&([e.x,e.y,F,S]=D);for(v=b;v<=w;++v)for(_=A;_<=M;++_)D=u(v,_,F,f,c),D&&([e.x,e.y,F,S]=D);S||n||(E=Math.abs(d-h+g-p),m=(h+d)/2,y=(p+g)/2,E>=T&&!ZC(m,y,f,c,i,o)&&!QC(t,m,y,c,f,c,a,null)&&(T=E,e.x=m,e.y=y,$=!0))}return!(!S&&!$)&&(m=f/2,y=c/2,a.setRange(t(e.x-m),t(e.y-y),t(e.x+m),t(e.y+y)),e.align=\"center\",e.baseline=\"middle\",!0)}},floodfill:function(t,e,n,r){const i=t.width,o=t.height,a=e[0],s=e[1],u=t.bitmap();return function(e){const l=e.datum.datum.items[r].items,c=l.length,f=e.datum.fontSize,h=py.width(e.datum,e.datum.text),d=[];let p,g,m,y,v,_,x,b,w,k,A,M,E=n?f:0,D=!1,C=!1,F=0;for(let r=0;r=1;)A=(w+k)/2,QC(t,v,_,f,h,A,a,s)?k=A:w=A;w>E&&(e.x=v,e.y=_,E=w,D=!0)}}D||n||(M=Math.abs(g-p+y-m),v=(p+g)/2,_=(m+y)/2,M>=F&&!ZC(v,_,h,f,i,o)&&!QC(t,v,_,f,h,f,a,null)&&(F=M,e.x=v,e.y=_,C=!0))}return!(!D&&!C)&&(v=h/2,_=f/2,a.setRange(t(e.x-v),t(e.y-_),t(e.x+v),t(e.y+_)),e.align=\"center\",e.baseline=\"middle\",!0)}}};function aF(t,e,n,r,i,o,a,s,u,l,c){if(!t.length)return t;const f=Math.max(r.length,i.length),h=function(t,e){const n=new Float64Array(e),r=t.length;for(let e=0;e[t.x,t.x,t.x,t.y,t.y,t.y];return t?\"line\"===t||\"area\"===t?t=>i(t.datum):\"line\"===e?t=>{const e=t.datum.items[r].items;return i(e.length?e[\"start\"===n?0:e.length-1]:{x:NaN,y:NaN})}:t=>{const e=t.datum.bounds;return[e.x1,(e.x1+e.x2)/2,e.x2,e.y1,(e.y1+e.y2)/2,e.y2]}:i}(p,g,s,u),v=null===l||l===1/0,_=m&&\"naive\"===c;var x;let b=-1,w=-1;const k=t.map((t=>{const e=v?py.width(t,t.text):void 0;return b=Math.max(b,e),w=Math.max(w,t.fontSize),{datum:t,opacity:0,x:void 0,y:void 0,align:void 0,baseline:void 0,boundary:y(t),textWidth:e}}));l=null===l||l===1/0?Math.max(b,w)+Math.max(...r):l;const A=JC(e[0],e[1],l);let M;if(!_){n&&k.sort(((t,e)=>n(t.datum,e.datum)));let e=!1;for(let t=0;tt.datum));M=o.length||r?function(t,e,n,r,i){const o=t.width,a=t.height,s=r||i,u=$c(o,a).getContext(\"2d\"),l=$c(o,a).getContext(\"2d\"),c=s&&$c(o,a).getContext(\"2d\");n.forEach((t=>IC(u,t,!1))),IC(l,e,!1),s&&IC(c,e,!0);const f=jC(u,o,a),h=jC(l,o,a),d=s&&jC(c,o,a),p=t.bitmap(),g=s&&t.bitmap();let m,y,v,_,x,b,w,k;for(y=0;yn.set(t(e.boundary[0]),t(e.boundary[3])))),[n,void 0]}(A,a&&k)}const E=m?oF[c](A,M,a,u):function(t,e,n,r){const i=t.width,o=t.height,a=e[0],s=e[1],u=r.length;return function(e){const l=e.boundary,c=e.datum.fontSize;if(l[2]<0||l[5]<0||l[0]>i||l[3]>o)return!1;let f,h,d,p,g,m,y,v,_,x,b,w,k,A,M,E=e.textWidth??0;for(let i=0;i>>2&3)-1,d=0===f&&0===h||r[i]<0,p=f&&h?Math.SQRT1_2:1,g=r[i]<0?-1:1,m=l[1+f]+r[i]*f*p,b=l[4+h]+g*c*h/2+r[i]*h*p,v=b-c/2,_=b+c/2,w=t(m),A=t(v),M=t(_),!E){if(!rF(w,w,A,M,a,s,0,0,0,0,0,d))continue;E=py.width(e.datum,e.datum.text)}if(x=m+g*E*f/2,m=x-E/2,y=x+E/2,w=t(m),k=t(y),rF(w,k,A,M,a,s,0,0,0,0,0,d))return e.x=f?f*g<0?y:m:x,e.y=h?h*g<0?_:v:b,e.align=eF[f*g+1],e.baseline=nF[h*g+1],a.setRange(w,A,k,M),!0}return!1}}(A,M,d,h);return k.forEach((t=>t.opacity=+E(t))),k}const sF=[\"x\",\"y\",\"opacity\",\"align\",\"baseline\"],uF=[\"top-left\",\"left\",\"bottom-left\",\"top\",\"bottom\",\"top-right\",\"right\",\"bottom-right\"];function lF(t){Ja.call(this,null,t)}lF.Definition={type:\"Label\",metadata:{modifies:!0},params:[{name:\"size\",type:\"number\",array:!0,length:2,required:!0},{name:\"sort\",type:\"compare\"},{name:\"anchor\",type:\"string\",array:!0,default:uF},{name:\"offset\",type:\"number\",array:!0,default:[1]},{name:\"padding\",type:\"number\",default:0,null:!0},{name:\"lineAnchor\",type:\"string\",values:[\"start\",\"end\"],default:\"end\"},{name:\"markIndex\",type:\"number\",default:0},{name:\"avoidBaseMark\",type:\"boolean\",default:!0},{name:\"avoidMarks\",type:\"data\",array:!0},{name:\"method\",type:\"string\",default:\"naive\"},{name:\"as\",type:\"string\",array:!0,length:sF.length,default:sF}]},dt(lF,Ja,{transform(t,e){const n=t.modified();if(!(n||e.changed(e.ADD_REM)||function(n){const r=t[n];return J(r)&&e.modified(r.fields)}(\"sort\")))return;t.size&&2===t.size.length||s(\"Size parameter should be specified as a [width, height] array.\");const r=t.as||sF;return aF(e.materialize(e.SOURCE).source||[],t.size,t.sort,V(null==t.offset?1:t.offset),V(t.anchor||uF),t.avoidMarks||[],!1!==t.avoidBaseMark,t.lineAnchor||\"end\",t.markIndex||0,void 0===t.padding?0:t.padding,t.method||\"naive\").forEach((t=>{const e=t.datum;e[r[0]]=t.x,e[r[1]]=t.y,e[r[2]]=t.opacity,e[r[3]]=t.align,e[r[4]]=t.baseline})),e.reflow(n).modifies(r)}});var cF=Object.freeze({__proto__:null,label:lF});function fF(t,e){var n,r,i,o,a,s,u=[],l=function(t){return t(o)};if(null==e)u.push(t);else for(n={},r=0,i=t.length;r{Ls(e,t.x,t.y,t.bandwidth||.3).forEach((t=>{const n={};for(let t=0;t\"poly\"===t?e:\"quad\"===t?2:1)(a,u),c=t.as||[n(t.x),n(t.y)],f=dF[a],h=[];let d=t.extent;lt(dF,a)||s(\"Invalid regression method: \"+a),null!=d&&\"log\"===a&&d[0]<=0&&(e.dataflow.warn(\"Ignoring extent with values <= 0 for log regression.\"),d=null),i.forEach((n=>{if(n.length<=l)return void e.dataflow.warn(\"Skipping regression with more parameters than data points.\");const r=f(n,t.x,t.y,u);if(t.params)return void h.push(_a({keys:n.dims,coef:r.coef,rSquared:r.rSquared}));const i=d||at(n,t.x),s=t=>{const e={};for(let t=0;ts([t,r.predict(t)]))):Is(r.predict,i,25,200).forEach(s)})),this.value&&(r.rem=this.value),this.value=r.add=r.source=h}return r}});var gF=Object.freeze({__proto__:null,loess:hF,regression:pF});const mF=134217729,yF=33306690738754706e-32;function vF(t,e,n,r,i){let o,a,s,u,l=e[0],c=r[0],f=0,h=0;c>l==c>-l?(o=l,l=e[++f]):(o=c,c=r[++h]);let d=0;if(fl==c>-l?(a=l+o,s=o-(a-l),l=e[++f]):(a=c+o,s=o-(a-c),c=r[++h]),o=a,0!==s&&(i[d++]=s);fl==c>-l?(a=o+l,u=a-o,s=o-(a-u)+(l-u),l=e[++f]):(a=o+c,u=a-o,s=o-(a-u)+(c-u),c=r[++h]),o=a,0!==s&&(i[d++]=s);for(;f0!=s>0)return u;const l=Math.abs(a+s);return Math.abs(u)>=33306690738754716e-32*l?u:-function(t,e,n,r,i,o,a){let s,u,l,c,f,h,d,p,g,m,y,v,_,x,b,w,k,A;const M=t-i,E=n-i,D=e-o,C=r-o;x=M*C,h=mF*M,d=h-(h-M),p=M-d,h=mF*C,g=h-(h-C),m=C-g,b=p*m-(x-d*g-p*g-d*m),w=D*E,h=mF*D,d=h-(h-D),p=D-d,h=mF*E,g=h-(h-E),m=E-g,k=p*m-(w-d*g-p*g-d*m),y=b-k,f=b-y,wF[0]=b-(y+f)+(f-k),v=x+y,f=v-x,_=x-(v-f)+(y-f),y=_-w,f=_-y,wF[1]=_-(y+f)+(f-w),A=v+y,f=A-v,wF[2]=v-(A-f)+(y-f),wF[3]=A;let F=function(t,e){let n=e[0];for(let r=1;r=S||-F>=S)return F;if(f=t-M,s=t-(M+f)+(f-i),f=n-E,l=n-(E+f)+(f-i),f=e-D,u=e-(D+f)+(f-o),f=r-C,c=r-(C+f)+(f-o),0===s&&0===u&&0===l&&0===c)return F;if(S=bF*a+yF*Math.abs(F),F+=M*c+C*s-(D*l+E*u),F>=S||-F>=S)return F;x=s*C,h=mF*s,d=h-(h-s),p=s-d,h=mF*C,g=h-(h-C),m=C-g,b=p*m-(x-d*g-p*g-d*m),w=u*E,h=mF*u,d=h-(h-u),p=u-d,h=mF*E,g=h-(h-E),m=E-g,k=p*m-(w-d*g-p*g-d*m),y=b-k,f=b-y,EF[0]=b-(y+f)+(f-k),v=x+y,f=v-x,_=x-(v-f)+(y-f),y=_-w,f=_-y,EF[1]=_-(y+f)+(f-w),A=v+y,f=A-v,EF[2]=v-(A-f)+(y-f),EF[3]=A;const $=vF(4,wF,4,EF,kF);x=M*c,h=mF*M,d=h-(h-M),p=M-d,h=mF*c,g=h-(h-c),m=c-g,b=p*m-(x-d*g-p*g-d*m),w=D*l,h=mF*D,d=h-(h-D),p=D-d,h=mF*l,g=h-(h-l),m=l-g,k=p*m-(w-d*g-p*g-d*m),y=b-k,f=b-y,EF[0]=b-(y+f)+(f-k),v=x+y,f=v-x,_=x-(v-f)+(y-f),y=_-w,f=_-y,EF[1]=_-(y+f)+(f-w),A=v+y,f=A-v,EF[2]=v-(A-f)+(y-f),EF[3]=A;const T=vF($,kF,4,EF,AF);x=s*c,h=mF*s,d=h-(h-s),p=s-d,h=mF*c,g=h-(h-c),m=c-g,b=p*m-(x-d*g-p*g-d*m),w=u*l,h=mF*u,d=h-(h-u),p=u-d,h=mF*l,g=h-(h-l),m=l-g,k=p*m-(w-d*g-p*g-d*m),y=b-k,f=b-y,EF[0]=b-(y+f)+(f-k),v=x+y,f=v-x,_=x-(v-f)+(y-f),y=_-w,f=_-y,EF[1]=_-(y+f)+(f-w),A=v+y,f=A-v,EF[2]=v-(A-f)+(y-f),EF[3]=A;const B=vF(T,AF,4,EF,MF);return MF[B-1]}(t,e,n,r,i,o,l)}const CF=Math.pow(2,-52),FF=new Uint32Array(512);class SF{static from(t){let e=arguments.length>1&&void 0!==arguments[1]?arguments[1]:OF,n=arguments.length>2&&void 0!==arguments[2]?arguments[2]:RF;const r=t.length,i=new Float64Array(2*r);for(let o=0;o>1;if(e>0&&\"number\"!=typeof t[0])throw new Error(\"Expected coords to contain numbers.\");this.coords=t;const n=Math.max(2*e-5,0);this._triangles=new Uint32Array(3*n),this._halfedges=new Int32Array(3*n),this._hashSize=Math.ceil(Math.sqrt(e)),this._hullPrev=new Uint32Array(e),this._hullNext=new Uint32Array(e),this._hullTri=new Uint32Array(e),this._hullHash=new Int32Array(this._hashSize).fill(-1),this._ids=new Uint32Array(e),this._dists=new Float64Array(e),this.update()}update(){const{coords:t,_hullPrev:e,_hullNext:n,_hullTri:r,_hullHash:i}=this,o=t.length>>1;let a=1/0,s=1/0,u=-1/0,l=-1/0;for(let e=0;eu&&(u=n),r>l&&(l=r),this._ids[e]=e}const c=(a+u)/2,f=(s+l)/2;let h,d,p,g=1/0;for(let e=0;e0&&(d=e,g=n)}let v=t[2*d],_=t[2*d+1],x=1/0;for(let e=0;er&&(e[n++]=i,r=this._dists[i])}return this.hull=e.subarray(0,n),this.triangles=new Uint32Array(0),void(this.halfedges=new Uint32Array(0))}if(DF(m,y,v,_,b,w)<0){const t=d,e=v,n=_;d=p,v=b,_=w,p=t,b=e,w=n}const k=function(t,e,n,r,i,o){const a=n-t,s=r-e,u=i-t,l=o-e,c=a*a+s*s,f=u*u+l*l,h=.5/(a*l-s*u),d=t+(l*c-s*f)*h,p=e+(a*f-u*c)*h;return{x:d,y:p}}(m,y,v,_,b,w);this._cx=k.x,this._cy=k.y;for(let e=0;e0&&Math.abs(l-o)<=CF&&Math.abs(c-a)<=CF)continue;if(o=l,a=c,u===h||u===d||u===p)continue;let f=0;for(let t=0,e=this._hashKey(l,c);t=0;)if(m=g,m===f){m=-1;break}if(-1===m)continue;let y=this._addTriangle(m,u,n[m],-1,-1,r[m]);r[u]=this._legalize(y+2),r[m]=y,A++;let v=n[m];for(;g=n[v],DF(l,c,t[2*v],t[2*v+1],t[2*g],t[2*g+1])<0;)y=this._addTriangle(v,u,g,r[u],-1,r[v]),r[u]=this._legalize(y+2),n[v]=v,A--,v=g;if(m===f)for(;g=e[m],DF(l,c,t[2*g],t[2*g+1],t[2*m],t[2*m+1])<0;)y=this._addTriangle(g,u,m,-1,r[m],r[g]),this._legalize(y+2),r[g]=y,n[m]=m,A--,m=g;this._hullStart=e[u]=m,n[m]=e[v]=u,n[u]=v,i[this._hashKey(l,c)]=u,i[this._hashKey(t[2*m],t[2*m+1])]=m}this.hull=new Uint32Array(A);for(let t=0,e=this._hullStart;t0?3-n:1+n)/4}(t-this._cx,e-this._cy)*this._hashSize)%this._hashSize}_legalize(t){const{_triangles:e,_halfedges:n,coords:r}=this;let i=0,o=0;for(;;){const a=n[t],s=t-t%3;if(o=s+(t+2)%3,-1===a){if(0===i)break;t=FF[--i];continue}const u=a-a%3,l=s+(t+1)%3,c=u+(a+2)%3,f=e[o],h=e[t],d=e[l],p=e[c];if(TF(r[2*f],r[2*f+1],r[2*h],r[2*h+1],r[2*d],r[2*d+1],r[2*p],r[2*p+1])){e[t]=p,e[a]=f;const r=n[c];if(-1===r){let e=this._hullStart;do{if(this._hullTri[e]===c){this._hullTri[e]=t;break}e=this._hullPrev[e]}while(e!==this._hullStart)}this._link(t,r),this._link(a,n[o]),this._link(o,c);const s=u+(a+1)%3;i=n&&e[t[a]]>o;)t[a+1]=t[a--];t[a+1]=r}else{let i=n+1,o=r;NF(t,n+r>>1,i),e[t[n]]>e[t[r]]&&NF(t,n,r),e[t[i]]>e[t[r]]&&NF(t,i,r),e[t[n]]>e[t[i]]&&NF(t,n,i);const a=t[i],s=e[a];for(;;){do{i++}while(e[t[i]]s);if(o=o-n?(zF(t,e,i,r),zF(t,e,n,o-1)):(zF(t,e,n,o-1),zF(t,e,i,r))}}function NF(t,e,n){const r=t[e];t[e]=t[n],t[n]=r}function OF(t){return t[0]}function RF(t){return t[1]}const UF=1e-6;class LF{constructor(){this._x0=this._y0=this._x1=this._y1=null,this._=\"\"}moveTo(t,e){this._+=`M${this._x0=this._x1=+t},${this._y0=this._y1=+e}`}closePath(){null!==this._x1&&(this._x1=this._x0,this._y1=this._y0,this._+=\"Z\")}lineTo(t,e){this._+=`L${this._x1=+t},${this._y1=+e}`}arc(t,e,n){const r=(t=+t)+(n=+n),i=e=+e;if(n<0)throw new Error(\"negative radius\");null===this._x1?this._+=`M${r},${i}`:(Math.abs(this._x1-r)>UF||Math.abs(this._y1-i)>UF)&&(this._+=\"L\"+r+\",\"+i),n&&(this._+=`A${n},${n},0,1,1,${t-n},${e}A${n},${n},0,1,1,${this._x1=r},${this._y1=i}`)}rect(t,e,n,r){this._+=`M${this._x0=this._x1=+t},${this._y0=this._y1=+e}h${+n}v${+r}h${-n}Z`}value(){return this._||null}}class qF{constructor(){this._=[]}moveTo(t,e){this._.push([t,e])}closePath(){this._.push(this._[0].slice())}lineTo(t,e){this._.push([t,e])}value(){return this._.length?this._:null}}let PF=class{constructor(t){let[e,n,r,i]=arguments.length>1&&void 0!==arguments[1]?arguments[1]:[0,0,960,500];if(!((r=+r)>=(e=+e)&&(i=+i)>=(n=+n)))throw new Error(\"invalid bounds\");this.delaunay=t,this._circumcenters=new Float64Array(2*t.points.length),this.vectors=new Float64Array(2*t.points.length),this.xmax=r,this.xmin=e,this.ymax=i,this.ymin=n,this._init()}update(){return this.delaunay.update(),this._init(),this}_init(){const{delaunay:{points:t,hull:e,triangles:n},vectors:r}=this;let i,o;const a=this.circumcenters=this._circumcenters.subarray(0,n.length/3*2);for(let r,s,u=0,l=0,c=n.length;u1;)i-=2;for(let t=2;t0){if(e>=this.ymax)return null;(i=(this.ymax-e)/r)0){if(t>=this.xmax)return null;(i=(this.xmax-t)/n)this.xmax?2:0)|(ethis.ymax?8:0)}_simplify(t){if(t&&t.length>4){for(let e=0;e1&&void 0!==arguments[1]?arguments[1]:WF,n=arguments.length>2&&void 0!==arguments[2]?arguments[2]:HF,r=arguments.length>3?arguments[3]:void 0;return new GF(\"length\"in t?function(t,e,n,r){const i=t.length,o=new Float64Array(2*i);for(let a=0;a2&&function(t){const{triangles:e,coords:n}=t;for(let t=0;t1e-10)return!1}return!0}(t)){this.collinear=Int32Array.from({length:e.length/2},((t,e)=>e)).sort(((t,n)=>e[2*t]-e[2*n]||e[2*t+1]-e[2*n+1]));const t=this.collinear[0],n=this.collinear[this.collinear.length-1],r=[e[2*t],e[2*t+1],e[2*n],e[2*n+1]],i=1e-8*Math.hypot(r[3]-r[1],r[2]-r[0]);for(let t=0,n=e.length/2;t0&&(this.triangles=new Int32Array(3).fill(-1),this.halfedges=new Int32Array(3).fill(-1),this.triangles[0]=r[0],o[r[0]]=1,2===r.length&&(o[r[1]]=0,this.triangles[1]=r[1],this.triangles[2]=r[1]))}voronoi(t){return new PF(this,t)}*neighbors(t){const{inedges:e,hull:n,_hullIndex:r,halfedges:i,triangles:o,collinear:a}=this;if(a){const e=a.indexOf(t);return e>0&&(yield a[e-1]),void(e2&&void 0!==arguments[2]?arguments[2]:0;if((t=+t)!=t||(e=+e)!=e)return-1;const r=n;let i;for(;(i=this._step(n,t,e))>=0&&i!==n&&i!==r;)n=i;return i}_step(t,e,n){const{inedges:r,hull:i,_hullIndex:o,halfedges:a,triangles:s,points:u}=this;if(-1===r[t]||!u.length)return(t+1)%(u.length>>1);let l=t,c=IF(e-u[2*t],2)+IF(n-u[2*t+1],2);const f=r[t];let h=f;do{let r=s[h];const f=IF(e-u[2*r],2)+IF(n-u[2*r+1],2);if(f=f));)if(e.x=a+i,e.y=l+o,!(e.x+e.x0<0||e.y+e.y0<0||e.x+e.x1>s[0]||e.y+e.y1>s[1])&&(!n||!rS(e,t,s[0]))&&(!n||oS(e,n))){for(var g,m=e.sprite,y=e.width>>5,v=s[0]>>5,_=e.x-(y<<4),x=127&_,b=32-x,w=e.y1-e.y0,k=(e.y+e.y0)*v+(_>>5),A=0;A>>x:0);k+=v}return e.sprite=null,!0}return!1}return f.layout=function(){for(var u=function(t){t.width=t.height=1;var e=Math.sqrt(t.getContext(\"2d\").getImageData(0,0,1,1).data.length>>2);t.width=(KF<<5)/e,t.height=tS/e;var n=t.getContext(\"2d\");return n.fillStyle=n.strokeStyle=\"red\",n.textAlign=\"center\",{context:n,ratio:e}}($c()),f=function(t){var e=[],n=-1;for(;++n>5)*s[1]),d=null,p=l.length,g=-1,m=[],y=l.map((s=>({text:t(s),font:e(s),style:r(s),weight:i(s),rotate:o(s),size:~~(n(s)+1e-14),padding:a(s),xoff:0,yoff:0,x1:0,y1:0,x0:0,y0:0,hasText:!1,sprite:null,datum:s}))).sort(((t,e)=>e.size-t.size));++g>1,v.y=s[1]*(c()+.5)>>1,nS(u,v,y,g),v.hasText&&h(f,v,d)&&(m.push(v),d?iS(d,v):d=[{x:v.x+v.x0,y:v.y+v.y0},{x:v.x+v.x1,y:v.y+v.y1}],v.x-=s[0]>>1,v.y-=s[1]>>1)}return m},f.words=function(t){return arguments.length?(l=t,f):l},f.size=function(t){return arguments.length?(s=[+t[0],+t[1]],f):s},f.font=function(t){return arguments.length?(e=sS(t),f):e},f.fontStyle=function(t){return arguments.length?(r=sS(t),f):r},f.fontWeight=function(t){return arguments.length?(i=sS(t),f):i},f.rotate=function(t){return arguments.length?(o=sS(t),f):o},f.text=function(e){return arguments.length?(t=sS(e),f):t},f.spiral=function(t){return arguments.length?(u=uS[t]||t,f):u},f.fontSize=function(t){return arguments.length?(n=sS(t),f):n},f.padding=function(t){return arguments.length?(a=sS(t),f):a},f.random=function(t){return arguments.length?(c=t,f):c},f}function nS(t,e,n,r){if(!e.sprite){var i=t.context,o=t.ratio;i.clearRect(0,0,(KF<<5)/o,tS/o);var a,s,u,l,c,f=0,h=0,d=0,p=n.length;for(--r;++r>5<<5,u=~~Math.max(Math.abs(v+_),Math.abs(v-_))}else a=a+31>>5<<5;if(u>d&&(d=u),f+a>=KF<<5&&(f=0,h+=d,d=0),h+u>=tS)break;i.translate((f+(a>>1))/o,(h+(u>>1))/o),e.rotate&&i.rotate(e.rotate*QF),i.fillText(e.text,0,0),e.padding&&(i.lineWidth=2*e.padding,i.strokeText(e.text,0,0)),i.restore(),e.width=a,e.height=u,e.xoff=f,e.yoff=h,e.x1=a>>1,e.y1=u>>1,e.x0=-e.x1,e.y0=-e.y1,e.hasText=!0,f+=a}for(var b=i.getImageData(0,0,(KF<<5)/o,tS/o).data,w=[];--r>=0;)if((e=n[r]).hasText){for(s=(a=e.width)>>5,u=e.y1-e.y0,l=0;l>5),E=b[(h+c)*(KF<<5)+(f+l)<<2]?1<<31-l%32:0;w[M]|=E,k|=E}k?A=c:(e.y0++,u--,c--,h++)}e.y1=e.y0+A,e.sprite=w.slice(0,(e.y1-e.y0)*s)}}}function rS(t,e,n){n>>=5;for(var r,i=t.sprite,o=t.width>>5,a=t.x-(o<<4),s=127&a,u=32-s,l=t.y1-t.y0,c=(t.y+t.y0)*n+(a>>5),f=0;f>>s:0))&e[c+h])return!0;c+=n}return!1}function iS(t,e){var n=t[0],r=t[1];e.x+e.x0r.x&&(r.x=e.x+e.x1),e.y+e.y1>r.y&&(r.y=e.y+e.y1)}function oS(t,e){return t.x+t.x1>e[0].x&&t.x+t.x0e[0].y&&t.y+t.y0e(t(n))}i.forEach((t=>{t[a[0]]=NaN,t[a[1]]=NaN,t[a[3]]=0}));const c=o.words(i).text(e.text).size(e.size||[500,500]).padding(e.padding||1).spiral(e.spiral||\"archimedean\").rotate(e.rotate||0).font(e.font||\"sans-serif\").fontStyle(e.fontStyle||\"normal\").fontWeight(e.fontWeight||\"normal\").fontSize(l).random(t.random).layout(),f=o.size(),h=f[0]>>1,d=f[1]>>1,p=c.length;for(let t,e,n=0;nnew Uint8Array(t),pS=t=>new Uint16Array(t),gS=t=>new Uint32Array(t);function mS(t,e,n){const r=(e<257?dS:e<65537?pS:gS)(t);return n&&r.set(n),r}function yS(t,e,n){const r=1<{const r=t[e],i=t[n];return ri?1:0})),function(t,e){return Array.from(e,(e=>t[e]))}(t,e)}(h,u),a)l=e,c=t,e=Array(a+s),t=gS(a+s),function(t,e,n,r,i,o,a,s,u){let l,c=0,f=0;for(l=0;c0)for(f=0;ft,size:()=>n}}function _S(t){Ja.call(this,function(){let t=8,e=[],n=gS(0),r=mS(0,t),i=mS(0,t);return{data:()=>e,seen:()=>n=function(t,e,n){return t.length>=e?t:((n=n||new t.constructor(e)).set(t),n)}(n,e.length),add(t){for(let n,r=0,i=e.length,o=t.length;re.length,curr:()=>r,prev:()=>i,reset:t=>i[t]=r[t],all:()=>t<257?255:t<65537?65535:4294967295,set(t,e){r[t]|=e},clear(t,e){r[t]&=~e},resize(e,n){(e>r.length||n>t)&&(t=Math.max(n,t),r=mS(e,t,r),i=mS(e,t))}}}(),t),this._indices=null,this._dims=null}function xS(t){Ja.call(this,null,t)}_S.Definition={type:\"CrossFilter\",metadata:{},params:[{name:\"fields\",type:\"field\",array:!0,required:!0},{name:\"query\",type:\"array\",array:!0,required:!0,content:{type:\"number\",array:!0,length:2}}]},dt(_S,Ja,{transform(t,e){return this._dims?t.modified(\"fields\")||t.fields.some((t=>e.modified(t.fields)))?this.reinit(t,e):this.eval(t,e):this.init(t,e)},init(t,e){const n=t.fields,r=t.query,i=this._indices={},o=this._dims=[],a=r.length;let s,u,l=0;for(;l{const t=i.remove(e,n);for(const e in r)r[e].reindex(t)}))},update(t,e,n){const r=this._dims,i=t.query,o=e.stamp,a=r.length;let s,u,l=0;for(n.filters=0,u=0;ud)for(m=d,y=Math.min(f,p);mp)for(m=Math.max(f,p),y=h;mc)for(d=c,p=Math.min(u,f);df)for(d=Math.max(u,f),p=l;ds[t]&n?null:a[t];return o.filter(o.MOD,l),i&i-1?(o.filter(o.ADD,(t=>{const e=s[t]&n;return!e&&e^u[t]&n?a[t]:null})),o.filter(o.REM,(t=>{const e=s[t]&n;return e&&!(e^e^u[t]&n)?a[t]:null}))):(o.filter(o.ADD,l),o.filter(o.REM,(t=>(s[t]&n)===i?a[t]:null))),o.filter(o.SOURCE,(t=>l(t._index)))}});var bS=Object.freeze({__proto__:null,crossfilter:_S,resolvefilter:xS});const wS=\"Literal\",kS=\"Property\",AS=\"ArrayExpression\",MS=\"BinaryExpression\",ES=\"CallExpression\",DS=\"ConditionalExpression\",CS=\"LogicalExpression\",FS=\"MemberExpression\",SS=\"ObjectExpression\",$S=\"UnaryExpression\";function TS(t){this.type=t}var BS,zS,NS,OS,RS;TS.prototype.visit=function(t){let e,n,r;if(t(this))return 1;for(e=function(t){switch(t.type){case AS:return t.elements;case MS:case CS:return[t.left,t.right];case ES:return[t.callee].concat(t.arguments);case DS:return[t.test,t.consequent,t.alternate];case FS:return[t.object,t.property];case SS:return t.properties;case kS:return[t.key,t.value];case $S:return[t.argument];default:return[]}}(this),n=0,r=e.length;n\",BS[qS]=\"Identifier\",BS[PS]=\"Keyword\",BS[jS]=\"Null\",BS[IS]=\"Numeric\",BS[WS]=\"Punctuator\",BS[HS]=\"String\",BS[9]=\"RegularExpression\";var YS=\"ArrayExpression\",GS=\"BinaryExpression\",VS=\"CallExpression\",XS=\"ConditionalExpression\",JS=\"Identifier\",ZS=\"Literal\",QS=\"LogicalExpression\",KS=\"MemberExpression\",t$=\"ObjectExpression\",e$=\"Property\",n$=\"UnaryExpression\",r$=\"Unexpected token %0\",i$=\"Unexpected number\",o$=\"Unexpected string\",a$=\"Unexpected identifier\",s$=\"Unexpected reserved word\",u$=\"Unexpected end of input\",l$=\"Invalid regular expression\",c$=\"Invalid regular expression: missing /\",f$=\"Octal literals are not allowed in strict mode.\",h$=\"Duplicate data property in object literal not allowed in strict mode\",d$=\"ILLEGAL\",p$=\"Disabled.\",g$=new RegExp(\"[\\\\xAA\\\\xB5\\\\xBA\\\\xC0-\\\\xD6\\\\xD8-\\\\xF6\\\\xF8-\\\\u02C1\\\\u02C6-\\\\u02D1\\\\u02E0-\\\\u02E4\\\\u02EC\\\\u02EE\\\\u0370-\\\\u0374\\\\u0376\\\\u0377\\\\u037A-\\\\u037D\\\\u037F\\\\u0386\\\\u0388-\\\\u038A\\\\u038C\\\\u038E-\\\\u03A1\\\\u03A3-\\\\u03F5\\\\u03F7-\\\\u0481\\\\u048A-\\\\u052F\\\\u0531-\\\\u0556\\\\u0559\\\\u0561-\\\\u0587\\\\u05D0-\\\\u05EA\\\\u05F0-\\\\u05F2\\\\u0620-\\\\u064A\\\\u066E\\\\u066F\\\\u0671-\\\\u06D3\\\\u06D5\\\\u06E5\\\\u06E6\\\\u06EE\\\\u06EF\\\\u06FA-\\\\u06FC\\\\u06FF\\\\u0710\\\\u0712-\\\\u072F\\\\u074D-\\\\u07A5\\\\u07B1\\\\u07CA-\\\\u07EA\\\\u07F4\\\\u07F5\\\\u07FA\\\\u0800-\\\\u0815\\\\u081A\\\\u0824\\\\u0828\\\\u0840-\\\\u0858\\\\u08A0-\\\\u08B2\\\\u0904-\\\\u0939\\\\u093D\\\\u0950\\\\u0958-\\\\u0961\\\\u0971-\\\\u0980\\\\u0985-\\\\u098C\\\\u098F\\\\u0990\\\\u0993-\\\\u09A8\\\\u09AA-\\\\u09B0\\\\u09B2\\\\u09B6-\\\\u09B9\\\\u09BD\\\\u09CE\\\\u09DC\\\\u09DD\\\\u09DF-\\\\u09E1\\\\u09F0\\\\u09F1\\\\u0A05-\\\\u0A0A\\\\u0A0F\\\\u0A10\\\\u0A13-\\\\u0A28\\\\u0A2A-\\\\u0A30\\\\u0A32\\\\u0A33\\\\u0A35\\\\u0A36\\\\u0A38\\\\u0A39\\\\u0A59-\\\\u0A5C\\\\u0A5E\\\\u0A72-\\\\u0A74\\\\u0A85-\\\\u0A8D\\\\u0A8F-\\\\u0A91\\\\u0A93-\\\\u0AA8\\\\u0AAA-\\\\u0AB0\\\\u0AB2\\\\u0AB3\\\\u0AB5-\\\\u0AB9\\\\u0ABD\\\\u0AD0\\\\u0AE0\\\\u0AE1\\\\u0B05-\\\\u0B0C\\\\u0B0F\\\\u0B10\\\\u0B13-\\\\u0B28\\\\u0B2A-\\\\u0B30\\\\u0B32\\\\u0B33\\\\u0B35-\\\\u0B39\\\\u0B3D\\\\u0B5C\\\\u0B5D\\\\u0B5F-\\\\u0B61\\\\u0B71\\\\u0B83\\\\u0B85-\\\\u0B8A\\\\u0B8E-\\\\u0B90\\\\u0B92-\\\\u0B95\\\\u0B99\\\\u0B9A\\\\u0B9C\\\\u0B9E\\\\u0B9F\\\\u0BA3\\\\u0BA4\\\\u0BA8-\\\\u0BAA\\\\u0BAE-\\\\u0BB9\\\\u0BD0\\\\u0C05-\\\\u0C0C\\\\u0C0E-\\\\u0C10\\\\u0C12-\\\\u0C28\\\\u0C2A-\\\\u0C39\\\\u0C3D\\\\u0C58\\\\u0C59\\\\u0C60\\\\u0C61\\\\u0C85-\\\\u0C8C\\\\u0C8E-\\\\u0C90\\\\u0C92-\\\\u0CA8\\\\u0CAA-\\\\u0CB3\\\\u0CB5-\\\\u0CB9\\\\u0CBD\\\\u0CDE\\\\u0CE0\\\\u0CE1\\\\u0CF1\\\\u0CF2\\\\u0D05-\\\\u0D0C\\\\u0D0E-\\\\u0D10\\\\u0D12-\\\\u0D3A\\\\u0D3D\\\\u0D4E\\\\u0D60\\\\u0D61\\\\u0D7A-\\\\u0D7F\\\\u0D85-\\\\u0D96\\\\u0D9A-\\\\u0DB1\\\\u0DB3-\\\\u0DBB\\\\u0DBD\\\\u0DC0-\\\\u0DC6\\\\u0E01-\\\\u0E30\\\\u0E32\\\\u0E33\\\\u0E40-\\\\u0E46\\\\u0E81\\\\u0E82\\\\u0E84\\\\u0E87\\\\u0E88\\\\u0E8A\\\\u0E8D\\\\u0E94-\\\\u0E97\\\\u0E99-\\\\u0E9F\\\\u0EA1-\\\\u0EA3\\\\u0EA5\\\\u0EA7\\\\u0EAA\\\\u0EAB\\\\u0EAD-\\\\u0EB0\\\\u0EB2\\\\u0EB3\\\\u0EBD\\\\u0EC0-\\\\u0EC4\\\\u0EC6\\\\u0EDC-\\\\u0EDF\\\\u0F00\\\\u0F40-\\\\u0F47\\\\u0F49-\\\\u0F6C\\\\u0F88-\\\\u0F8C\\\\u1000-\\\\u102A\\\\u103F\\\\u1050-\\\\u1055\\\\u105A-\\\\u105D\\\\u1061\\\\u1065\\\\u1066\\\\u106E-\\\\u1070\\\\u1075-\\\\u1081\\\\u108E\\\\u10A0-\\\\u10C5\\\\u10C7\\\\u10CD\\\\u10D0-\\\\u10FA\\\\u10FC-\\\\u1248\\\\u124A-\\\\u124D\\\\u1250-\\\\u1256\\\\u1258\\\\u125A-\\\\u125D\\\\u1260-\\\\u1288\\\\u128A-\\\\u128D\\\\u1290-\\\\u12B0\\\\u12B2-\\\\u12B5\\\\u12B8-\\\\u12BE\\\\u12C0\\\\u12C2-\\\\u12C5\\\\u12C8-\\\\u12D6\\\\u12D8-\\\\u1310\\\\u1312-\\\\u1315\\\\u1318-\\\\u135A\\\\u1380-\\\\u138F\\\\u13A0-\\\\u13F4\\\\u1401-\\\\u166C\\\\u166F-\\\\u167F\\\\u1681-\\\\u169A\\\\u16A0-\\\\u16EA\\\\u16EE-\\\\u16F8\\\\u1700-\\\\u170C\\\\u170E-\\\\u1711\\\\u1720-\\\\u1731\\\\u1740-\\\\u1751\\\\u1760-\\\\u176C\\\\u176E-\\\\u1770\\\\u1780-\\\\u17B3\\\\u17D7\\\\u17DC\\\\u1820-\\\\u1877\\\\u1880-\\\\u18A8\\\\u18AA\\\\u18B0-\\\\u18F5\\\\u1900-\\\\u191E\\\\u1950-\\\\u196D\\\\u1970-\\\\u1974\\\\u1980-\\\\u19AB\\\\u19C1-\\\\u19C7\\\\u1A00-\\\\u1A16\\\\u1A20-\\\\u1A54\\\\u1AA7\\\\u1B05-\\\\u1B33\\\\u1B45-\\\\u1B4B\\\\u1B83-\\\\u1BA0\\\\u1BAE\\\\u1BAF\\\\u1BBA-\\\\u1BE5\\\\u1C00-\\\\u1C23\\\\u1C4D-\\\\u1C4F\\\\u1C5A-\\\\u1C7D\\\\u1CE9-\\\\u1CEC\\\\u1CEE-\\\\u1CF1\\\\u1CF5\\\\u1CF6\\\\u1D00-\\\\u1DBF\\\\u1E00-\\\\u1F15\\\\u1F18-\\\\u1F1D\\\\u1F20-\\\\u1F45\\\\u1F48-\\\\u1F4D\\\\u1F50-\\\\u1F57\\\\u1F59\\\\u1F5B\\\\u1F5D\\\\u1F5F-\\\\u1F7D\\\\u1F80-\\\\u1FB4\\\\u1FB6-\\\\u1FBC\\\\u1FBE\\\\u1FC2-\\\\u1FC4\\\\u1FC6-\\\\u1FCC\\\\u1FD0-\\\\u1FD3\\\\u1FD6-\\\\u1FDB\\\\u1FE0-\\\\u1FEC\\\\u1FF2-\\\\u1FF4\\\\u1FF6-\\\\u1FFC\\\\u2071\\\\u207F\\\\u2090-\\\\u209C\\\\u2102\\\\u2107\\\\u210A-\\\\u2113\\\\u2115\\\\u2119-\\\\u211D\\\\u2124\\\\u2126\\\\u2128\\\\u212A-\\\\u212D\\\\u212F-\\\\u2139\\\\u213C-\\\\u213F\\\\u2145-\\\\u2149\\\\u214E\\\\u2160-\\\\u2188\\\\u2C00-\\\\u2C2E\\\\u2C30-\\\\u2C5E\\\\u2C60-\\\\u2CE4\\\\u2CEB-\\\\u2CEE\\\\u2CF2\\\\u2CF3\\\\u2D00-\\\\u2D25\\\\u2D27\\\\u2D2D\\\\u2D30-\\\\u2D67\\\\u2D6F\\\\u2D80-\\\\u2D96\\\\u2DA0-\\\\u2DA6\\\\u2DA8-\\\\u2DAE\\\\u2DB0-\\\\u2DB6\\\\u2DB8-\\\\u2DBE\\\\u2DC0-\\\\u2DC6\\\\u2DC8-\\\\u2DCE\\\\u2DD0-\\\\u2DD6\\\\u2DD8-\\\\u2DDE\\\\u2E2F\\\\u3005-\\\\u3007\\\\u3021-\\\\u3029\\\\u3031-\\\\u3035\\\\u3038-\\\\u303C\\\\u3041-\\\\u3096\\\\u309D-\\\\u309F\\\\u30A1-\\\\u30FA\\\\u30FC-\\\\u30FF\\\\u3105-\\\\u312D\\\\u3131-\\\\u318E\\\\u31A0-\\\\u31BA\\\\u31F0-\\\\u31FF\\\\u3400-\\\\u4DB5\\\\u4E00-\\\\u9FCC\\\\uA000-\\\\uA48C\\\\uA4D0-\\\\uA4FD\\\\uA500-\\\\uA60C\\\\uA610-\\\\uA61F\\\\uA62A\\\\uA62B\\\\uA640-\\\\uA66E\\\\uA67F-\\\\uA69D\\\\uA6A0-\\\\uA6EF\\\\uA717-\\\\uA71F\\\\uA722-\\\\uA788\\\\uA78B-\\\\uA78E\\\\uA790-\\\\uA7AD\\\\uA7B0\\\\uA7B1\\\\uA7F7-\\\\uA801\\\\uA803-\\\\uA805\\\\uA807-\\\\uA80A\\\\uA80C-\\\\uA822\\\\uA840-\\\\uA873\\\\uA882-\\\\uA8B3\\\\uA8F2-\\\\uA8F7\\\\uA8FB\\\\uA90A-\\\\uA925\\\\uA930-\\\\uA946\\\\uA960-\\\\uA97C\\\\uA984-\\\\uA9B2\\\\uA9CF\\\\uA9E0-\\\\uA9E4\\\\uA9E6-\\\\uA9EF\\\\uA9FA-\\\\uA9FE\\\\uAA00-\\\\uAA28\\\\uAA40-\\\\uAA42\\\\uAA44-\\\\uAA4B\\\\uAA60-\\\\uAA76\\\\uAA7A\\\\uAA7E-\\\\uAAAF\\\\uAAB1\\\\uAAB5\\\\uAAB6\\\\uAAB9-\\\\uAABD\\\\uAAC0\\\\uAAC2\\\\uAADB-\\\\uAADD\\\\uAAE0-\\\\uAAEA\\\\uAAF2-\\\\uAAF4\\\\uAB01-\\\\uAB06\\\\uAB09-\\\\uAB0E\\\\uAB11-\\\\uAB16\\\\uAB20-\\\\uAB26\\\\uAB28-\\\\uAB2E\\\\uAB30-\\\\uAB5A\\\\uAB5C-\\\\uAB5F\\\\uAB64\\\\uAB65\\\\uABC0-\\\\uABE2\\\\uAC00-\\\\uD7A3\\\\uD7B0-\\\\uD7C6\\\\uD7CB-\\\\uD7FB\\\\uF900-\\\\uFA6D\\\\uFA70-\\\\uFAD9\\\\uFB00-\\\\uFB06\\\\uFB13-\\\\uFB17\\\\uFB1D\\\\uFB1F-\\\\uFB28\\\\uFB2A-\\\\uFB36\\\\uFB38-\\\\uFB3C\\\\uFB3E\\\\uFB40\\\\uFB41\\\\uFB43\\\\uFB44\\\\uFB46-\\\\uFBB1\\\\uFBD3-\\\\uFD3D\\\\uFD50-\\\\uFD8F\\\\uFD92-\\\\uFDC7\\\\uFDF0-\\\\uFDFB\\\\uFE70-\\\\uFE74\\\\uFE76-\\\\uFEFC\\\\uFF21-\\\\uFF3A\\\\uFF41-\\\\uFF5A\\\\uFF66-\\\\uFFBE\\\\uFFC2-\\\\uFFC7\\\\uFFCA-\\\\uFFCF\\\\uFFD2-\\\\uFFD7\\\\uFFDA-\\\\uFFDC]\"),m$=new RegExp(\"[\\\\xAA\\\\xB5\\\\xBA\\\\xC0-\\\\xD6\\\\xD8-\\\\xF6\\\\xF8-\\\\u02C1\\\\u02C6-\\\\u02D1\\\\u02E0-\\\\u02E4\\\\u02EC\\\\u02EE\\\\u0300-\\\\u0374\\\\u0376\\\\u0377\\\\u037A-\\\\u037D\\\\u037F\\\\u0386\\\\u0388-\\\\u038A\\\\u038C\\\\u038E-\\\\u03A1\\\\u03A3-\\\\u03F5\\\\u03F7-\\\\u0481\\\\u0483-\\\\u0487\\\\u048A-\\\\u052F\\\\u0531-\\\\u0556\\\\u0559\\\\u0561-\\\\u0587\\\\u0591-\\\\u05BD\\\\u05BF\\\\u05C1\\\\u05C2\\\\u05C4\\\\u05C5\\\\u05C7\\\\u05D0-\\\\u05EA\\\\u05F0-\\\\u05F2\\\\u0610-\\\\u061A\\\\u0620-\\\\u0669\\\\u066E-\\\\u06D3\\\\u06D5-\\\\u06DC\\\\u06DF-\\\\u06E8\\\\u06EA-\\\\u06FC\\\\u06FF\\\\u0710-\\\\u074A\\\\u074D-\\\\u07B1\\\\u07C0-\\\\u07F5\\\\u07FA\\\\u0800-\\\\u082D\\\\u0840-\\\\u085B\\\\u08A0-\\\\u08B2\\\\u08E4-\\\\u0963\\\\u0966-\\\\u096F\\\\u0971-\\\\u0983\\\\u0985-\\\\u098C\\\\u098F\\\\u0990\\\\u0993-\\\\u09A8\\\\u09AA-\\\\u09B0\\\\u09B2\\\\u09B6-\\\\u09B9\\\\u09BC-\\\\u09C4\\\\u09C7\\\\u09C8\\\\u09CB-\\\\u09CE\\\\u09D7\\\\u09DC\\\\u09DD\\\\u09DF-\\\\u09E3\\\\u09E6-\\\\u09F1\\\\u0A01-\\\\u0A03\\\\u0A05-\\\\u0A0A\\\\u0A0F\\\\u0A10\\\\u0A13-\\\\u0A28\\\\u0A2A-\\\\u0A30\\\\u0A32\\\\u0A33\\\\u0A35\\\\u0A36\\\\u0A38\\\\u0A39\\\\u0A3C\\\\u0A3E-\\\\u0A42\\\\u0A47\\\\u0A48\\\\u0A4B-\\\\u0A4D\\\\u0A51\\\\u0A59-\\\\u0A5C\\\\u0A5E\\\\u0A66-\\\\u0A75\\\\u0A81-\\\\u0A83\\\\u0A85-\\\\u0A8D\\\\u0A8F-\\\\u0A91\\\\u0A93-\\\\u0AA8\\\\u0AAA-\\\\u0AB0\\\\u0AB2\\\\u0AB3\\\\u0AB5-\\\\u0AB9\\\\u0ABC-\\\\u0AC5\\\\u0AC7-\\\\u0AC9\\\\u0ACB-\\\\u0ACD\\\\u0AD0\\\\u0AE0-\\\\u0AE3\\\\u0AE6-\\\\u0AEF\\\\u0B01-\\\\u0B03\\\\u0B05-\\\\u0B0C\\\\u0B0F\\\\u0B10\\\\u0B13-\\\\u0B28\\\\u0B2A-\\\\u0B30\\\\u0B32\\\\u0B33\\\\u0B35-\\\\u0B39\\\\u0B3C-\\\\u0B44\\\\u0B47\\\\u0B48\\\\u0B4B-\\\\u0B4D\\\\u0B56\\\\u0B57\\\\u0B5C\\\\u0B5D\\\\u0B5F-\\\\u0B63\\\\u0B66-\\\\u0B6F\\\\u0B71\\\\u0B82\\\\u0B83\\\\u0B85-\\\\u0B8A\\\\u0B8E-\\\\u0B90\\\\u0B92-\\\\u0B95\\\\u0B99\\\\u0B9A\\\\u0B9C\\\\u0B9E\\\\u0B9F\\\\u0BA3\\\\u0BA4\\\\u0BA8-\\\\u0BAA\\\\u0BAE-\\\\u0BB9\\\\u0BBE-\\\\u0BC2\\\\u0BC6-\\\\u0BC8\\\\u0BCA-\\\\u0BCD\\\\u0BD0\\\\u0BD7\\\\u0BE6-\\\\u0BEF\\\\u0C00-\\\\u0C03\\\\u0C05-\\\\u0C0C\\\\u0C0E-\\\\u0C10\\\\u0C12-\\\\u0C28\\\\u0C2A-\\\\u0C39\\\\u0C3D-\\\\u0C44\\\\u0C46-\\\\u0C48\\\\u0C4A-\\\\u0C4D\\\\u0C55\\\\u0C56\\\\u0C58\\\\u0C59\\\\u0C60-\\\\u0C63\\\\u0C66-\\\\u0C6F\\\\u0C81-\\\\u0C83\\\\u0C85-\\\\u0C8C\\\\u0C8E-\\\\u0C90\\\\u0C92-\\\\u0CA8\\\\u0CAA-\\\\u0CB3\\\\u0CB5-\\\\u0CB9\\\\u0CBC-\\\\u0CC4\\\\u0CC6-\\\\u0CC8\\\\u0CCA-\\\\u0CCD\\\\u0CD5\\\\u0CD6\\\\u0CDE\\\\u0CE0-\\\\u0CE3\\\\u0CE6-\\\\u0CEF\\\\u0CF1\\\\u0CF2\\\\u0D01-\\\\u0D03\\\\u0D05-\\\\u0D0C\\\\u0D0E-\\\\u0D10\\\\u0D12-\\\\u0D3A\\\\u0D3D-\\\\u0D44\\\\u0D46-\\\\u0D48\\\\u0D4A-\\\\u0D4E\\\\u0D57\\\\u0D60-\\\\u0D63\\\\u0D66-\\\\u0D6F\\\\u0D7A-\\\\u0D7F\\\\u0D82\\\\u0D83\\\\u0D85-\\\\u0D96\\\\u0D9A-\\\\u0DB1\\\\u0DB3-\\\\u0DBB\\\\u0DBD\\\\u0DC0-\\\\u0DC6\\\\u0DCA\\\\u0DCF-\\\\u0DD4\\\\u0DD6\\\\u0DD8-\\\\u0DDF\\\\u0DE6-\\\\u0DEF\\\\u0DF2\\\\u0DF3\\\\u0E01-\\\\u0E3A\\\\u0E40-\\\\u0E4E\\\\u0E50-\\\\u0E59\\\\u0E81\\\\u0E82\\\\u0E84\\\\u0E87\\\\u0E88\\\\u0E8A\\\\u0E8D\\\\u0E94-\\\\u0E97\\\\u0E99-\\\\u0E9F\\\\u0EA1-\\\\u0EA3\\\\u0EA5\\\\u0EA7\\\\u0EAA\\\\u0EAB\\\\u0EAD-\\\\u0EB9\\\\u0EBB-\\\\u0EBD\\\\u0EC0-\\\\u0EC4\\\\u0EC6\\\\u0EC8-\\\\u0ECD\\\\u0ED0-\\\\u0ED9\\\\u0EDC-\\\\u0EDF\\\\u0F00\\\\u0F18\\\\u0F19\\\\u0F20-\\\\u0F29\\\\u0F35\\\\u0F37\\\\u0F39\\\\u0F3E-\\\\u0F47\\\\u0F49-\\\\u0F6C\\\\u0F71-\\\\u0F84\\\\u0F86-\\\\u0F97\\\\u0F99-\\\\u0FBC\\\\u0FC6\\\\u1000-\\\\u1049\\\\u1050-\\\\u109D\\\\u10A0-\\\\u10C5\\\\u10C7\\\\u10CD\\\\u10D0-\\\\u10FA\\\\u10FC-\\\\u1248\\\\u124A-\\\\u124D\\\\u1250-\\\\u1256\\\\u1258\\\\u125A-\\\\u125D\\\\u1260-\\\\u1288\\\\u128A-\\\\u128D\\\\u1290-\\\\u12B0\\\\u12B2-\\\\u12B5\\\\u12B8-\\\\u12BE\\\\u12C0\\\\u12C2-\\\\u12C5\\\\u12C8-\\\\u12D6\\\\u12D8-\\\\u1310\\\\u1312-\\\\u1315\\\\u1318-\\\\u135A\\\\u135D-\\\\u135F\\\\u1380-\\\\u138F\\\\u13A0-\\\\u13F4\\\\u1401-\\\\u166C\\\\u166F-\\\\u167F\\\\u1681-\\\\u169A\\\\u16A0-\\\\u16EA\\\\u16EE-\\\\u16F8\\\\u1700-\\\\u170C\\\\u170E-\\\\u1714\\\\u1720-\\\\u1734\\\\u1740-\\\\u1753\\\\u1760-\\\\u176C\\\\u176E-\\\\u1770\\\\u1772\\\\u1773\\\\u1780-\\\\u17D3\\\\u17D7\\\\u17DC\\\\u17DD\\\\u17E0-\\\\u17E9\\\\u180B-\\\\u180D\\\\u1810-\\\\u1819\\\\u1820-\\\\u1877\\\\u1880-\\\\u18AA\\\\u18B0-\\\\u18F5\\\\u1900-\\\\u191E\\\\u1920-\\\\u192B\\\\u1930-\\\\u193B\\\\u1946-\\\\u196D\\\\u1970-\\\\u1974\\\\u1980-\\\\u19AB\\\\u19B0-\\\\u19C9\\\\u19D0-\\\\u19D9\\\\u1A00-\\\\u1A1B\\\\u1A20-\\\\u1A5E\\\\u1A60-\\\\u1A7C\\\\u1A7F-\\\\u1A89\\\\u1A90-\\\\u1A99\\\\u1AA7\\\\u1AB0-\\\\u1ABD\\\\u1B00-\\\\u1B4B\\\\u1B50-\\\\u1B59\\\\u1B6B-\\\\u1B73\\\\u1B80-\\\\u1BF3\\\\u1C00-\\\\u1C37\\\\u1C40-\\\\u1C49\\\\u1C4D-\\\\u1C7D\\\\u1CD0-\\\\u1CD2\\\\u1CD4-\\\\u1CF6\\\\u1CF8\\\\u1CF9\\\\u1D00-\\\\u1DF5\\\\u1DFC-\\\\u1F15\\\\u1F18-\\\\u1F1D\\\\u1F20-\\\\u1F45\\\\u1F48-\\\\u1F4D\\\\u1F50-\\\\u1F57\\\\u1F59\\\\u1F5B\\\\u1F5D\\\\u1F5F-\\\\u1F7D\\\\u1F80-\\\\u1FB4\\\\u1FB6-\\\\u1FBC\\\\u1FBE\\\\u1FC2-\\\\u1FC4\\\\u1FC6-\\\\u1FCC\\\\u1FD0-\\\\u1FD3\\\\u1FD6-\\\\u1FDB\\\\u1FE0-\\\\u1FEC\\\\u1FF2-\\\\u1FF4\\\\u1FF6-\\\\u1FFC\\\\u200C\\\\u200D\\\\u203F\\\\u2040\\\\u2054\\\\u2071\\\\u207F\\\\u2090-\\\\u209C\\\\u20D0-\\\\u20DC\\\\u20E1\\\\u20E5-\\\\u20F0\\\\u2102\\\\u2107\\\\u210A-\\\\u2113\\\\u2115\\\\u2119-\\\\u211D\\\\u2124\\\\u2126\\\\u2128\\\\u212A-\\\\u212D\\\\u212F-\\\\u2139\\\\u213C-\\\\u213F\\\\u2145-\\\\u2149\\\\u214E\\\\u2160-\\\\u2188\\\\u2C00-\\\\u2C2E\\\\u2C30-\\\\u2C5E\\\\u2C60-\\\\u2CE4\\\\u2CEB-\\\\u2CF3\\\\u2D00-\\\\u2D25\\\\u2D27\\\\u2D2D\\\\u2D30-\\\\u2D67\\\\u2D6F\\\\u2D7F-\\\\u2D96\\\\u2DA0-\\\\u2DA6\\\\u2DA8-\\\\u2DAE\\\\u2DB0-\\\\u2DB6\\\\u2DB8-\\\\u2DBE\\\\u2DC0-\\\\u2DC6\\\\u2DC8-\\\\u2DCE\\\\u2DD0-\\\\u2DD6\\\\u2DD8-\\\\u2DDE\\\\u2DE0-\\\\u2DFF\\\\u2E2F\\\\u3005-\\\\u3007\\\\u3021-\\\\u302F\\\\u3031-\\\\u3035\\\\u3038-\\\\u303C\\\\u3041-\\\\u3096\\\\u3099\\\\u309A\\\\u309D-\\\\u309F\\\\u30A1-\\\\u30FA\\\\u30FC-\\\\u30FF\\\\u3105-\\\\u312D\\\\u3131-\\\\u318E\\\\u31A0-\\\\u31BA\\\\u31F0-\\\\u31FF\\\\u3400-\\\\u4DB5\\\\u4E00-\\\\u9FCC\\\\uA000-\\\\uA48C\\\\uA4D0-\\\\uA4FD\\\\uA500-\\\\uA60C\\\\uA610-\\\\uA62B\\\\uA640-\\\\uA66F\\\\uA674-\\\\uA67D\\\\uA67F-\\\\uA69D\\\\uA69F-\\\\uA6F1\\\\uA717-\\\\uA71F\\\\uA722-\\\\uA788\\\\uA78B-\\\\uA78E\\\\uA790-\\\\uA7AD\\\\uA7B0\\\\uA7B1\\\\uA7F7-\\\\uA827\\\\uA840-\\\\uA873\\\\uA880-\\\\uA8C4\\\\uA8D0-\\\\uA8D9\\\\uA8E0-\\\\uA8F7\\\\uA8FB\\\\uA900-\\\\uA92D\\\\uA930-\\\\uA953\\\\uA960-\\\\uA97C\\\\uA980-\\\\uA9C0\\\\uA9CF-\\\\uA9D9\\\\uA9E0-\\\\uA9FE\\\\uAA00-\\\\uAA36\\\\uAA40-\\\\uAA4D\\\\uAA50-\\\\uAA59\\\\uAA60-\\\\uAA76\\\\uAA7A-\\\\uAAC2\\\\uAADB-\\\\uAADD\\\\uAAE0-\\\\uAAEF\\\\uAAF2-\\\\uAAF6\\\\uAB01-\\\\uAB06\\\\uAB09-\\\\uAB0E\\\\uAB11-\\\\uAB16\\\\uAB20-\\\\uAB26\\\\uAB28-\\\\uAB2E\\\\uAB30-\\\\uAB5A\\\\uAB5C-\\\\uAB5F\\\\uAB64\\\\uAB65\\\\uABC0-\\\\uABEA\\\\uABEC\\\\uABED\\\\uABF0-\\\\uABF9\\\\uAC00-\\\\uD7A3\\\\uD7B0-\\\\uD7C6\\\\uD7CB-\\\\uD7FB\\\\uF900-\\\\uFA6D\\\\uFA70-\\\\uFAD9\\\\uFB00-\\\\uFB06\\\\uFB13-\\\\uFB17\\\\uFB1D-\\\\uFB28\\\\uFB2A-\\\\uFB36\\\\uFB38-\\\\uFB3C\\\\uFB3E\\\\uFB40\\\\uFB41\\\\uFB43\\\\uFB44\\\\uFB46-\\\\uFBB1\\\\uFBD3-\\\\uFD3D\\\\uFD50-\\\\uFD8F\\\\uFD92-\\\\uFDC7\\\\uFDF0-\\\\uFDFB\\\\uFE00-\\\\uFE0F\\\\uFE20-\\\\uFE2D\\\\uFE33\\\\uFE34\\\\uFE4D-\\\\uFE4F\\\\uFE70-\\\\uFE74\\\\uFE76-\\\\uFEFC\\\\uFF10-\\\\uFF19\\\\uFF21-\\\\uFF3A\\\\uFF3F\\\\uFF41-\\\\uFF5A\\\\uFF66-\\\\uFFBE\\\\uFFC2-\\\\uFFC7\\\\uFFCA-\\\\uFFCF\\\\uFFD2-\\\\uFFD7\\\\uFFDA-\\\\uFFDC]\");function y$(t,e){if(!t)throw new Error(\"ASSERT: \"+e)}function v$(t){return t>=48&&t<=57}function _$(t){return\"0123456789abcdefABCDEF\".includes(t)}function x$(t){return\"01234567\".includes(t)}function b$(t){return 32===t||9===t||11===t||12===t||160===t||t>=5760&&[5760,6158,8192,8193,8194,8195,8196,8197,8198,8199,8200,8201,8202,8239,8287,12288,65279].includes(t)}function w$(t){return 10===t||13===t||8232===t||8233===t}function k$(t){return 36===t||95===t||t>=65&&t<=90||t>=97&&t<=122||92===t||t>=128&&g$.test(String.fromCharCode(t))}function A$(t){return 36===t||95===t||t>=65&&t<=90||t>=97&&t<=122||t>=48&&t<=57||92===t||t>=128&&m$.test(String.fromCharCode(t))}const M$={if:1,in:1,do:1,var:1,for:1,new:1,try:1,let:1,this:1,else:1,case:1,void:1,with:1,enum:1,while:1,break:1,catch:1,throw:1,const:1,yield:1,class:1,super:1,return:1,typeof:1,delete:1,switch:1,export:1,import:1,public:1,static:1,default:1,finally:1,extends:1,package:1,private:1,function:1,continue:1,debugger:1,interface:1,protected:1,instanceof:1,implements:1};function E$(){for(;NS1114111||\"}\"!==t)&&I$({},r$,d$),e<=65535?String.fromCharCode(e):(n=55296+(e-65536>>10),r=56320+(e-65536&1023),String.fromCharCode(n,r))}function F$(){var t,e;for(t=zS.charCodeAt(NS++),e=String.fromCharCode(t),92===t&&(117!==zS.charCodeAt(NS)&&I$({},r$,d$),++NS,(t=D$(\"u\"))&&\"\\\\\"!==t&&k$(t.charCodeAt(0))||I$({},r$,d$),e=t);NS>>=\"===(r=zS.substr(NS,4))?{type:WS,value:r,start:i,end:NS+=4}:\">>>\"===(n=r.substr(0,3))||\"<<=\"===n||\">>=\"===n?{type:WS,value:n,start:i,end:NS+=3}:a===(e=n.substr(0,2))[1]&&\"+-<>&|\".includes(a)||\"=>\"===e?{type:WS,value:e,start:i,end:NS+=2}:(\"//\"===e&&I$({},r$,d$),\"<>=!+-*%&|^/\".includes(a)?(++NS,{type:WS,value:a,start:i,end:NS}):void I$({},r$,d$))}function T$(){var t,e,n;if(y$(v$((n=zS[NS]).charCodeAt(0))||\".\"===n,\"Numeric literal must start with a decimal digit or a decimal point\"),e=NS,t=\"\",\".\"!==n){if(t=zS[NS++],n=zS[NS],\"0\"===t){if(\"x\"===n||\"X\"===n)return++NS,function(t){let e=\"\";for(;NS=0&&I$({},l$,n),{value:n,literal:e}}(),r=function(t,e){let n=t;e.includes(\"u\")&&(n=n.replace(/\\\\u\\{([0-9a-fA-F]+)\\}/g,((t,e)=>{if(parseInt(e,16)<=1114111)return\"x\";I$({},l$)})).replace(/[\\uD800-\\uDBFF][\\uDC00-\\uDFFF]/g,\"x\"));try{new RegExp(n)}catch(t){I$({},l$)}try{return new RegExp(t,e)}catch(t){return null}}(e.value,n.value),{literal:e.literal+n.literal,value:r,regex:{pattern:e.value,flags:n.value},start:t,end:NS}}function z$(){if(E$(),NS>=OS)return{type:LS,start:NS,end:NS};const t=zS.charCodeAt(NS);return k$(t)?S$():40===t||41===t||59===t?$$():39===t||34===t?function(){var t,e,n,r,i=\"\",o=!1;for(y$(\"'\"===(t=zS[NS])||'\"'===t,\"String literal must starts with a quote\"),e=NS,++NS;NS(y$(e\":case\"<=\":case\">=\":case\"instanceof\":case\"in\":e=7;break;case\"<<\":case\">>\":case\">>>\":e=8;break;case\"+\":case\"-\":e=9;break;case\"*\":case\"/\":case\"%\":e=11}return e}function aT(){var t,e;return t=function(){var t,e,n,r,i,o,a,s,u,l;if(t=RS,u=iT(),0===(i=oT(r=RS)))return u;for(r.prec=i,N$(),e=[t,RS],o=[u,r,a=iT()];(i=oT(RS))>0;){for(;o.length>2&&i<=o[o.length-2].prec;)a=o.pop(),s=o.pop().value,u=o.pop(),e.pop(),n=R$(s,u,a),o.push(n);(r=N$()).prec=i,o.push(r),e.push(RS),n=iT(),o.push(n)}for(n=o[l=o.length-1],e.pop();l>1;)e.pop(),n=R$(o[l-1].value,o[l-2],n),l-=2;return n}(),Y$(\"?\")&&(N$(),e=aT(),H$(\":\"),t=function(t,e,n){const r=new TS(XS);return r.test=t,r.consequent=e,r.alternate=n,r}(t,e,aT())),t}function sT(){const t=aT();if(Y$(\",\"))throw new Error(p$);return t}function uT(t){NS=0,OS=(zS=t).length,RS=null,O$();const e=sT();if(RS.type!==LS)throw new Error(\"Unexpect token after expression.\");return e}var lT={NaN:\"NaN\",E:\"Math.E\",LN2:\"Math.LN2\",LN10:\"Math.LN10\",LOG2E:\"Math.LOG2E\",LOG10E:\"Math.LOG10E\",PI:\"Math.PI\",SQRT1_2:\"Math.SQRT1_2\",SQRT2:\"Math.SQRT2\",MIN_VALUE:\"Number.MIN_VALUE\",MAX_VALUE:\"Number.MAX_VALUE\"};function cT(t){function e(e,n,r){return i=>function(e,n,r,i){let o=t(n[0]);return r&&(o=r+\"(\"+o+\")\",0===r.lastIndexOf(\"new \",0)&&(o=\"(\"+o+\")\")),o+\".\"+e+(i<0?\"\":0===i?\"()\":\"(\"+n.slice(1).map(t).join(\",\")+\")\")}(e,i,n,r)}const n=\"new Date\",r=\"String\",i=\"RegExp\";return{isNaN:\"Number.isNaN\",isFinite:\"Number.isFinite\",abs:\"Math.abs\",acos:\"Math.acos\",asin:\"Math.asin\",atan:\"Math.atan\",atan2:\"Math.atan2\",ceil:\"Math.ceil\",cos:\"Math.cos\",exp:\"Math.exp\",floor:\"Math.floor\",hypot:\"Math.hypot\",log:\"Math.log\",max:\"Math.max\",min:\"Math.min\",pow:\"Math.pow\",random:\"Math.random\",round:\"Math.round\",sin:\"Math.sin\",sqrt:\"Math.sqrt\",tan:\"Math.tan\",clamp:function(e){e.length<3&&s(\"Missing arguments to clamp function.\"),e.length>3&&s(\"Too many arguments to clamp function.\");const n=e.map(t);return\"Math.max(\"+n[1]+\", Math.min(\"+n[2]+\",\"+n[0]+\"))\"},now:\"Date.now\",utc:\"Date.UTC\",datetime:n,date:e(\"getDate\",n,0),day:e(\"getDay\",n,0),year:e(\"getFullYear\",n,0),month:e(\"getMonth\",n,0),hours:e(\"getHours\",n,0),minutes:e(\"getMinutes\",n,0),seconds:e(\"getSeconds\",n,0),milliseconds:e(\"getMilliseconds\",n,0),time:e(\"getTime\",n,0),timezoneoffset:e(\"getTimezoneOffset\",n,0),utcdate:e(\"getUTCDate\",n,0),utcday:e(\"getUTCDay\",n,0),utcyear:e(\"getUTCFullYear\",n,0),utcmonth:e(\"getUTCMonth\",n,0),utchours:e(\"getUTCHours\",n,0),utcminutes:e(\"getUTCMinutes\",n,0),utcseconds:e(\"getUTCSeconds\",n,0),utcmilliseconds:e(\"getUTCMilliseconds\",n,0),length:e(\"length\",null,-1),parseFloat:\"parseFloat\",parseInt:\"parseInt\",upper:e(\"toUpperCase\",r,0),lower:e(\"toLowerCase\",r,0),substring:e(\"substring\",r),split:e(\"split\",r),trim:e(\"trim\",r,0),regexp:i,test:e(\"test\",i),if:function(e){e.length<3&&s(\"Missing arguments to if function.\"),e.length>3&&s(\"Too many arguments to if function.\");const n=e.map(t);return\"(\"+n[0]+\"?\"+n[1]+\":\"+n[2]+\")\"}}}function fT(t){const e=(t=t||{}).allowed?Bt(t.allowed):{},n=t.forbidden?Bt(t.forbidden):{},r=t.constants||lT,i=(t.functions||cT)(h),o=t.globalvar,a=t.fieldvar,u=J(o)?o:t=>`${o}[\"${t}\"]`;let l={},c={},f=0;function h(t){if(xt(t))return t;const e=d[t.type];return null==e&&s(\"Unsupported type: \"+t.type),e(t)}const d={Literal:t=>t.raw,Identifier:t=>{const i=t.name;return f>0?i:lt(n,i)?s(\"Illegal identifier: \"+i):lt(r,i)?r[i]:lt(e,i)?i:(l[i]=1,u(i))},MemberExpression:t=>{const e=!t.computed,n=h(t.object);e&&(f+=1);const r=h(t.property);return n===a&&(c[function(t){const e=t&&t.length-1;return e&&('\"'===t[0]&&'\"'===t[e]||\"'\"===t[0]&&\"'\"===t[e])?t.slice(1,-1):t}(r)]=1),e&&(f-=1),n+(e?\".\"+r:\"[\"+r+\"]\")},CallExpression:t=>{\"Identifier\"!==t.callee.type&&s(\"Illegal callee type: \"+t.callee.type);const e=t.callee.name,n=t.arguments,r=lt(i,e)&&i[e];return r||s(\"Unrecognized function: \"+e),J(r)?r(n):r+\"(\"+n.map(h).join(\",\")+\")\"},ArrayExpression:t=>\"[\"+t.elements.map(h).join(\",\")+\"]\",BinaryExpression:t=>\"(\"+h(t.left)+\" \"+t.operator+\" \"+h(t.right)+\")\",UnaryExpression:t=>\"(\"+t.operator+h(t.argument)+\")\",ConditionalExpression:t=>\"(\"+h(t.test)+\"?\"+h(t.consequent)+\":\"+h(t.alternate)+\")\",LogicalExpression:t=>\"(\"+h(t.left)+t.operator+h(t.right)+\")\",ObjectExpression:t=>\"{\"+t.properties.map(h).join(\",\")+\"}\",Property:t=>{f+=1;const e=h(t.key);return f-=1,e+\":\"+h(t.value)}};function p(t){const e={code:h(t),globals:Object.keys(l),fields:Object.keys(c)};return l={},c={},e}return p.functions=i,p.constants=r,p}const hT=Symbol(\"vega_selection_getter\");function dT(t){return t.getter&&t.getter[hT]||(t.getter=l(t.field),t.getter[hT]=!0),t.getter}const pT=\"intersect\",gT=\"union\",mT=\"_vgsid_\",yT=l(mT),vT=\"E\",_T=\"R\",xT=\"R-E\",bT=\"R-LE\",wT=\"R-RE\",kT=\"index:unit\";function AT(t,e){for(var n,r,i=e.fields,o=e.values,a=i.length,s=0;s1?e-1:0),r=1;re.includes(t))):e},R_union:function(t,e){var n=S(e[0]),r=S(e[1]);return n>r&&(n=e[1],r=e[0]),t.length?(t[0]>n&&(t[0]=n),t[1]r&&(n=e[1],r=e[0]),t.length?rr&&(t[1]=r),t):[n,r]}};function FT(t,e,n,r){e[0].type!==wS&&s(\"First argument to selection functions must be a string literal.\");const i=e[0].value,o=\"unit\",a=\"@\"+o,u=\":\"+i;(e.length>=2&&F(e).value)!==pT||lt(r,a)||(r[a]=n.getData(i).indataRef(n,o)),lt(r,u)||(r[u]=n.getData(i).tuplesRef())}function ST(t){const e=this.context.data[t];return e?e.values.value:[]}const $T=t=>function(e,n){return this.context.dataflow.locale()[t](n)(e)},TT=$T(\"format\"),BT=$T(\"timeFormat\"),zT=$T(\"utcFormat\"),NT=$T(\"timeParse\"),OT=$T(\"utcParse\"),RT=new Date(2e3,0,1);function UT(t,e,n){return Number.isInteger(t)&&Number.isInteger(e)?(RT.setYear(2e3),RT.setMonth(t),RT.setDate(e),BT.call(this,RT,n)):\"\"}const LT=\"%\",qT=\"$\";function PT(t,e,n,r){e[0].type!==wS&&s(\"First argument to data functions must be a string literal.\");const i=e[0].value,o=\":\"+i;if(!lt(o,r))try{r[o]=n.getData(i).tuplesRef()}catch(t){}}function jT(t,e,n,r){if(e[0].type===wS)IT(n,r,e[0].value);else for(t in n.scales)IT(n,r,t)}function IT(t,e,n){const r=LT+n;if(!lt(e,r))try{e[r]=t.scaleRef(n)}catch(t){}}function WT(t,e){if(J(t))return t;if(xt(t)){const n=e.scales[t];return n&&function(t){return t&&!0===t[Gd]}(n.value)?n.value:void 0}}function HT(t,e,n){e.__bandwidth=t=>t&&t.bandwidth?t.bandwidth():0,n._bandwidth=jT,n._range=jT,n._scale=jT;const r=e=>\"_[\"+(e.type===wS?Ct(LT+e.value):Ct(LT)+\"+\"+t(e))+\"]\";return{_bandwidth:t=>`this.__bandwidth(${r(t[0])})`,_range:t=>`${r(t[0])}.range()`,_scale:e=>`${r(e[0])}(${t(e[1])})`}}function YT(t,e){return function(n,r,i){if(n){const e=WT(n,(i||this).context);return e&&e.path[t](r)}return e(r)}}const GT=YT(\"area\",(function(t){return bw=new se,rw(t,ww),2*bw})),VT=YT(\"bounds\",(function(t){var e,n,r,i,o,a,s;if(hw=fw=-(lw=cw=1/0),vw=[],rw(t,Jw),n=vw.length){for(vw.sort(ok),e=1,o=[r=vw[0]];eik(r[0],r[1])&&(r[1]=i[1]),ik(i[0],r[1])>ik(r[0],r[1])&&(r[0]=i[0])):o.push(r=i);for(a=-1/0,e=0,r=o[n=o.length-1];e<=n;r=i,++e)i=o[e],(s=ik(r[1],i[0]))>a&&(a=s,lw=i[0],fw=r[1])}return vw=_w=null,lw===1/0||cw===1/0?[[NaN,NaN],[NaN,NaN]]:[[lw,cw],[fw,hw]]})),XT=YT(\"centroid\",(function(t){zw=Nw=Ow=Rw=Uw=Lw=qw=Pw=0,jw=new se,Iw=new se,Ww=new se,rw(t,sk);var e=+jw,n=+Iw,r=+Ww,i=jb(e,n,r);return itB(t,e)}const nB={};function rB(t){return k(t)||ArrayBuffer.isView(t)?t:null}function iB(t){return rB(t)||(xt(t)?t:null)}const oB=t=>t.data;function aB(t,e){const n=ST.call(e,t);return n.root&&n.root.lookup||{}}const sB=()=>\"undefined\"!=typeof window&&window||null;function uB(t,e,n){if(!t)return[];const[r,i]=t,o=(new Rg).set(r[0],r[1],i[0],i[1]);return w_(n||this.context.dataflow.scenegraph().root,o,function(t){let e=null;if(t){const n=V(t.marktype),r=V(t.markname);e=t=>(!n.length||n.some((e=>t.marktype===e)))&&(!r.length||r.some((e=>t.name===e)))}return e}(e))}const lB={random:()=>t.random(),cumulativeNormal:hs,cumulativeLogNormal:vs,cumulativeUniform:As,densityNormal:fs,densityLogNormal:ys,densityUniform:ks,quantileNormal:ds,quantileLogNormal:_s,quantileUniform:Ms,sampleNormal:cs,sampleLogNormal:ms,sampleUniform:ws,isArray:k,isBoolean:gt,isDate:mt,isDefined:t=>void 0!==t,isNumber:vt,isObject:A,isRegExp:_t,isString:xt,isTuple:ma,isValid:t=>null!=t&&t==t,toBoolean:Ft,toDate:t=>$t(t),toNumber:S,toString:Tt,indexof:function(t){for(var e=arguments.length,n=new Array(e>1?e-1:0),r=1;r1?e-1:0),r=1;r1?e-1:0),r=1;r1?e-1:0),r=1;rat(t),inScope:function(t){const e=this.context.group;let n=!1;if(e)for(;t;){if(t===e){n=!0;break}t=t.mark.group}return n},intersect:uB,clampRange:X,pinchDistance:function(t){const e=t.touches,n=e[0].clientX-e[1].clientX,r=e[0].clientY-e[1].clientY;return Math.hypot(n,r)},pinchAngle:function(t){const e=t.touches;return Math.atan2(e[0].clientY-e[1].clientY,e[0].clientX-e[1].clientX)},screen:function(){const t=sB();return t?t.screen:{}},containerSize:function(){const t=this.context.dataflow,e=t.container&&t.container();return e?[e.clientWidth,e.clientHeight]:[void 0,void 0]},windowSize:function(){const t=sB();return t?[t.innerWidth,t.innerHeight]:[void 0,void 0]},bandspace:function(t,e,n){return xd(t||0,e||0,n||0)},setdata:function(t,e){const n=this.context.dataflow,r=this.context.data[t].input;return n.pulse(r,n.changeset().remove(p).insert(e)),1},pathShape:function(t){let e=null;return function(n){return n?ag(n,e=e||Xp(t)):t}},panLinear:R,panLog:U,panPow:L,panSymlog:q,zoomLinear:j,zoomLog:I,zoomPow:W,zoomSymlog:H,encode:function(t,e,n){if(t){const n=this.context.dataflow,r=t.mark.source;n.pulse(r,n.changeset().encode(t,e))}return void 0!==n?n:t},modify:function(t,e,n,r,i,o){const a=this.context.dataflow,s=this.context.data[t],u=s.input,l=a.stamp();let c,f,h=s.changes;if(!1===a._trigger||!(u.value.length||e||r))return 0;if((!h||h.stamp{s.modified=!0,a.pulse(u,h).run()}),!0,1)),n&&(c=!0===n?p:k(n)||ma(n)?n:eB(n),h.remove(c)),e&&h.insert(e),r&&(c=eB(r),u.value.some(c)?h.remove(c):h.insert(r)),i)for(f in o)h.modify(i,f,o[f]);return 1},lassoAppend:function(t,e,n){let r=arguments.length>3&&void 0!==arguments[3]?arguments[3]:5;const i=(t=V(t))[t.length-1];return void 0===i||Math.hypot(i[0]-e,i[1]-n)>r?[...t,[e,n]]:t},lassoPath:function(t){return V(t).reduce(((e,n,r)=>{let[i,o]=n;return e+(0==r?`M ${i},${o} `:r===t.length-1?\" Z\":`L ${i},${o} `)}),\"\")},intersectLasso:function(t,e,n){const{x:r,y:i,mark:o}=n,a=(new Rg).set(Number.MAX_SAFE_INTEGER,Number.MAX_SAFE_INTEGER,Number.MIN_SAFE_INTEGER,Number.MIN_SAFE_INTEGER);for(const[t,n]of e)ta.x2&&(a.x2=t),na.y2&&(a.y2=n);return a.translate(r,i),uB([[a.x1,a.y1],[a.x2,a.y2]],t,o).filter((t=>function(t,e,n){let r=0;for(let i=0,o=n.length-1;ie!=s>e&&t<(a-u)*(e-l)/(s-l)+u&&r++}return 1&r}(t.x,t.y,e)))}},cB=[\"view\",\"item\",\"group\",\"xy\",\"x\",\"y\"],fB=\"this.\",hB={},dB={forbidden:[\"_\"],allowed:[\"datum\",\"event\",\"item\"],fieldvar:\"datum\",globalvar:t=>`_[${Ct(qT+t)}]`,functions:function(t){const e=cT(t);cB.forEach((t=>e[t]=\"event.vega.\"+t));for(const t in lB)e[t]=fB+t;return ot(e,HT(t,lB,hB)),e},constants:lT,visitors:hB},pB=fT(dB);function gB(t,e,n){return 1===arguments.length?lB[t]:(lB[t]=e,n&&(hB[t]=n),pB&&(pB.functions[t]=fB+t),this)}function mB(t,e){const n={};let r;try{r=uT(t=xt(t)?t:Ct(t)+\"\")}catch(e){s(\"Expression parse error: \"+t)}r.visit((t=>{if(t.type!==ES)return;const r=t.callee.name,i=dB.visitors[r];i&&i(r,t.arguments,e,n)}));const i=pB(r);return i.globals.forEach((t=>{const r=qT+t;!lt(n,r)&&e.getSignal(t)&&(n[r]=e.signalRef(t))})),{$expr:ot({code:i.code},e.options.ast?{ast:r}:null),$fields:i.fields,$params:n}}gB(\"bandwidth\",(function(t,e){const n=WT(t,(e||this).context);return n&&n.bandwidth?n.bandwidth():0}),jT),gB(\"copy\",(function(t,e){const n=WT(t,(e||this).context);return n?n.copy():void 0}),jT),gB(\"domain\",(function(t,e){const n=WT(t,(e||this).context);return n?n.domain():[]}),jT),gB(\"range\",(function(t,e){const n=WT(t,(e||this).context);return n&&n.range?n.range():[]}),jT),gB(\"invert\",(function(t,e,n){const r=WT(t,(n||this).context);return r?k(e)?(r.invertRange||r.invert)(e):(r.invert||r.invertExtent)(e):void 0}),jT),gB(\"scale\",(function(t,e,n){const r=WT(t,(n||this).context);return r?r(e):void 0}),jT),gB(\"gradient\",(function(t,e,n,r,i){t=WT(t,(i||this).context);const o=Pp(e,n);let a=t.domain(),s=a[0],u=F(a),l=f;return u-s?l=up(t,s,u):t=(t.interpolator?Xd(\"sequential\")().interpolator(t.interpolator()):Xd(\"linear\")().interpolate(t.interpolate()).range(t.range())).domain([s=0,u=1]),t.ticks&&(a=t.ticks(+r||15),s!==a[0]&&a.unshift(s),u!==F(a)&&a.push(u)),a.forEach((e=>o.stop(l(e),t(e)))),o}),jT),gB(\"geoArea\",GT,jT),gB(\"geoBounds\",VT,jT),gB(\"geoCentroid\",XT,jT),gB(\"geoShape\",(function(t,e,n){const r=WT(t,(n||this).context);return function(t){return r?r.path.context(t)(e):\"\"}}),jT),gB(\"geoScale\",(function(t,e){const n=WT(t,(e||this).context);return n&&n.scale()}),jT),gB(\"indata\",(function(t,e,n){const r=this.context.data[t][\"index:\"+e],i=r?r.value.get(n):void 0;return i?i.count:i}),(function(t,e,n,r){e[0].type!==wS&&s(\"First argument to indata must be a string literal.\"),e[1].type!==wS&&s(\"Second argument to indata must be a string literal.\");const i=e[0].value,o=e[1].value,a=\"@\"+o;lt(a,r)||(r[a]=n.getData(i).indataRef(n,o))})),gB(\"data\",ST,PT),gB(\"treePath\",(function(t,e,n){const r=aB(t,this),i=r[e],o=r[n];return i&&o?i.path(o).map(oB):void 0}),PT),gB(\"treeAncestors\",(function(t,e){const n=aB(t,this)[e];return n?n.ancestors().map(oB):void 0}),PT),gB(\"vlSelectionTest\",(function(t,e,n){for(var r,i,o,a,s,u=this.context.data[t],l=u?u.values.value:[],c=u?u[kT]&&u[kT].value:void 0,f=n===pT,h=l.length,d=0;d(t[o[n].field]=e,t)),{}))}else u=mT,l=yT(i),(f=(c=v[u]||(v[u]={}))[s]||(c[s]=[])).push(l),n&&(f=_[s]||(_[s]=[])).push({[mT]:l});if(e=e||gT,v[mT]?v[mT]=CT[`${mT}_${e}`](...Object.values(v[mT])):Object.keys(v).forEach((t=>{v[t]=Object.keys(v[t]).map((e=>v[t][e])).reduce(((n,r)=>void 0===n?r:CT[`${x[t]}_${e}`](n,r)))})),y=Object.keys(_),n&&y.length){v[r?\"vlPoint\":\"vlMulti\"]=e===gT?{or:y.reduce(((t,e)=>(t.push(..._[e]),t)),[])}:{and:y.map((t=>({or:_[t]})))}}return v}),FT),gB(\"vlSelectionTuples\",(function(t,e){return t.map((t=>ot(e.fields?{values:e.fields.map((e=>dT(e)(t.datum)))}:{[mT]:yT(t.datum)},e)))}));const yB=Bt([\"rule\"]),vB=Bt([\"group\",\"image\",\"rect\"]);function _B(t){return(t+\"\").toLowerCase()}function xB(t,e,n){n.endsWith(\";\")||(n=\"return(\"+n+\");\");const r=Function(...e.concat(n));return t&&t.functions?r.bind(t.functions):r}var bB={operator:(t,e)=>xB(t,[\"_\"],e.code),parameter:(t,e)=>xB(t,[\"datum\",\"_\"],e.code),event:(t,e)=>xB(t,[\"event\"],e.code),handler:(t,e)=>xB(t,[\"_\",\"event\"],`var datum=event.item&&event.item.datum;return ${e.code};`),encode:(t,e)=>{const{marktype:n,channels:r}=e;let i=\"var o=item,datum=o.datum,m=0,$;\";for(const t in r){const e=\"o[\"+Ct(t)+\"]\";i+=`$=${r[t].code};if(${e}!==$)${e}=$,m=1;`}return i+=function(t,e){let n=\"\";return yB[e]||(t.x2&&(t.x?(vB[e]&&(n+=\"if(o.x>o.x2)$=o.x,o.x=o.x2,o.x2=$;\"),n+=\"o.width=o.x2-o.x;\"):n+=\"o.x=o.x2-(o.width||0);\"),t.xc&&(n+=\"o.x=o.xc-(o.width||0)/2;\"),t.y2&&(t.y?(vB[e]&&(n+=\"if(o.y>o.y2)$=o.y,o.y=o.y2,o.y2=$;\"),n+=\"o.height=o.y2-o.y;\"):n+=\"o.y=o.y2-(o.height||0);\"),t.yc&&(n+=\"o.y=o.yc-(o.height||0)/2;\")),n}(r,n),i+=\"return m;\",xB(t,[\"item\",\"_\"],i)},codegen:{get(t){const e=`[${t.map(Ct).join(\"][\")}]`,n=Function(\"_\",`return _${e};`);return n.path=e,n},comparator(t,e){let n;const r=Function(\"a\",\"b\",\"var u, v; return \"+t.map(((t,r)=>{const i=e[r];let o,a;return t.path?(o=`a${t.path}`,a=`b${t.path}`):((n=n||{})[\"f\"+r]=t,o=`this.f${r}(a)`,a=`this.f${r}(b)`),function(t,e,n,r){return`((u = ${t}) < (v = ${e}) || u == null) && v != null ? ${n}\\n : (u > v || v == null) && u != null ? ${r}\\n : ((v = v instanceof Date ? +v : v), (u = u instanceof Date ? +u : u)) !== u && v === v ? ${n}\\n : v !== v && u === u ? ${r} : `}(o,a,-i,i)})).join(\"\")+\"0;\");return n?r.bind(n):r}}};function wB(t,e,n){if(!t||!A(t))return t;for(let r,i=0,o=kB.length;it&&t.$tupleid?ya:t));return e.fn[n]||(e.fn[n]=Q(r,t.$order,e.expr.codegen))}},{key:\"$context\",parse:function(t,e){return e}},{key:\"$subflow\",parse:function(t,e){const n=t.$subflow;return function(t,r,i){const o=e.fork().parse(n),a=o.get(n.operators[0].id),s=o.signals.parent;return s&&s.set(i),a.detachSubflow=()=>e.detach(o),a}}},{key:\"$tupleid\",parse:function(){return ya}}];const AB={skip:!0};function MB(t,e,n,r){return new EB(t,e,n,r)}function EB(t,e,n,r){this.dataflow=t,this.transforms=e,this.events=t.events.bind(t),this.expr=r||bB,this.signals={},this.scales={},this.nodes={},this.data={},this.fn={},n&&(this.functions=Object.create(n),this.functions.context=this)}function DB(t){this.dataflow=t.dataflow,this.transforms=t.transforms,this.events=t.events,this.expr=t.expr,this.signals=Object.create(t.signals),this.scales=Object.create(t.scales),this.nodes=Object.create(t.nodes),this.data=Object.create(t.data),this.fn=Object.create(t.fn),t.functions&&(this.functions=Object.create(t.functions),this.functions.context=this)}function CB(t,e){t&&(null==e?t.removeAttribute(\"aria-label\"):t.setAttribute(\"aria-label\",e))}EB.prototype=DB.prototype={fork(){const t=new DB(this);return(this.subcontext||(this.subcontext=[])).push(t),t},detach(t){this.subcontext=this.subcontext.filter((e=>e!==t));const e=Object.keys(t.nodes);for(const n of e)t.nodes[n]._targets=null;for(const n of e)t.nodes[n].detach();t.nodes=null},get(t){return this.nodes[t]},set(t,e){return this.nodes[t]=e},add(t,e){const n=this,r=n.dataflow,i=t.value;if(n.set(t.id,e),function(t){return\"collect\"===_B(t)}(t.type)&&i&&(i.$ingest?r.ingest(e,i.$ingest,i.$format):i.$request?r.preload(e,i.$request,i.$format):r.pulse(e,r.changeset().insert(i))),t.root&&(n.root=e),t.parent){let i=n.get(t.parent.$ref);i?(r.connect(i,[e]),e.targets().add(i)):(n.unresolved=n.unresolved||[]).push((()=>{i=n.get(t.parent.$ref),r.connect(i,[e]),e.targets().add(i)}))}if(t.signal&&(n.signals[t.signal]=e),t.scale&&(n.scales[t.scale]=e),t.data)for(const r in t.data){const i=n.data[r]||(n.data[r]={});t.data[r].forEach((t=>i[t]=e))}},resolve(){return(this.unresolved||[]).forEach((t=>t())),delete this.unresolved,this},operator(t,e){this.add(t,this.dataflow.add(t.value,e))},transform(t,e){this.add(t,this.dataflow.add(this.transforms[_B(e)]))},stream(t,e){this.set(t.id,e)},update(t,e,n,r,i){this.dataflow.on(e,n,r,i,t.options)},operatorExpression(t){return this.expr.operator(this,t)},parameterExpression(t){return this.expr.parameter(this,t)},eventExpression(t){return this.expr.event(this,t)},handlerExpression(t){return this.expr.handler(this,t)},encodeExpression(t){return this.expr.encode(this,t)},parse:function(t){const e=this,n=t.operators||[];return t.background&&(e.background=t.background),t.eventConfig&&(e.eventConfig=t.eventConfig),t.locale&&(e.locale=t.locale),n.forEach((t=>e.parseOperator(t))),n.forEach((t=>e.parseOperatorParameters(t))),(t.streams||[]).forEach((t=>e.parseStream(t))),(t.updates||[]).forEach((t=>e.parseUpdate(t))),e.resolve()},parseOperator:function(t){const e=this;!function(t){return\"operator\"===_B(t)}(t.type)&&t.type?e.transform(t,t.type):e.operator(t,t.update?e.operatorExpression(t.update):null)},parseOperatorParameters:function(t){const e=this;if(t.params){const n=e.get(t.id);n||s(\"Invalid operator id: \"+t.id),e.dataflow.connect(n,n.parameters(e.parseParameters(t.params),t.react,t.initonly))}},parseParameters:function(t,e){e=e||{};const n=this;for(const r in t){const i=t[r];e[r]=k(i)?i.map((t=>wB(t,n,e))):wB(i,n,e)}return e},parseStream:function(t){var e,n=this,r=null!=t.filter?n.eventExpression(t.filter):void 0,i=null!=t.stream?n.get(t.stream):void 0;t.source?i=n.events(t.source,t.type,r):t.merge&&(i=(e=t.merge.map((t=>n.get(t))))[0].merge.apply(e[0],e.slice(1))),t.between&&(e=t.between.map((t=>n.get(t))),i=i.between(e[0],e[1])),t.filter&&(i=i.filter(r)),null!=t.throttle&&(i=i.throttle(+t.throttle)),null!=t.debounce&&(i=i.debounce(+t.debounce)),null==i&&s(\"Invalid stream definition: \"+JSON.stringify(t)),t.consume&&i.consume(!0),n.stream(t,i)},parseUpdate:function(t){var e,n=this,r=A(r=t.source)?r.$ref:r,i=n.get(r),o=t.update,a=void 0;i||s(\"Source not defined: \"+t.source),e=t.target&&t.target.$expr?n.eventExpression(t.target.$expr):n.get(t.target),o&&o.$expr&&(o.$params&&(a=n.parseParameters(o.$params)),o=n.handlerExpression(o.$expr)),n.update(t,i,e,o,a)},getState:function(t){var e=this,n={};if(t.signals){var r=n.signals={};Object.keys(e.signals).forEach((n=>{const i=e.signals[n];t.signals(n,i)&&(r[n]=i.value)}))}if(t.data){var i=n.data={};Object.keys(e.data).forEach((n=>{const r=e.data[n];t.data(n,r)&&(i[n]=r.input.value)}))}return e.subcontext&&!1!==t.recurse&&(n.subcontext=e.subcontext.map((e=>e.getState(t)))),n},setState:function(t){var e=this,n=e.dataflow,r=t.data,i=t.signals;Object.keys(i||{}).forEach((t=>{n.update(e.signals[t],i[t],AB)})),Object.keys(r||{}).forEach((t=>{n.pulse(e.data[t].input,n.changeset().remove(p).insert(r[t]))})),(t.subcontext||[]).forEach(((t,n)=>{const r=e.subcontext[n];r&&r.setState(t)}))}};const FB=\"default\";function SB(t,e){const n=t.globalCursor()?\"undefined\"!=typeof document&&document.body:t.container();if(n)return null==e?n.style.removeProperty(\"cursor\"):n.style.cursor=e}function $B(t,e){var n=t._runtime.data;return lt(n,e)||s(\"Unrecognized data set: \"+e),n[e]}function TB(t,e){Aa(e)||s(\"Second argument to changes must be a changeset.\");const n=$B(this,t);return n.modified=!0,this.pulse(n.input,e)}function BB(t){var e=t.padding();return Math.max(0,t._viewWidth+e.left+e.right)}function zB(t){var e=t.padding();return Math.max(0,t._viewHeight+e.top+e.bottom)}function NB(t){var e=t.padding(),n=t._origin;return[e.left+n[0],e.top+n[1]]}function OB(t,e,n){var r,i,o=t._renderer,a=o&&o.canvas();return a&&(i=NB(t),(r=Xy(e.changedTouches?e.changedTouches[0]:e,a))[0]-=i[0],r[1]-=i[1]),e.dataflow=t,e.item=n,e.vega=function(t,e,n){const r=e?\"group\"===e.mark.marktype?e:e.mark.group:null;function i(t){var n,i=r;if(t)for(n=e;n;n=n.mark.group)if(n.mark.name===t){i=n;break}return i&&i.mark&&i.mark.interactive?i:{}}function o(t){if(!t)return n;xt(t)&&(t=i(t));const e=n.slice();for(;t;)e[0]-=t.x||0,e[1]-=t.y||0,t=t.mark&&t.mark.group;return e}return{view:rt(t),item:rt(e||{}),group:i,xy:o,x:t=>o(t)[0],y:t=>o(t)[1]}}(t,n,r),e}const RB=\"view\",UB={trap:!1};function LB(t,e,n,r){t._eventListeners.push({type:n,sources:V(e),handler:r})}function qB(t,e,n){const r=t._eventConfig&&t._eventConfig[e];return!(!1===r||A(r)&&!r[n])||(t.warn(`Blocked ${e} ${n} event listener.`),!1)}function PB(t){return t.item}function jB(t){return t.item.mark.source}function IB(t){return function(e,n){return n.vega.view().changeset().encode(n.item,t)}}function WB(t,e,n){const r=document.createElement(t);for(const t in e)r.setAttribute(t,e[t]);return null!=n&&(r.textContent=n),r}const HB=\"vega-bind\",YB=\"vega-bind-name\",GB=\"vega-bind-radio\";function VB(t,e,n,r){const i=n.event||\"input\",o=()=>t.update(e.value);r.signal(n.signal,e.value),e.addEventListener(i,o),LB(r,e,i,o),t.set=t=>{e.value=t,e.dispatchEvent(function(t){return\"undefined\"!=typeof Event?new Event(t):{type:t}}(i))}}function XB(t,e,n,r){const i=r.signal(n.signal),o=WB(\"div\",{class:HB}),a=\"radio\"===n.input?o:o.appendChild(WB(\"label\"));a.appendChild(WB(\"span\",{class:YB},n.name||n.signal)),e.appendChild(o);let s=JB;switch(n.input){case\"checkbox\":s=ZB;break;case\"select\":s=QB;break;case\"radio\":s=KB;break;case\"range\":s=tz}s(t,a,n,i)}function JB(t,e,n,r){const i=WB(\"input\");for(const t in n)\"signal\"!==t&&\"element\"!==t&&i.setAttribute(\"input\"===t?\"type\":t,n[t]);i.setAttribute(\"name\",n.signal),i.value=r,e.appendChild(i),i.addEventListener(\"input\",(()=>t.update(i.value))),t.elements=[i],t.set=t=>i.value=t}function ZB(t,e,n,r){const i={type:\"checkbox\",name:n.signal};r&&(i.checked=!0);const o=WB(\"input\",i);e.appendChild(o),o.addEventListener(\"change\",(()=>t.update(o.checked))),t.elements=[o],t.set=t=>o.checked=!!t||null}function QB(t,e,n,r){const i=WB(\"select\",{name:n.signal}),o=n.labels||[];n.options.forEach(((t,e)=>{const n={value:t};ez(t,r)&&(n.selected=!0),i.appendChild(WB(\"option\",n,(o[e]||t)+\"\"))})),e.appendChild(i),i.addEventListener(\"change\",(()=>{t.update(n.options[i.selectedIndex])})),t.elements=[i],t.set=t=>{for(let e=0,r=n.options.length;e{const s={type:\"radio\",name:n.signal,value:e};ez(e,r)&&(s.checked=!0);const u=WB(\"input\",s);u.addEventListener(\"change\",(()=>t.update(e)));const l=WB(\"label\",{},(o[a]||e)+\"\");return l.prepend(u),i.appendChild(l),u})),t.set=e=>{const n=t.elements,r=n.length;for(let t=0;t{u.textContent=s.value,t.update(+s.value)};s.addEventListener(\"input\",l),s.addEventListener(\"change\",l),t.elements=[s],t.set=t=>{s.value=t,u.textContent=t}}function ez(t,e){return t===e||t+\"\"==e+\"\"}function nz(t,e,n,r,i,o){return(e=e||new r(t.loader())).initialize(n,BB(t),zB(t),NB(t),i,o).background(t.background())}function rz(t,e){return e?function(){try{e.apply(this,arguments)}catch(e){t.error(e)}}:null}function iz(t,e,n){if(\"string\"==typeof e){if(\"undefined\"==typeof document)return t.error(\"DOM document instance not found.\"),null;if(!(e=document.querySelector(e)))return t.error(\"Signal bind element not found: \"+e),null}if(e&&n)try{e.textContent=\"\"}catch(n){e=null,t.error(n)}return e}const oz=t=>+t||0;function az(t){return A(t)?{top:oz(t.top),bottom:oz(t.bottom),left:oz(t.left),right:oz(t.right)}:(t=>({top:t,bottom:t,left:t,right:t}))(oz(t))}async function sz(t,e,n,r){const i=b_(e),o=i&&i.headless;return o||s(\"Unrecognized renderer type: \"+e),await t.runAsync(),nz(t,null,null,o,n,r).renderAsync(t._scenegraph.root)}var uz=\"width\",lz=\"height\",cz=\"padding\",fz={skip:!0};function hz(t,e){var n=t.autosize(),r=t.padding();return e-(n&&n.contains===cz?r.left+r.right:0)}function dz(t,e){var n=t.autosize(),r=t.padding();return e-(n&&n.contains===cz?r.top+r.bottom:0)}function pz(t,e){return e.modified&&k(e.input.value)&&!t.startsWith(\"_:vega:_\")}function gz(t,e){return!(\"parent\"===t||e instanceof Za.proxy)}function mz(t,e,n,r){const i=t.element();i&&i.setAttribute(\"title\",function(t){return null==t?\"\":k(t)?yz(t):A(t)&&!mt(t)?(e=t,Object.keys(e).map((t=>{const n=e[t];return t+\": \"+(k(n)?yz(n):vz(n))})).join(\"\\n\")):t+\"\";var e}(r))}function yz(t){return\"[\"+t.map(vz).join(\", \")+\"]\"}function vz(t){return k(t)?\"[…]\":A(t)&&!mt(t)?\"{…}\":t}function _z(t,e){const n=this;if(e=e||{},Va.call(n),e.loader&&n.loader(e.loader),e.logger&&n.logger(e.logger),null!=e.logLevel&&n.logLevel(e.logLevel),e.locale||t.locale){const r=ot({},t.locale,e.locale);n.locale(Ro(r.number,r.time))}n._el=null,n._elBind=null,n._renderType=e.renderer||__.Canvas,n._scenegraph=new jy;const r=n._scenegraph.root;n._renderer=null,n._tooltip=e.tooltip||mz,n._redraw=!0,n._handler=(new vv).scene(r),n._globalCursor=!1,n._preventDefault=!1,n._timers=[],n._eventListeners=[],n._resizeListeners=[],n._eventConfig=function(t){const e=ot({defaults:{}},t),n=(t,e)=>{e.forEach((e=>{k(t[e])&&(t[e]=Bt(t[e]))}))};return n(e.defaults,[\"prevent\",\"allow\"]),n(e,[\"view\",\"window\",\"selector\"]),e}(t.eventConfig),n.globalCursor(n._eventConfig.globalCursor);const i=function(t,e,n){return MB(t,Za,lB,n).parse(e)}(n,t,e.expr);n._runtime=i,n._signals=i.signals,n._bind=(t.bindings||[]).map((t=>({state:null,param:ot({},t)}))),i.root&&i.root.set(r),r.source=i.data.root.input,n.pulse(i.data.root.input,n.changeset().insert(r.items)),n._width=n.width(),n._height=n.height(),n._viewWidth=hz(n,n._width),n._viewHeight=dz(n,n._height),n._origin=[0,0],n._resize=0,n._autosize=1,function(t){var e=t._signals,n=e[uz],r=e[lz],i=e[cz];function o(){t._autosize=t._resize=1}t._resizeWidth=t.add(null,(e=>{t._width=e.size,t._viewWidth=hz(t,e.size),o()}),{size:n}),t._resizeHeight=t.add(null,(e=>{t._height=e.size,t._viewHeight=dz(t,e.size),o()}),{size:r});const a=t.add(null,o,{pad:i});t._resizeWidth.rank=n.rank+1,t._resizeHeight.rank=r.rank+1,a.rank=i.rank+1}(n),function(t){t.add(null,(e=>(t._background=e.bg,t._resize=1,e.bg)),{bg:t._signals.background})}(n),function(t){const e=t._signals.cursor||(t._signals.cursor=t.add({user:FB,item:null}));t.on(t.events(\"view\",\"pointermove\"),e,((t,n)=>{const r=e.value,i=r?xt(r)?r:r.user:FB,o=n.item&&n.item.cursor||null;return r&&i===r.user&&o==r.item?r:{user:i,item:o}})),t.add(null,(function(e){let n=e.cursor,r=this.value;return xt(n)||(r=n.item,n=n.user),SB(t,n&&n!==FB?n:r||n),r}),{cursor:e})}(n),n.description(t.description),e.hover&&n.hover(),e.container&&n.initialize(e.container,e.bind),e.watchPixelRatio&&n._watchPixelRatio()}function xz(t,e){return lt(t._signals,e)?t._signals[e]:s(\"Unrecognized signal name: \"+Ct(e))}function bz(t,e){const n=(t._targets||[]).filter((t=>t._update&&t._update.handler===e));return n.length?n[0]:null}function wz(t,e,n,r){let i=bz(n,r);return i||(i=rz(t,(()=>r(e,n.value))),i.handler=r,t.on(n,null,i)),t}function kz(t,e,n){const r=bz(e,n);return r&&e._targets.remove(r),t}dt(_z,Va,{async evaluate(t,e,n){if(await Va.prototype.evaluate.call(this,t,e),this._redraw||this._resize)try{this._renderer&&(this._resize&&(this._resize=0,function(t){var e=NB(t),n=BB(t),r=zB(t);t._renderer.background(t.background()),t._renderer.resize(n,r,e),t._handler.origin(e),t._resizeListeners.forEach((e=>{try{e(n,r)}catch(e){t.error(e)}}))}(this)),await this._renderer.renderAsync(this._scenegraph.root)),this._redraw=!1}catch(t){this.error(t)}return n&&da(this,n),this},dirty(t){this._redraw=!0,this._renderer&&this._renderer.dirty(t)},description(t){if(arguments.length){const e=null!=t?t+\"\":null;return e!==this._desc&&CB(this._el,this._desc=e),this}return this._desc},container(){return this._el},scenegraph(){return this._scenegraph},origin(){return this._origin.slice()},signal(t,e,n){const r=xz(this,t);return 1===arguments.length?r.value:this.update(r,e,n)},width(t){return arguments.length?this.signal(\"width\",t):this.signal(\"width\")},height(t){return arguments.length?this.signal(\"height\",t):this.signal(\"height\")},padding(t){return arguments.length?this.signal(\"padding\",az(t)):az(this.signal(\"padding\"))},autosize(t){return arguments.length?this.signal(\"autosize\",t):this.signal(\"autosize\")},background(t){return arguments.length?this.signal(\"background\",t):this.signal(\"background\")},renderer(t){return arguments.length?(b_(t)||s(\"Unrecognized renderer type: \"+t),t!==this._renderType&&(this._renderType=t,this._resetRenderer()),this):this._renderType},tooltip(t){return arguments.length?(t!==this._tooltip&&(this._tooltip=t,this._resetRenderer()),this):this._tooltip},loader(t){return arguments.length?(t!==this._loader&&(Va.prototype.loader.call(this,t),this._resetRenderer()),this):this._loader},resize(){return this._autosize=1,this.touch(xz(this,\"autosize\"))},_resetRenderer(){this._renderer&&(this._renderer=null,this.initialize(this._el,this._elBind))},_resizeView:function(t,e,n,r,i,o){this.runAfter((a=>{let s=0;a._autosize=0,a.width()!==n&&(s=1,a.signal(uz,n,fz),a._resizeWidth.skip(!0)),a.height()!==r&&(s=1,a.signal(lz,r,fz),a._resizeHeight.skip(!0)),a._viewWidth!==t&&(a._resize=1,a._viewWidth=t),a._viewHeight!==e&&(a._resize=1,a._viewHeight=e),a._origin[0]===i[0]&&a._origin[1]===i[1]||(a._resize=1,a._origin=i),s&&a.run(\"enter\"),o&&a.runAfter((t=>t.resize()))}),!1,1)},addEventListener(t,e,n){let r=e;return n&&!1===n.trap||(r=rz(this,e),r.raw=e),this._handler.on(t,r),this},removeEventListener(t,e){for(var n,r,i=this._handler.handlers(t),o=i.length;--o>=0;)if(r=i[o].type,n=i[o].handler,t===r&&(e===n||e===n.raw)){this._handler.off(r,n);break}return this},addResizeListener(t){const e=this._resizeListeners;return e.includes(t)||e.push(t),this},removeResizeListener(t){var e=this._resizeListeners,n=e.indexOf(t);return n>=0&&e.splice(n,1),this},addSignalListener(t,e){return wz(this,t,xz(this,t),e)},removeSignalListener(t,e){return kz(this,xz(this,t),e)},addDataListener(t,e){return wz(this,t,$B(this,t).values,e)},removeDataListener(t,e){return kz(this,$B(this,t).values,e)},globalCursor(t){if(arguments.length){if(this._globalCursor!==!!t){const e=SB(this,null);this._globalCursor=!!t,e&&SB(this,e)}return this}return this._globalCursor},preventDefault(t){return arguments.length?(this._preventDefault=t,this):this._preventDefault},timer:function(t,e){this._timers.push(function(t,e,n){var r=new QE,i=e;return null==e?(r.restart(t,e,n),r):(r._restart=r.restart,r.restart=function(t,e,n){e=+e,n=null==n?JE():+n,r._restart((function o(a){a+=i,r._restart(o,i+=e,n),t(a)}),e,n)},r.restart(t,e,n),r)}((function(e){t({timestamp:Date.now(),elapsed:e})}),e))},events:function(t,e,n){var r,i=this,o=new Ba(n),a=function(n,r){i.runAsync(null,(()=>{t===RB&&function(t,e){var n=t._eventConfig.defaults,r=n.prevent,i=n.allow;return!1!==r&&!0!==i&&(!0===r||!1===i||(r?r[e]:i?!i[e]:t.preventDefault()))}(i,e)&&n.preventDefault(),o.receive(OB(i,n,r))}))};if(\"timer\"===t)qB(i,\"timer\",e)&&i.timer(a,e);else if(t===RB)qB(i,\"view\",e)&&i.addEventListener(e,a,UB);else if(\"window\"===t?qB(i,\"window\",e)&&\"undefined\"!=typeof window&&(r=[window]):\"undefined\"!=typeof document&&qB(i,\"selector\",e)&&(r=Array.from(document.querySelectorAll(t))),r){for(var s=0,u=r.length;s=0;)i[t].stop();for(t=o.length;--t>=0;)for(e=(n=o[t]).sources.length;--e>=0;)n.sources[e].removeEventListener(n.type,n.handler);return r&&r.call(this,this._handler,null,null,null),this},hover:function(t,e){return e=[e||\"update\",(t=[t||\"hover\"])[0]],this.on(this.events(\"view\",\"pointerover\",PB),jB,IB(t)),this.on(this.events(\"view\",\"pointerout\",PB),jB,IB(e)),this},data:function(t,e){return arguments.length<2?$B(this,t).values.value:TB.call(this,t,Ma().remove(p).insert(e))},change:TB,insert:function(t,e){return TB.call(this,t,Ma().insert(e))},remove:function(t,e){return TB.call(this,t,Ma().remove(e))},scale:function(t){var e=this._runtime.scales;return lt(e,t)||s(\"Unrecognized scale or projection: \"+t),e[t].value},initialize:function(t,e){const n=this,r=n._renderType,i=n._eventConfig.bind,o=b_(r);t=n._el=t?iz(n,t,!0):null,function(t){const e=t.container();e&&(e.setAttribute(\"role\",\"graphics-document\"),e.setAttribute(\"aria-roleDescription\",\"visualization\"),CB(e,t.description()))}(n),o||n.error(\"Unrecognized renderer type: \"+r);const a=o.handler||vv,s=t?o.renderer:o.headless;return n._renderer=s?nz(n,n._renderer,t,s):null,n._handler=function(t,e,n,r){const i=new r(t.loader(),rz(t,t.tooltip())).scene(t.scenegraph().root).initialize(n,NB(t),t);return e&&e.handlers().forEach((t=>{i.on(t.type,t.handler)})),i}(n,n._handler,t,a),n._redraw=!0,t&&\"none\"!==i&&(e=e?n._elBind=iz(n,e,!0):t.appendChild(WB(\"form\",{class:\"vega-bindings\"})),n._bind.forEach((t=>{t.param.element&&\"container\"!==i&&(t.element=iz(n,t.param.element,!!t.param.input))})),n._bind.forEach((t=>{!function(t,e,n){if(!e)return;const r=n.param;let i=n.state;i||(i=n.state={elements:null,active:!1,set:null,update:e=>{e!=t.signal(r.signal)&&t.runAsync(null,(()=>{i.source=!0,t.signal(r.signal,e)}))}},r.debounce&&(i.update=it(r.debounce,i.update))),(null==r.input&&r.element?VB:XB)(i,e,r,t),i.active||(t.on(t._signals[r.signal],null,(()=>{i.source?i.source=!1:i.set(t.signal(r.signal))})),i.active=!0)}(n,t.element||e,t)}))),n},toImageURL:async function(t,e){t!==__.Canvas&&t!==__.SVG&&t!==__.PNG&&s(\"Unrecognized image type: \"+t);const n=await sz(this,t,e);return t===__.SVG?function(t,e){const n=new Blob([t],{type:e});return window.URL.createObjectURL(n)}(n.svg(),\"image/svg+xml\"):n.canvas().toDataURL(\"image/png\")},toCanvas:async function(t,e){return(await sz(this,__.Canvas,t,e)).canvas()},toSVG:async function(t){return(await sz(this,__.SVG,t)).svg()},getState:function(t){return this._runtime.getState(t||{data:pz,signals:gz,recurse:!0})},setState:function(t){return this.runAsync(null,(e=>{e._trigger=!1,e._runtime.setState(t)}),(t=>{t._trigger=!0})),this},_watchPixelRatio:function(){if(\"canvas\"===this.renderer()&&this._renderer._canvas){let t=null;const e=()=>{null!=t&&t();const n=matchMedia(`(resolution: ${window.devicePixelRatio}dppx)`);n.addEventListener(\"change\",e),t=()=>{n.removeEventListener(\"change\",e)},this._renderer._canvas.getContext(\"2d\").pixelRatio=window.devicePixelRatio||1,this._redraw=!0,this._resize=1,this.resize().runAsync()};e()}}});const Az=\"view\",Mz=\"[\",Ez=\"]\",Dz=\"{\",Cz=\"}\",Fz=\":\",Sz=\",\",$z=\"@\",Tz=\">\",Bz=/[[\\]{}]/,zz={\"*\":1,arc:1,area:1,group:1,image:1,line:1,path:1,rect:1,rule:1,shape:1,symbol:1,text:1,trail:1};let Nz,Oz;function Rz(t,e,n){return Nz=e||Az,Oz=n||zz,Lz(t.trim()).map(qz)}function Uz(t,e,n,r,i){const o=t.length;let a,s=0;for(;e' after between selector: \"+t;n=n.map(qz);const i=qz(t.slice(1).trim());if(i.between)return{between:n,stream:i};i.between=n;return i}(t):function(t){const e={source:Nz},n=[];let r,i,o=[0,0],a=0,s=0,u=t.length,l=0;if(t[u-1]===Cz){if(l=t.lastIndexOf(Dz),!(l>=0))throw\"Unmatched right brace: \"+t;try{o=function(t){const e=t.split(Sz);if(!t.length||e.length>2)throw t;return e.map((e=>{const n=+e;if(n!=n)throw t;return n}))}(t.substring(l+1,u-1))}catch(e){throw\"Invalid throttle specification: \"+t}u=(t=t.slice(0,l).trim()).length,l=0}if(!u)throw t;t[0]===$z&&(a=++l);r=Uz(t,l,Fz),r1?(e.type=n[1],a?e.markname=n[0].slice(1):!function(t){return Oz[t]}(n[0])?e.source=n[0]:e.marktype=n[0]):e.type=n[0];\"!\"===e.type.slice(-1)&&(e.consume=!0,e.type=e.type.slice(0,-1));null!=i&&(e.filter=i);o[0]&&(e.throttle=o[0]);o[1]&&(e.debounce=o[1]);return e}(t)}function Pz(t){return A(t)?t:{type:t||\"pad\"}}const jz=t=>+t||0,Iz=t=>({top:t,bottom:t,left:t,right:t});function Wz(t){return A(t)?t.signal?t:{top:jz(t.top),bottom:jz(t.bottom),left:jz(t.left),right:jz(t.right)}:Iz(jz(t))}const Hz=t=>A(t)&&!k(t)?ot({},t):{value:t};function Yz(t,e,n,r){if(null!=n){return A(n)&&!k(n)||k(n)&&n.length&&A(n[0])?t.update[e]=n:t[r||\"enter\"][e]={value:n},1}return 0}function Gz(t,e,n){for(const n in e)Yz(t,n,e[n]);for(const e in n)Yz(t,e,n[e],\"update\")}function Vz(t,e,n){for(const r in e)n&<(n,r)||(t[r]=ot(t[r]||{},e[r]));return t}function Xz(t,e){return e&&(e.enter&&e.enter[t]||e.update&&e.update[t])}const Jz=\"mark\",Zz=\"frame\",Qz=\"scope\",Kz=\"axis\",tN=\"axis-domain\",eN=\"axis-grid\",nN=\"axis-label\",rN=\"axis-tick\",iN=\"axis-title\",oN=\"legend\",aN=\"legend-band\",sN=\"legend-entry\",uN=\"legend-gradient\",lN=\"legend-label\",cN=\"legend-symbol\",fN=\"legend-title\",hN=\"title\",dN=\"title-text\",pN=\"title-subtitle\";function gN(t,e,n){t[e]=n&&n.signal?{signal:n.signal}:{value:n}}const mN=t=>xt(t)?Ct(t):t.signal?`(${t.signal})`:xN(t);function yN(t){if(null!=t.gradient)return function(t){const e=[t.start,t.stop,t.count].map((t=>null==t?null:Ct(t)));for(;e.length&&null==F(e);)e.pop();return e.unshift(mN(t.gradient)),`gradient(${e.join(\",\")})`}(t);let e=t.signal?`(${t.signal})`:t.color?function(t){return t.c?vN(\"hcl\",t.h,t.c,t.l):t.h||t.s?vN(\"hsl\",t.h,t.s,t.l):t.l||t.a?vN(\"lab\",t.l,t.a,t.b):t.r||t.g||t.b?vN(\"rgb\",t.r,t.g,t.b):null}(t.color):null!=t.field?xN(t.field):void 0!==t.value?Ct(t.value):void 0;return null!=t.scale&&(e=function(t,e){const n=mN(t.scale);null!=t.range?e=`lerp(_range(${n}), ${+t.range})`:(void 0!==e&&(e=`_scale(${n}, ${e})`),t.band&&(e=(e?e+\"+\":\"\")+`_bandwidth(${n})`+(1==+t.band?\"\":\"*\"+_N(t.band)),t.extra&&(e=`(datum.extra ? _scale(${n}, datum.extra.value) : ${e})`)),null==e&&(e=\"0\"));return e}(t,e)),void 0===e&&(e=null),null!=t.exponent&&(e=`pow(${e},${_N(t.exponent)})`),null!=t.mult&&(e+=`*${_N(t.mult)}`),null!=t.offset&&(e+=`+${_N(t.offset)}`),t.round&&(e=`round(${e})`),e}const vN=(t,e,n,r)=>`(${t}(${[e,n,r].map(yN).join(\",\")})+'')`;function _N(t){return A(t)?\"(\"+yN(t)+\")\":t}function xN(t){return bN(A(t)?t:{datum:t})}function bN(t){let e,n,r;if(t.signal)e=\"datum\",r=t.signal;else if(t.group||t.parent){for(n=Math.max(1,t.level||1),e=\"item\";n-- >0;)e+=\".mark.group\";t.parent?(r=t.parent,e+=\".datum\"):r=t.group}else t.datum?(e=\"datum\",r=t.datum):s(\"Invalid field reference: \"+Ct(t));return t.signal||(r=xt(r)?u(r).map(Ct).join(\"][\"):bN(r)),e+\"[\"+r+\"]\"}function wN(t,e,n,r,i,o){const a={};(o=o||{}).encoders={$encode:a},t=function(t,e,n,r,i){const o={},a={};let s,u,l,c;for(u in u=\"lineBreak\",\"text\"!==e||null==i[u]||Xz(u,t)||gN(o,u,i[u]),(\"legend\"==n||String(n).startsWith(\"axis\"))&&(n=null),c=n===Zz?i.group:n===Jz?ot({},i.mark,i[e]):null,c)l=Xz(u,t)||(\"fill\"===u||\"stroke\"===u)&&(Xz(\"fill\",t)||Xz(\"stroke\",t)),l||gN(o,u,c[u]);for(u in V(r).forEach((e=>{const n=i.style&&i.style[e];for(const e in n)Xz(e,t)||gN(o,e,n[e])})),t=ot({},t),o)c=o[u],c.signal?(s=s||{})[u]=c:a[u]=c;return t.enter=ot(a,t.enter),s&&(t.update=ot(s,t.update)),t}(t,e,n,r,i.config);for(const n in t)a[n]=kN(t[n],e,o,i);return o}function kN(t,e,n,r){const i={},o={};for(const e in t)null!=t[e]&&(i[e]=AN((a=t[e],k(a)?function(t){let e=\"\";return t.forEach((t=>{const n=yN(t);e+=t.test?`(${t.test})?${n}:`:n})),\":\"===F(e)&&(e+=\"null\"),e}(a):yN(a)),r,n,o));var a;return{$expr:{marktype:e,channels:i},$fields:Object.keys(o),$output:Object.keys(t)}}function AN(t,e,n,r){const i=mB(t,e);return i.$fields.forEach((t=>r[t]=1)),ot(n,i.$params),i.$expr}const MN=\"outer\",EN=[\"value\",\"update\",\"init\",\"react\",\"bind\"];function DN(t,e){s(t+' for \"outer\" push: '+Ct(e))}function CN(t,e){const n=t.name;if(t.push===MN)e.signals[n]||DN(\"No prior signal definition\",n),EN.forEach((e=>{void 0!==t[e]&&DN(\"Invalid property \",e)}));else{const r=e.addSignal(n,t.value);!1===t.react&&(r.react=!1),t.bind&&e.addBinding(n,t.bind)}}function FN(t,e,n,r){this.id=-1,this.type=t,this.value=e,this.params=n,r&&(this.parent=r)}function SN(t,e,n,r){return new FN(t,e,n,r)}function $N(t,e){return SN(\"operator\",t,e)}function TN(t){const e={$ref:t.id};return t.id<0&&(t.refs=t.refs||[]).push(e),e}function BN(t,e){return e?{$field:t,$name:e}:{$field:t}}const zN=BN(\"key\");function NN(t,e){return{$compare:t,$order:e}}const ON=\"descending\";function RN(t,e){return(t&&t.signal?\"$\"+t.signal:t||\"\")+(t&&e?\"_\":\"\")+(e&&e.signal?\"$\"+e.signal:e||\"\")}const UN=\"scope\",LN=\"view\";function qN(t){return t&&t.signal}function PN(t){if(qN(t))return!0;if(A(t))for(const e in t)if(PN(t[e]))return!0;return!1}function jN(t,e){return null!=t?t:e}function IN(t){return t&&t.signal||t}const WN=\"timer\";function HN(t,e){return(t.merge?YN:t.stream?GN:t.type?VN:s(\"Invalid stream specification: \"+Ct(t)))(t,e)}function YN(t,e){const n=XN({merge:t.merge.map((t=>HN(t,e)))},t,e);return e.addStream(n).id}function GN(t,e){const n=XN({stream:HN(t.stream,e)},t,e);return e.addStream(n).id}function VN(t,e){let n;t.type===WN?(n=e.event(WN,t.throttle),t={between:t.between,filter:t.filter}):n=e.event(function(t){return t===UN?LN:t||LN}(t.source),t.type);const r=XN({stream:n},t,e);return 1===Object.keys(r).length?n:e.addStream(r).id}function XN(t,e,n){let r=e.between;return r&&(2!==r.length&&s('Stream \"between\" parameter must have 2 entries: '+Ct(e)),t.between=[HN(r[0],n),HN(r[1],n)]),r=e.filter?[].concat(e.filter):[],(e.marktype||e.markname||e.markrole)&&r.push(function(t,e,n){const r=\"event.item\";return r+(t&&\"*\"!==t?\"&&\"+r+\".mark.marktype==='\"+t+\"'\":\"\")+(n?\"&&\"+r+\".mark.role==='\"+n+\"'\":\"\")+(e?\"&&\"+r+\".mark.name==='\"+e+\"'\":\"\")}(e.marktype,e.markname,e.markrole)),e.source===UN&&r.push(\"inScope(event.item)\"),r.length&&(t.filter=mB(\"(\"+r.join(\")&&(\")+\")\",n).$expr),null!=(r=e.throttle)&&(t.throttle=+r),null!=(r=e.debounce)&&(t.debounce=+r),e.consume&&(t.consume=!0),t}const JN={code:\"_.$value\",ast:{type:\"Identifier\",value:\"value\"}};function ZN(t,e,n){const r=t.encode,i={target:n};let o=t.events,a=t.update,u=[];o||s(\"Signal update missing events specification.\"),xt(o)&&(o=Rz(o,e.isSubscope()?UN:LN)),o=V(o).filter((t=>t.signal||t.scale?(u.push(t),0):1)),u.length>1&&(u=[QN(u)]),o.length&&u.push(o.length>1?{merge:o}:o[0]),null!=r&&(a&&s(\"Signal encode and update are mutually exclusive.\"),a=\"encode(item(),\"+Ct(r)+\")\"),i.update=xt(a)?mB(a,e):null!=a.expr?mB(a.expr,e):null!=a.value?a.value:null!=a.signal?{$expr:JN,$params:{$value:e.signalRef(a.signal)}}:s(\"Invalid signal update specification.\"),t.force&&(i.options={force:!0}),u.forEach((t=>e.addUpdate(ot(function(t,e){return{source:t.signal?e.signalRef(t.signal):t.scale?e.scaleRef(t.scale):HN(t,e)}}(t,e),i))))}function QN(t){return{signal:\"[\"+t.map((t=>t.scale?'scale(\"'+t.scale+'\")':t.signal))+\"]\"}}const KN=t=>(e,n,r)=>SN(t,n,e||void 0,r),tO=KN(\"aggregate\"),eO=KN(\"axisticks\"),nO=KN(\"bound\"),rO=KN(\"collect\"),iO=KN(\"compare\"),oO=KN(\"datajoin\"),aO=KN(\"encode\"),sO=KN(\"expression\"),uO=KN(\"facet\"),lO=KN(\"field\"),cO=KN(\"key\"),fO=KN(\"legendentries\"),hO=KN(\"load\"),dO=KN(\"mark\"),pO=KN(\"multiextent\"),gO=KN(\"multivalues\"),mO=KN(\"overlap\"),yO=KN(\"params\"),vO=KN(\"prefacet\"),_O=KN(\"projection\"),xO=KN(\"proxy\"),bO=KN(\"relay\"),wO=KN(\"render\"),kO=KN(\"scale\"),AO=KN(\"sieve\"),MO=KN(\"sortitems\"),EO=KN(\"viewlayout\"),DO=KN(\"values\");let CO=0;const FO={min:\"min\",max:\"max\",count:\"sum\"};function SO(t,e){const n=e.getScale(t.name).params;let r;for(r in n.domain=zO(t.domain,t,e),null!=t.range&&(n.range=jO(t,e,n)),null!=t.interpolate&&function(t,e){e.interpolate=$O(t.type||t),null!=t.gamma&&(e.interpolateGamma=$O(t.gamma))}(t.interpolate,n),null!=t.nice&&(n.nice=function(t){return A(t)?{interval:$O(t.interval),step:$O(t.step)}:$O(t)}(t.nice)),null!=t.bins&&(n.bins=function(t,e){return t.signal||k(t)?TO(t,e):e.objectProperty(t)}(t.bins,e)),t)lt(n,r)||\"name\"===r||(n[r]=$O(t[r],e))}function $O(t,e){return A(t)?t.signal?e.signalRef(t.signal):s(\"Unsupported object: \"+Ct(t)):t}function TO(t,e){return t.signal?e.signalRef(t.signal):t.map((t=>$O(t,e)))}function BO(t){s(\"Can not find data set: \"+Ct(t))}function zO(t,e,n){if(t)return t.signal?n.signalRef(t.signal):(k(t)?NO:t.fields?RO:OO)(t,e,n);null==e.domainMin&&null==e.domainMax||s(\"No scale domain defined for domainMin/domainMax to override.\")}function NO(t,e,n){return t.map((t=>$O(t,n)))}function OO(t,e,n){const r=n.getData(t.data);return r||BO(t.data),Kd(e.type)?r.valuesRef(n,t.field,LO(t.sort,!1)):rp(e.type)?r.domainRef(n,t.field):r.extentRef(n,t.field)}function RO(t,e,n){const r=t.data,i=t.fields.reduce(((t,e)=>(e=xt(e)?{data:r,field:e}:k(e)||e.signal?function(t,e){const n=\"_:vega:_\"+CO++,r=rO({});if(k(t))r.value={$ingest:t};else if(t.signal){const i=\"setdata(\"+Ct(n)+\",\"+t.signal+\")\";r.params.input=e.signalRef(i)}return e.addDataPipeline(n,[r,AO({})]),{data:n,field:\"data\"}}(e,n):e,t.push(e),t)),[]);return(Kd(e.type)?UO:rp(e.type)?qO:PO)(t,n,i)}function UO(t,e,n){const r=LO(t.sort,!0);let i,o;const a=n.map((t=>{const n=e.getData(t.data);return n||BO(t.data),n.countsRef(e,t.field,r)})),s={groupby:zN,pulse:a};r&&(i=r.op||\"count\",o=r.field?RN(i,r.field):\"count\",s.ops=[FO[i]],s.fields=[e.fieldRef(o)],s.as=[o]),i=e.add(tO(s));const u=e.add(rO({pulse:TN(i)}));return o=e.add(DO({field:zN,sort:e.sortRef(r),pulse:TN(u)})),TN(o)}function LO(t,e){return t&&(t.field||t.op?t.field||\"count\"===t.op?e&&t.field&&t.op&&!FO[t.op]&&s(\"Multiple domain scales can not be sorted using \"+t.op):s(\"No field provided for sort aggregate op: \"+t.op):A(t)?t.field=\"key\":t={field:\"key\"}),t}function qO(t,e,n){const r=n.map((t=>{const n=e.getData(t.data);return n||BO(t.data),n.domainRef(e,t.field)}));return TN(e.add(gO({values:r})))}function PO(t,e,n){const r=n.map((t=>{const n=e.getData(t.data);return n||BO(t.data),n.extentRef(e,t.field)}));return TN(e.add(pO({extents:r})))}function jO(t,e,n){const r=e.config.range;let i=t.range;if(i.signal)return e.signalRef(i.signal);if(xt(i)){if(r&<(r,i))return jO(t=ot({},t,{range:r[i]}),e,n);\"width\"===i?i=[0,{signal:\"width\"}]:\"height\"===i?i=Kd(t.type)?[0,{signal:\"height\"}]:[{signal:\"height\"},0]:s(\"Unrecognized scale range value: \"+Ct(i))}else{if(i.scheme)return n.scheme=k(i.scheme)?TO(i.scheme,e):$O(i.scheme,e),i.extent&&(n.schemeExtent=TO(i.extent,e)),void(i.count&&(n.schemeCount=$O(i.count,e)));if(i.step)return void(n.rangeStep=$O(i.step,e));if(Kd(t.type)&&!k(i))return zO(i,t,e);k(i)||s(\"Unsupported range type: \"+Ct(i))}return i.map((t=>(k(t)?TO:$O)(t,e)))}function IO(t,e,n){return k(t)?t.map((t=>IO(t,e,n))):A(t)?t.signal?n.signalRef(t.signal):\"fit\"===e?t:s(\"Unsupported parameter object: \"+Ct(t)):t}const WO=\"top\",HO=\"left\",YO=\"right\",GO=\"bottom\",VO=\"center\",XO=\"vertical\",JO=\"start\",ZO=\"end\",QO=\"index\",KO=\"label\",tR=\"offset\",eR=\"perc\",nR=\"perc2\",rR=\"value\",iR=\"guide-label\",oR=\"guide-title\",aR=\"group-title\",sR=\"group-subtitle\",uR=\"symbol\",lR=\"gradient\",cR=\"discrete\",fR=\"size\",hR=[fR,\"shape\",\"fill\",\"stroke\",\"strokeWidth\",\"strokeDash\",\"opacity\"],dR={name:1,style:1,interactive:1},pR={value:0},gR={value:1},mR=\"group\",yR=\"rect\",vR=\"rule\",_R=\"symbol\",xR=\"text\";function bR(t){return t.type=mR,t.interactive=t.interactive||!1,t}function wR(t,e){const n=(n,r)=>jN(t[n],jN(e[n],r));return n.isVertical=n=>XO===jN(t.direction,e.direction||(n?e.symbolDirection:e.gradientDirection)),n.gradientLength=()=>jN(t.gradientLength,e.gradientLength||e.gradientWidth),n.gradientThickness=()=>jN(t.gradientThickness,e.gradientThickness||e.gradientHeight),n.entryColumns=()=>jN(t.columns,jN(e.columns,+n.isVertical(!0))),n}function kR(t,e){const n=e&&(e.update&&e.update[t]||e.enter&&e.enter[t]);return n&&n.signal?n:n?n.value:null}function AR(t,e,n){return`item.anchor === '${JO}' ? ${t} : item.anchor === '${ZO}' ? ${e} : ${n}`}const MR=AR(Ct(HO),Ct(YO),Ct(VO));function ER(t,e){return e?t?A(t)?Object.assign({},t,{offset:ER(t.offset,e)}):{value:t,offset:e}:e:t}function DR(t,e){return e?(t.name=e.name,t.style=e.style||t.style,t.interactive=!!e.interactive,t.encode=Vz(t.encode,e,dR)):t.interactive=!1,t}function CR(t,e,n,r){const i=wR(t,n),o=i.isVertical(),a=i.gradientThickness(),s=i.gradientLength();let u,l,c,f,h;o?(l=[0,1],c=[0,0],f=a,h=s):(l=[0,0],c=[1,0],f=s,h=a);const d={enter:u={opacity:pR,x:pR,y:pR,width:Hz(f),height:Hz(h)},update:ot({},u,{opacity:gR,fill:{gradient:e,start:l,stop:c}}),exit:{opacity:pR}};return Gz(d,{stroke:i(\"gradientStrokeColor\"),strokeWidth:i(\"gradientStrokeWidth\")},{opacity:i(\"gradientOpacity\")}),DR({type:yR,role:uN,encode:d},r)}function FR(t,e,n,r,i){const o=wR(t,n),a=o.isVertical(),s=o.gradientThickness(),u=o.gradientLength();let l,c,f,h,d=\"\";a?(l=\"y\",f=\"y2\",c=\"x\",h=\"width\",d=\"1-\"):(l=\"x\",f=\"x2\",c=\"y\",h=\"height\");const p={opacity:pR,fill:{scale:e,field:rR}};p[l]={signal:d+\"datum.\"+eR,mult:u},p[c]=pR,p[f]={signal:d+\"datum.\"+nR,mult:u},p[h]=Hz(s);const g={enter:p,update:ot({},p,{opacity:gR}),exit:{opacity:pR}};return Gz(g,{stroke:o(\"gradientStrokeColor\"),strokeWidth:o(\"gradientStrokeWidth\")},{opacity:o(\"gradientOpacity\")}),DR({type:yR,role:aN,key:rR,from:i,encode:g},r)}const SR=`datum.${eR}<=0?\"${HO}\":datum.${eR}>=1?\"${YO}\":\"${VO}\"`,$R=`datum.${eR}<=0?\"${GO}\":datum.${eR}>=1?\"${WO}\":\"middle\"`;function TR(t,e,n,r){const i=wR(t,e),o=i.isVertical(),a=Hz(i.gradientThickness()),s=i.gradientLength();let u,l,c,f,h=i(\"labelOverlap\"),d=\"\";const p={enter:u={opacity:pR},update:l={opacity:gR,text:{field:KO}},exit:{opacity:pR}};return Gz(p,{fill:i(\"labelColor\"),fillOpacity:i(\"labelOpacity\"),font:i(\"labelFont\"),fontSize:i(\"labelFontSize\"),fontStyle:i(\"labelFontStyle\"),fontWeight:i(\"labelFontWeight\"),limit:jN(t.labelLimit,e.gradientLabelLimit)}),o?(u.align={value:\"left\"},u.baseline=l.baseline={signal:$R},c=\"y\",f=\"x\",d=\"1-\"):(u.align=l.align={signal:SR},u.baseline={value:\"top\"},c=\"x\",f=\"y\"),u[c]=l[c]={signal:d+\"datum.\"+eR,mult:s},u[f]=l[f]=a,a.offset=jN(t.labelOffset,e.gradientLabelOffset)||0,h=h?{separation:i(\"labelSeparation\"),method:h,order:\"datum.\"+QO}:void 0,DR({type:xR,role:lN,style:iR,key:rR,from:r,encode:p,overlap:h},n)}function BR(t,e,n,r,i){const o=wR(t,e),a=n.entries,s=!(!a||!a.interactive),u=a?a.name:void 0,l=o(\"clipHeight\"),c=o(\"symbolOffset\"),f={data:\"value\"},h=`(${i}) ? datum.${tR} : datum.${fR}`,d=l?Hz(l):{field:fR},p=`datum.${QO}`,g=`max(1, ${i})`;let m,y,v,_,x;d.mult=.5,m={enter:y={opacity:pR,x:{signal:h,mult:.5,offset:c},y:d},update:v={opacity:gR,x:y.x,y:y.y},exit:{opacity:pR}};let b=null,w=null;t.fill||(b=e.symbolBaseFillColor,w=e.symbolBaseStrokeColor),Gz(m,{fill:o(\"symbolFillColor\",b),shape:o(\"symbolType\"),size:o(\"symbolSize\"),stroke:o(\"symbolStrokeColor\",w),strokeDash:o(\"symbolDash\"),strokeDashOffset:o(\"symbolDashOffset\"),strokeWidth:o(\"symbolStrokeWidth\")},{opacity:o(\"symbolOpacity\")}),hR.forEach((e=>{t[e]&&(v[e]=y[e]={scale:t[e],field:rR})}));const k=DR({type:_R,role:cN,key:rR,from:f,clip:!!l||void 0,encode:m},n.symbols),A=Hz(c);A.offset=o(\"labelOffset\"),m={enter:y={opacity:pR,x:{signal:h,offset:A},y:d},update:v={opacity:gR,text:{field:KO},x:y.x,y:y.y},exit:{opacity:pR}},Gz(m,{align:o(\"labelAlign\"),baseline:o(\"labelBaseline\"),fill:o(\"labelColor\"),fillOpacity:o(\"labelOpacity\"),font:o(\"labelFont\"),fontSize:o(\"labelFontSize\"),fontStyle:o(\"labelFontStyle\"),fontWeight:o(\"labelFontWeight\"),limit:o(\"labelLimit\")});const M=DR({type:xR,role:lN,style:iR,key:rR,from:f,encode:m},n.labels);return m={enter:{noBound:{value:!l},width:pR,height:l?Hz(l):pR,opacity:pR},exit:{opacity:pR},update:v={opacity:gR,row:{signal:null},column:{signal:null}}},o.isVertical(!0)?(_=`ceil(item.mark.items.length / ${g})`,v.row.signal=`${p}%${_}`,v.column.signal=`floor(${p} / ${_})`,x={field:[\"row\",p]}):(v.row.signal=`floor(${p} / ${g})`,v.column.signal=`${p} % ${g}`,x={field:p}),v.column.signal=`(${i})?${v.column.signal}:${p}`,bR({role:Qz,from:r={facet:{data:r,name:\"value\",groupby:QO}},encode:Vz(m,a,dR),marks:[k,M],name:u,interactive:s,sort:x})}const zR='item.orient === \"left\"',NR='item.orient === \"right\"',OR=`(${zR} || ${NR})`,RR=`datum.vgrad && ${OR}`,UR=AR('\"top\"','\"bottom\"','\"middle\"'),LR=`datum.vgrad && ${NR} ? (${AR('\"right\"','\"left\"','\"center\"')}) : (${OR} && !(datum.vgrad && ${zR})) ? \"left\" : ${MR}`,qR=`item._anchor || (${OR} ? \"middle\" : \"start\")`,PR=`${RR} ? (${zR} ? -90 : 90) : 0`,jR=`${OR} ? (datum.vgrad ? (${NR} ? \"bottom\" : \"top\") : ${UR}) : \"top\"`;function IR(t,e){let n;return A(t)&&(t.signal?n=t.signal:t.path?n=\"pathShape(\"+WR(t.path)+\")\":t.sphere&&(n=\"geoShape(\"+WR(t.sphere)+', {type: \"Sphere\"})')),n?e.signalRef(n):!!t}function WR(t){return A(t)&&t.signal?t.signal:Ct(t)}function HR(t){const e=t.role||\"\";return e.startsWith(\"axis\")||e.startsWith(\"legend\")||e.startsWith(\"title\")?e:t.type===mR?Qz:e||Jz}function YR(t){return{marktype:t.type,name:t.name||void 0,role:t.role||HR(t),zindex:+t.zindex||void 0,aria:t.aria,description:t.description}}function GR(t,e){return t&&t.signal?e.signalRef(t.signal):!1!==t}function VR(t,e){const n=Qa(t.type);n||s(\"Unrecognized transform type: \"+Ct(t.type));const r=SN(n.type.toLowerCase(),null,XR(n,t,e));return t.signal&&e.addSignal(t.signal,e.proxy(r)),r.metadata=n.metadata||{},r}function XR(t,e,n){const r={},i=t.params.length;for(let o=0;oQR(t,e,n)))):QR(t,r,n)}(t,e,n):\"projection\"===r?n.projectionRef(e[t.name]):t.array&&!qN(i)?i.map((e=>ZR(t,e,n))):ZR(t,i,n):void(t.required&&s(\"Missing required \"+Ct(e.type)+\" parameter: \"+Ct(t.name)))}function ZR(t,e,n){const r=t.type;if(qN(e))return nU(r)?s(\"Expression references can not be signals.\"):rU(r)?n.fieldRef(e):iU(r)?n.compareRef(e):n.signalRef(e.signal);{const i=t.expr||rU(r);return i&&KR(e)?n.exprRef(e.expr,e.as):i&&tU(e)?BN(e.field,e.as):nU(r)?mB(e,n):eU(r)?TN(n.getData(e).values):rU(r)?BN(e):iU(r)?n.compareRef(e):e}}function QR(t,e,n){const r=t.params.length;let i;for(let n=0;nt&&t.expr,tU=t=>t&&t.field,eU=t=>\"data\"===t,nU=t=>\"expr\"===t,rU=t=>\"field\"===t,iU=t=>\"compare\"===t;function oU(t,e){return t.$ref?t:t.data&&t.data.$ref?t.data:TN(e.getData(t.data).output)}function aU(t,e,n,r,i){this.scope=t,this.input=e,this.output=n,this.values=r,this.aggregate=i,this.index={}}function sU(t){return xt(t)?t:null}function uU(t,e,n){const r=RN(n.op,n.field);let i;if(e.ops){for(let t=0,n=e.as.length;tnull==t?\"null\":t)).join(\",\")+\"),0)\",e);u.update=l.$expr,u.params=l.$params}function fU(t,e){const n=HR(t),r=t.type===mR,i=t.from&&t.from.facet,o=t.overlap;let a,u,l,c,f,h,d,p=t.layout||n===Qz||n===Zz;const g=n===Jz||p||i,m=function(t,e,n){let r,i,o,a,u;return t?(r=t.facet)&&(e||s(\"Only group marks can be faceted.\"),null!=r.field?a=u=oU(r,n):(t.data?u=TN(n.getData(t.data).aggregate):(o=VR(ot({type:\"aggregate\",groupby:V(r.groupby)},r.aggregate),n),o.params.key=n.keyRef(r.groupby),o.params.pulse=oU(r,n),a=u=TN(n.add(o))),i=n.keyRef(r.groupby,!0))):a=TN(n.add(rO(null,[{}]))),a||(a=oU(t,n)),{key:i,pulse:a,parent:u}}(t.from,r,e);u=e.add(oO({key:m.key||(t.key?BN(t.key):void 0),pulse:m.pulse,clean:!r}));const y=TN(u);u=l=e.add(rO({pulse:y})),u=e.add(dO({markdef:YR(t),interactive:GR(t.interactive,e),clip:IR(t.clip,e),context:{$context:!0},groups:e.lookup(),parent:e.signals.parent?e.signalRef(\"parent\"):null,index:e.markpath(),pulse:TN(u)}));const v=TN(u);u=c=e.add(aO(wN(t.encode,t.type,n,t.style,e,{mod:!1,pulse:v}))),u.params.parent=e.encode(),t.transform&&t.transform.forEach((t=>{const n=VR(t,e),r=n.metadata;(r.generates||r.changes)&&s(\"Mark transforms should not generate new data.\"),r.nomod||(c.params.mod=!0),n.params.pulse=TN(u),e.add(u=n)})),t.sort&&(u=e.add(MO({sort:e.compareRef(t.sort),pulse:TN(u)})));const _=TN(u);(i||p)&&(p=e.add(EO({layout:e.objectProperty(t.layout),legends:e.legends,mark:v,pulse:_})),h=TN(p));const x=e.add(nO({mark:v,pulse:h||_}));d=TN(x),r&&(g&&(a=e.operators,a.pop(),p&&a.pop()),e.pushState(_,h||d,y),i?function(t,e,n){const r=t.from.facet,i=r.name,o=oU(r,e);let a;r.name||s(\"Facet must have a name: \"+Ct(r)),r.data||s(\"Facet must reference a data set: \"+Ct(r)),r.field?a=e.add(vO({field:e.fieldRef(r.field),pulse:o})):r.groupby?a=e.add(uO({key:e.keyRef(r.groupby),group:TN(e.proxy(n.parent)),pulse:o})):s(\"Facet must specify groupby or field: \"+Ct(r));const u=e.fork(),l=u.add(rO()),c=u.add(AO({pulse:TN(l)}));u.addData(i,new aU(u,l,l,c)),u.addSignal(\"parent\",null),a.params.subflow={$subflow:u.parse(t).toRuntime()}}(t,e,m):g?function(t,e,n){const r=e.add(vO({pulse:n.pulse})),i=e.fork();i.add(AO()),i.addSignal(\"parent\",null),r.params.subflow={$subflow:i.parse(t).toRuntime()}}(t,e,m):e.parse(t),e.popState(),g&&(p&&a.push(p),a.push(x))),o&&(d=function(t,e,n){const r=t.method,i=t.bound,o=t.separation,a={separation:qN(o)?n.signalRef(o.signal):o,method:qN(r)?n.signalRef(r.signal):r,pulse:e};t.order&&(a.sort=n.compareRef({field:t.order}));if(i){const t=i.tolerance;a.boundTolerance=qN(t)?n.signalRef(t.signal):+t,a.boundScale=n.scaleRef(i.scale),a.boundOrient=i.orient}return TN(n.add(mO(a)))}(o,d,e));const b=e.add(wO({pulse:d})),w=e.add(AO({pulse:TN(b)},void 0,e.parent()));null!=t.name&&(f=t.name,e.addData(f,new aU(e,l,b,w)),t.on&&t.on.forEach((t=>{(t.insert||t.remove||t.toggle)&&s(\"Marks only support modify triggers.\"),cU(t,e,f)})))}function hU(t,e){const n=e.config.legend,r=t.encode||{},i=wR(t,n),o=r.legend||{},a=o.name||void 0,u=o.interactive,l=o.style,c={};let f,h,d,p=0;hR.forEach((e=>t[e]?(c[e]=t[e],p=p||t[e]):0)),p||s(\"Missing valid scale for legend.\");const g=function(t,e){let n=t.type||uR;t.type||1!==function(t){return hR.reduce(((e,n)=>e+(t[n]?1:0)),0)}(t)||!t.fill&&!t.stroke||(n=Qd(e)?lR:tp(e)?cR:uR);return n!==lR?n:tp(e)?cR:lR}(t,e.scaleType(p)),m={title:null!=t.title,scales:c,type:g,vgrad:\"symbol\"!==g&&i.isVertical()},y=TN(e.add(rO(null,[m]))),v=TN(e.add(fO(h={type:g,scale:e.scaleRef(p),count:e.objectProperty(i(\"tickCount\")),limit:e.property(i(\"symbolLimit\")),values:e.objectProperty(t.values),minstep:e.property(t.tickMinStep),formatType:e.property(t.formatType),formatSpecifier:e.property(t.format)})));return g===lR?(d=[CR(t,p,n,r.gradient),TR(t,n,r.labels,v)],h.count=h.count||e.signalRef(`max(2,2*floor((${IN(i.gradientLength())})/100))`)):g===cR?d=[FR(t,p,n,r.gradient,v),TR(t,n,r.labels,v)]:(f=function(t,e){const n=wR(t,e);return{align:n(\"gridAlign\"),columns:n.entryColumns(),center:{row:!0,column:!1},padding:{row:n(\"rowPadding\"),column:n(\"columnPadding\")}}}(t,n),d=[BR(t,n,r,v,IN(f.columns))],h.size=function(t,e,n){const r=IN(pU(\"size\",t,n)),i=IN(pU(\"strokeWidth\",t,n)),o=IN(function(t,e,n){return kR(\"fontSize\",t)||function(t,e,n){const r=e.config.style[n];return r&&r[t]}(\"fontSize\",e,n)}(n[1].encode,e,iR));return mB(`max(ceil(sqrt(${r})+${i}),${o})`,e)}(t,e,d[0].marks)),d=[bR({role:sN,from:y,encode:{enter:{x:{value:0},y:{value:0}}},marks:d,layout:f,interactive:u})],m.title&&d.push(function(t,e,n,r){const i=wR(t,e),o={enter:{opacity:pR},update:{opacity:gR,x:{field:{group:\"padding\"}},y:{field:{group:\"padding\"}}},exit:{opacity:pR}};return Gz(o,{orient:i(\"titleOrient\"),_anchor:i(\"titleAnchor\"),anchor:{signal:qR},angle:{signal:PR},align:{signal:LR},baseline:{signal:jR},text:t.title,fill:i(\"titleColor\"),fillOpacity:i(\"titleOpacity\"),font:i(\"titleFont\"),fontSize:i(\"titleFontSize\"),fontStyle:i(\"titleFontStyle\"),fontWeight:i(\"titleFontWeight\"),limit:i(\"titleLimit\"),lineHeight:i(\"titleLineHeight\")},{align:i(\"titleAlign\"),baseline:i(\"titleBaseline\")}),DR({type:xR,role:fN,style:oR,from:r,encode:o},n)}(t,n,r.title,y)),fU(bR({role:oN,from:y,encode:Vz(dU(i,t,n),o,dR),marks:d,aria:i(\"aria\"),description:i(\"description\"),zindex:i(\"zindex\"),name:a,interactive:u,style:l}),e)}function dU(t,e,n){const r={enter:{},update:{}};return Gz(r,{orient:t(\"orient\"),offset:t(\"offset\"),padding:t(\"padding\"),titlePadding:t(\"titlePadding\"),cornerRadius:t(\"cornerRadius\"),fill:t(\"fillColor\"),stroke:t(\"strokeColor\"),strokeWidth:n.strokeWidth,strokeDash:n.strokeDash,x:t(\"legendX\"),y:t(\"legendY\"),format:e.format,formatType:e.formatType}),r}function pU(t,e,n){return e[t]?`scale(\"${e[t]}\",datum)`:kR(t,n[0].encode)}aU.fromEntries=function(t,e){const n=e.length,r=e[n-1],i=e[n-2];let o=e[0],a=null,s=1;for(o&&\"load\"===o.type&&(o=e[1]),t.add(e[0]);s{n.push(VR(t,e))})),t.on&&t.on.forEach((n=>{cU(n,e,t.name)})),e.addDataPipeline(t.name,function(t,e,n){const r=[];let i,o,a,s,u,l=null,c=!1,f=!1;t.values?qN(t.values)||PN(t.format)?(r.push(xU(e,t)),r.push(l=_U())):r.push(l=_U({$ingest:t.values,$format:t.format})):t.url?PN(t.url)||PN(t.format)?(r.push(xU(e,t)),r.push(l=_U())):r.push(l=_U({$request:t.url,$format:t.format})):t.source&&(l=i=V(t.source).map((t=>TN(e.getData(t).output))),r.push(null));for(o=0,a=n.length;ot===GO||t===WO,wU=(t,e,n)=>qN(t)?FU(t.signal,e,n):t===HO||t===WO?e:n,kU=(t,e,n)=>qN(t)?DU(t.signal,e,n):bU(t)?e:n,AU=(t,e,n)=>qN(t)?CU(t.signal,e,n):bU(t)?n:e,MU=(t,e,n)=>qN(t)?SU(t.signal,e,n):t===WO?{value:e}:{value:n},EU=(t,e,n)=>qN(t)?$U(t.signal,e,n):t===YO?{value:e}:{value:n},DU=(t,e,n)=>TU(`${t} === '${WO}' || ${t} === '${GO}'`,e,n),CU=(t,e,n)=>TU(`${t} !== '${WO}' && ${t} !== '${GO}'`,e,n),FU=(t,e,n)=>zU(`${t} === '${HO}' || ${t} === '${WO}'`,e,n),SU=(t,e,n)=>zU(`${t} === '${WO}'`,e,n),$U=(t,e,n)=>zU(`${t} === '${YO}'`,e,n),TU=(t,e,n)=>(e=null!=e?Hz(e):e,n=null!=n?Hz(n):n,BU(e)&&BU(n)?{signal:`${t} ? (${e=e?e.signal||Ct(e.value):null}) : (${n=n?n.signal||Ct(n.value):null})`}:[ot({test:t},e)].concat(n||[])),BU=t=>null==t||1===Object.keys(t).length,zU=(t,e,n)=>({signal:`${t} ? (${OU(e)}) : (${OU(n)})`}),NU=(t,e,n,r,i)=>({signal:(null!=r?`${t} === '${HO}' ? (${OU(r)}) : `:\"\")+(null!=n?`${t} === '${GO}' ? (${OU(n)}) : `:\"\")+(null!=i?`${t} === '${YO}' ? (${OU(i)}) : `:\"\")+(null!=e?`${t} === '${WO}' ? (${OU(e)}) : `:\"\")+\"(null)\"}),OU=t=>qN(t)?t.signal:null==t?null:Ct(t),RU=(t,e)=>0===e?0:qN(t)?{signal:`(${t.signal}) * ${e}`}:{value:t*e},UU=(t,e)=>{const n=t.signal;return n&&n.endsWith(\"(null)\")?{signal:n.slice(0,-6)+e.signal}:t};function LU(t,e,n,r){let i;if(e&<(e,t))return e[t];if(lt(n,t))return n[t];if(t.startsWith(\"title\")){switch(t){case\"titleColor\":i=\"fill\";break;case\"titleFont\":case\"titleFontSize\":case\"titleFontWeight\":i=t[5].toLowerCase()+t.slice(6)}return r[oR][i]}if(t.startsWith(\"label\")){switch(t){case\"labelColor\":i=\"fill\";break;case\"labelFont\":case\"labelFontSize\":i=t[5].toLowerCase()+t.slice(6)}return r[iR][i]}return null}function qU(t){const e={};for(const n of t)if(n)for(const t in n)e[t]=1;return Object.keys(e)}function PU(t,e){return{scale:t.scale,range:e}}function jU(t,e,n,r,i){const o=wR(t,e),a=t.orient,s=t.gridScale,u=wU(a,1,-1),l=function(t,e){if(1===e);else if(A(t)){let n=t=ot({},t);for(;null!=n.mult;){if(!A(n.mult))return n.mult=qN(e)?{signal:`(${n.mult}) * (${e.signal})`}:n.mult*e,t;n=n.mult=ot({},n.mult)}n.mult=e}else t=qN(e)?{signal:`(${e.signal}) * (${t||0})`}:e*(t||0);return t}(t.offset,u);let c,f,h;const d={enter:c={opacity:pR},update:h={opacity:gR},exit:f={opacity:pR}};Gz(d,{stroke:o(\"gridColor\"),strokeCap:o(\"gridCap\"),strokeDash:o(\"gridDash\"),strokeDashOffset:o(\"gridDashOffset\"),strokeOpacity:o(\"gridOpacity\"),strokeWidth:o(\"gridWidth\")});const p={scale:t.scale,field:rR,band:i.band,extra:i.extra,offset:i.offset,round:o(\"tickRound\")},g=kU(a,{signal:\"height\"},{signal:\"width\"}),m=s?{scale:s,range:0,mult:u,offset:l}:{value:0,offset:l},y=s?{scale:s,range:1,mult:u,offset:l}:ot(g,{mult:u,offset:l});return c.x=h.x=kU(a,p,m),c.y=h.y=AU(a,p,m),c.x2=h.x2=AU(a,y),c.y2=h.y2=kU(a,y),f.x=kU(a,p),f.y=AU(a,p),DR({type:vR,role:eN,key:rR,from:r,encode:d},n)}function IU(t,e,n,r,i){return{signal:'flush(range(\"'+t+'\"), scale(\"'+t+'\", datum.value), '+e+\",\"+n+\",\"+r+\",\"+i+\")\"}}function WU(t,e,n,r){const i=wR(t,e),o=t.orient,a=wU(o,-1,1);let s,u;const l={enter:s={opacity:pR,anchor:Hz(i(\"titleAnchor\",null)),align:{signal:MR}},update:u=ot({},s,{opacity:gR,text:Hz(t.title)}),exit:{opacity:pR}},c={signal:`lerp(range(\"${t.scale}\"), ${AR(0,1,.5)})`};return u.x=kU(o,c),u.y=AU(o,c),s.angle=kU(o,pR,RU(a,90)),s.baseline=kU(o,MU(o,GO,WO),{value:GO}),u.angle=s.angle,u.baseline=s.baseline,Gz(l,{fill:i(\"titleColor\"),fillOpacity:i(\"titleOpacity\"),font:i(\"titleFont\"),fontSize:i(\"titleFontSize\"),fontStyle:i(\"titleFontStyle\"),fontWeight:i(\"titleFontWeight\"),limit:i(\"titleLimit\"),lineHeight:i(\"titleLineHeight\")},{align:i(\"titleAlign\"),angle:i(\"titleAngle\"),baseline:i(\"titleBaseline\")}),function(t,e,n,r){const i=(t,e)=>null!=t?(n.update[e]=UU(Hz(t),n.update[e]),!1):!Xz(e,r),o=i(t(\"titleX\"),\"x\"),a=i(t(\"titleY\"),\"y\");n.enter.auto=a===o?Hz(a):kU(e,Hz(a),Hz(o))}(i,o,l,n),l.update.align=UU(l.update.align,s.align),l.update.angle=UU(l.update.angle,s.angle),l.update.baseline=UU(l.update.baseline,s.baseline),DR({type:xR,role:iN,style:oR,from:r,encode:l},n)}function HU(t,e){const n=function(t,e){var n,r,i,o=e.config,a=o.style,s=o.axis,u=\"band\"===e.scaleType(t.scale)&&o.axisBand,l=t.orient;if(qN(l)){const t=qU([o.axisX,o.axisY]),e=qU([o.axisTop,o.axisBottom,o.axisLeft,o.axisRight]);for(i of(n={},t))n[i]=kU(l,LU(i,o.axisX,s,a),LU(i,o.axisY,s,a));for(i of(r={},e))r[i]=NU(l.signal,LU(i,o.axisTop,s,a),LU(i,o.axisBottom,s,a),LU(i,o.axisLeft,s,a),LU(i,o.axisRight,s,a))}else n=l===WO||l===GO?o.axisX:o.axisY,r=o[\"axis\"+l[0].toUpperCase()+l.slice(1)];return n||r||u?ot({},s,n,r,u):s}(t,e),r=t.encode||{},i=r.axis||{},o=i.name||void 0,a=i.interactive,s=i.style,u=wR(t,n),l=function(t){const e=t(\"tickBand\");let n,r,i=t(\"tickOffset\");return e?e.signal?(n={signal:`(${e.signal}) === 'extent' ? 1 : 0.5`},r={signal:`(${e.signal}) === 'extent'`},A(i)||(i={signal:`(${e.signal}) === 'extent' ? 0 : ${i}`})):\"extent\"===e?(n=1,r=!0,i=0):(n=.5,r=!1):(n=t(\"bandPosition\"),r=t(\"tickExtra\")),{extra:r,band:n,offset:i}}(u),c={scale:t.scale,ticks:!!u(\"ticks\"),labels:!!u(\"labels\"),grid:!!u(\"grid\"),domain:!!u(\"domain\"),title:null!=t.title},f=TN(e.add(rO({},[c]))),h=TN(e.add(eO({scale:e.scaleRef(t.scale),extra:e.property(l.extra),count:e.objectProperty(t.tickCount),values:e.objectProperty(t.values),minstep:e.property(t.tickMinStep),formatType:e.property(t.formatType),formatSpecifier:e.property(t.format)}))),d=[];let p;return c.grid&&d.push(jU(t,n,r.grid,h,l)),c.ticks&&(p=u(\"tickSize\"),d.push(function(t,e,n,r,i,o){const a=wR(t,e),s=t.orient,u=wU(s,-1,1);let l,c,f;const h={enter:l={opacity:pR},update:f={opacity:gR},exit:c={opacity:pR}};Gz(h,{stroke:a(\"tickColor\"),strokeCap:a(\"tickCap\"),strokeDash:a(\"tickDash\"),strokeDashOffset:a(\"tickDashOffset\"),strokeOpacity:a(\"tickOpacity\"),strokeWidth:a(\"tickWidth\")});const d=Hz(i);d.mult=u;const p={scale:t.scale,field:rR,band:o.band,extra:o.extra,offset:o.offset,round:a(\"tickRound\")};return f.y=l.y=kU(s,pR,p),f.y2=l.y2=kU(s,d),c.x=kU(s,p),f.x=l.x=AU(s,pR,p),f.x2=l.x2=AU(s,d),c.y=AU(s,p),DR({type:vR,role:rN,key:rR,from:r,encode:h},n)}(t,n,r.ticks,h,p,l))),c.labels&&(p=c.ticks?p:0,d.push(function(t,e,n,r,i,o){const a=wR(t,e),s=t.orient,u=t.scale,l=wU(s,-1,1),c=IN(a(\"labelFlush\")),f=IN(a(\"labelFlushOffset\")),h=a(\"labelAlign\"),d=a(\"labelBaseline\");let p,g=0===c||!!c;const m=Hz(i);m.mult=l,m.offset=Hz(a(\"labelPadding\")||0),m.offset.mult=l;const y={scale:u,field:rR,band:.5,offset:ER(o.offset,a(\"labelOffset\"))},v=kU(s,g?IU(u,c,'\"left\"','\"right\"','\"center\"'):{value:\"center\"},EU(s,\"left\",\"right\")),_=kU(s,MU(s,\"bottom\",\"top\"),g?IU(u,c,'\"top\"','\"bottom\"','\"middle\"'):{value:\"middle\"}),x=IU(u,c,`-(${f})`,f,0);g=g&&f;const b={opacity:pR,x:kU(s,y,m),y:AU(s,y,m)},w={enter:b,update:p={opacity:gR,text:{field:KO},x:b.x,y:b.y,align:v,baseline:_},exit:{opacity:pR,x:b.x,y:b.y}};Gz(w,{dx:!h&&g?kU(s,x):null,dy:!d&&g?AU(s,x):null}),Gz(w,{angle:a(\"labelAngle\"),fill:a(\"labelColor\"),fillOpacity:a(\"labelOpacity\"),font:a(\"labelFont\"),fontSize:a(\"labelFontSize\"),fontWeight:a(\"labelFontWeight\"),fontStyle:a(\"labelFontStyle\"),limit:a(\"labelLimit\"),lineHeight:a(\"labelLineHeight\")},{align:h,baseline:d});const k=a(\"labelBound\");let A=a(\"labelOverlap\");return A=A||k?{separation:a(\"labelSeparation\"),method:A,order:\"datum.index\",bound:k?{scale:u,orient:s,tolerance:k}:null}:void 0,p.align!==v&&(p.align=UU(p.align,v)),p.baseline!==_&&(p.baseline=UU(p.baseline,_)),DR({type:xR,role:nN,style:iR,key:rR,from:r,encode:w,overlap:A},n)}(t,n,r.labels,h,p,l))),c.domain&&d.push(function(t,e,n,r){const i=wR(t,e),o=t.orient;let a,s;const u={enter:a={opacity:pR},update:s={opacity:gR},exit:{opacity:pR}};Gz(u,{stroke:i(\"domainColor\"),strokeCap:i(\"domainCap\"),strokeDash:i(\"domainDash\"),strokeDashOffset:i(\"domainDashOffset\"),strokeWidth:i(\"domainWidth\"),strokeOpacity:i(\"domainOpacity\")});const l=PU(t,0),c=PU(t,1);return a.x=s.x=kU(o,l,pR),a.x2=s.x2=kU(o,c),a.y=s.y=AU(o,l,pR),a.y2=s.y2=AU(o,c),DR({type:vR,role:tN,from:r,encode:u},n)}(t,n,r.domain,f)),c.title&&d.push(WU(t,n,r.title,f)),fU(bR({role:Kz,from:f,encode:Vz(YU(u,t),i,dR),marks:d,aria:u(\"aria\"),description:u(\"description\"),zindex:u(\"zindex\"),name:o,interactive:a,style:s}),e)}function YU(t,e){const n={enter:{},update:{}};return Gz(n,{orient:t(\"orient\"),offset:t(\"offset\")||0,position:jN(e.position,0),titlePadding:t(\"titlePadding\"),minExtent:t(\"minExtent\"),maxExtent:t(\"maxExtent\"),range:{signal:`abs(span(range(\"${e.scale}\")))`},translate:t(\"translate\"),format:e.format,formatType:e.formatType}),n}function GU(t,e,n){const r=V(t.signals),i=V(t.scales);return n||r.forEach((t=>CN(t,e))),V(t.projections).forEach((t=>function(t,e){const n=e.config.projection||{},r={};for(const n in t)\"name\"!==n&&(r[n]=IO(t[n],n,e));for(const t in n)null==r[t]&&(r[t]=IO(n[t],t,e));e.addProjection(t.name,r)}(t,e))),i.forEach((t=>function(t,e){const n=t.type||\"linear\";Jd(n)||s(\"Unrecognized scale type: \"+Ct(n)),e.addScale(t.name,{type:n,domain:void 0})}(t,e))),V(t.data).forEach((t=>vU(t,e))),i.forEach((t=>SO(t,e))),(n||r).forEach((t=>function(t,e){const n=e.getSignal(t.name);let r=t.update;t.init&&(r?s(\"Signals can not include both init and update expressions.\"):(r=t.init,n.initonly=!0)),r&&(r=mB(r,e),n.update=r.$expr,n.params=r.$params),t.on&&t.on.forEach((t=>ZN(t,e,n.id)))}(t,e))),V(t.axes).forEach((t=>HU(t,e))),V(t.marks).forEach((t=>fU(t,e))),V(t.legends).forEach((t=>hU(t,e))),t.title&&mU(t.title,e),e.parseLambdas(),e}const VU=t=>Vz({enter:{x:{value:0},y:{value:0}},update:{width:{signal:\"width\"},height:{signal:\"height\"}}},t);function XU(t,e){const n=e.config,r=TN(e.root=e.add($N())),i=function(t,e){const n=n=>jN(t[n],e[n]),r=[JU(\"background\",n(\"background\")),JU(\"autosize\",Pz(n(\"autosize\"))),JU(\"padding\",Wz(n(\"padding\"))),JU(\"width\",n(\"width\")||0),JU(\"height\",n(\"height\")||0)],i=r.reduce(((t,e)=>(t[e.name]=e,t)),{}),o={};return V(t.signals).forEach((t=>{lt(i,t.name)?t=ot(i[t.name],t):r.push(t),o[t.name]=t})),V(e.signals).forEach((t=>{lt(o,t.name)||lt(i,t.name)||r.push(t)})),r}(t,n);i.forEach((t=>CN(t,e))),e.description=t.description||n.description,e.eventConfig=n.events,e.legends=e.objectProperty(n.legend&&n.legend.layout),e.locale=n.locale;const o=e.add(rO()),a=e.add(aO(wN(VU(t.encode),mR,Zz,t.style,e,{pulse:TN(o)}))),s=e.add(EO({layout:e.objectProperty(t.layout),legends:e.legends,autosize:e.signalRef(\"autosize\"),mark:r,pulse:TN(a)}));e.operators.pop(),e.pushState(TN(a),TN(s),null),GU(t,e,i),e.operators.push(s);let u=e.add(nO({mark:r,pulse:TN(s)}));return u=e.add(wO({pulse:TN(u)})),u=e.add(AO({pulse:TN(u)})),e.addData(\"root\",new aU(e,o,o,u)),e}function JU(t,e){return e&&e.signal?{name:t,update:e.signal}:{name:t,value:e}}function ZU(t,e){this.config=t||{},this.options=e||{},this.bindings=[],this.field={},this.signals={},this.lambdas={},this.scales={},this.events={},this.data={},this.streams=[],this.updates=[],this.operators=[],this.eventConfig=null,this.locale=null,this._id=0,this._subid=0,this._nextsub=[0],this._parent=[],this._encode=[],this._lookup=[],this._markpath=[]}function QU(t){this.config=t.config,this.options=t.options,this.legends=t.legends,this.field=Object.create(t.field),this.signals=Object.create(t.signals),this.lambdas=Object.create(t.lambdas),this.scales=Object.create(t.scales),this.events=Object.create(t.events),this.data=Object.create(t.data),this.streams=[],this.updates=[],this.operators=[],this._id=0,this._subid=++t._nextsub[0],this._nextsub=t._nextsub,this._parent=t._parent.slice(),this._encode=t._encode.slice(),this._lookup=t._lookup.slice(),this._markpath=t._markpath}function KU(t){return(k(t)?tL:eL)(t)}function tL(t){const e=t.length;let n=\"[\";for(let r=0;r0?\",\":\"\")+(A(e)?e.signal||KU(e):Ct(e))}return n+\"]\"}function eL(t){let e,n,r=\"{\",i=0;for(e in t)n=t[e],r+=(++i>1?\",\":\"\")+Ct(e)+\":\"+(A(n)?n.signal||KU(n):Ct(n));return r+\"}\"}ZU.prototype=QU.prototype={parse(t){return GU(t,this)},fork(){return new QU(this)},isSubscope(){return this._subid>0},toRuntime(){return this.finish(),{description:this.description,operators:this.operators,streams:this.streams,updates:this.updates,bindings:this.bindings,eventConfig:this.eventConfig,locale:this.locale}},id(){return(this._subid?this._subid+\":\":0)+this._id++},add(t){return this.operators.push(t),t.id=this.id(),t.refs&&(t.refs.forEach((e=>{e.$ref=t.id})),t.refs=null),t},proxy(t){const e=t instanceof FN?TN(t):t;return this.add(xO({value:e}))},addStream(t){return this.streams.push(t),t.id=this.id(),t},addUpdate(t){return this.updates.push(t),t},finish(){let t,e;for(t in this.root&&(this.root.root=!0),this.signals)this.signals[t].signal=t;for(t in this.scales)this.scales[t].scale=t;function n(t,e,n){let r,i;t&&(r=t.data||(t.data={}),i=r[e]||(r[e]=[]),i.push(n))}for(t in this.data){e=this.data[t],n(e.input,t,\"input\"),n(e.output,t,\"output\"),n(e.values,t,\"values\");for(const r in e.index)n(e.index[r],t,\"index:\"+r)}return this},pushState(t,e,n){this._encode.push(TN(this.add(AO({pulse:t})))),this._parent.push(e),this._lookup.push(n?TN(this.proxy(n)):null),this._markpath.push(-1)},popState(){this._encode.pop(),this._parent.pop(),this._lookup.pop(),this._markpath.pop()},parent(){return F(this._parent)},encode(){return F(this._encode)},lookup(){return F(this._lookup)},markpath(){const t=this._markpath;return++t[t.length-1]},fieldRef(t,e){if(xt(t))return BN(t,e);t.signal||s(\"Unsupported field reference: \"+Ct(t));const n=t.signal;let r=this.field[n];if(!r){const t={name:this.signalRef(n)};e&&(t.as=e),this.field[n]=r=TN(this.add(lO(t)))}return r},compareRef(t){let e=!1;const n=t=>qN(t)?(e=!0,this.signalRef(t.signal)):function(t){return t&&t.expr}(t)?(e=!0,this.exprRef(t.expr)):t,r=V(t.field).map(n),i=V(t.order).map(n);return e?TN(this.add(iO({fields:r,orders:i}))):NN(r,i)},keyRef(t,e){let n=!1;const r=this.signals;return t=V(t).map((t=>qN(t)?(n=!0,TN(r[t.signal])):t)),n?TN(this.add(cO({fields:t,flat:e}))):function(t,e){const n={$key:t};return e&&(n.$flat=!0),n}(t,e)},sortRef(t){if(!t)return t;const e=RN(t.op,t.field),n=t.order||\"ascending\";return n.signal?TN(this.add(iO({fields:e,orders:this.signalRef(n.signal)}))):NN(e,n)},event(t,e){const n=t+\":\"+e;if(!this.events[n]){const r=this.id();this.streams.push({id:r,source:t,type:e}),this.events[n]=r}return this.events[n]},hasOwnSignal(t){return lt(this.signals,t)},addSignal(t,e){this.hasOwnSignal(t)&&s(\"Duplicate signal name: \"+Ct(t));const n=e instanceof FN?e:this.add($N(e));return this.signals[t]=n},getSignal(t){return this.signals[t]||s(\"Unrecognized signal name: \"+Ct(t)),this.signals[t]},signalRef(t){return this.signals[t]?TN(this.signals[t]):(lt(this.lambdas,t)||(this.lambdas[t]=this.add($N(null))),TN(this.lambdas[t]))},parseLambdas(){const t=Object.keys(this.lambdas);for(let e=0,n=t.length;er+Math.floor(o*t.random()),pdf:t=>t===Math.floor(t)&&t>=r&&t=i?1:(e-r+1)/o},icdf:t=>t>=0&&t<=1?r-1+Math.floor(t*o):NaN};return a.min(e).max(n)},t.randomKDE=gs,t.randomLCG=function(t){return function(){return(t=(1103515245*t+12345)%2147483647)/2147483647}},t.randomLogNormal=xs,t.randomMixture=bs,t.randomNormal=ps,t.randomUniform=Es,t.read=ca,t.regressionConstant=Ds,t.regressionExp=zs,t.regressionLinear=Ts,t.regressionLoess=Ls,t.regressionLog=Bs,t.regressionPoly=Rs,t.regressionPow=Ns,t.regressionQuad=Os,t.renderModule=b_,t.repeat=Mt,t.resetDefaultLocale=function(){return Co(),Bo(),Uo()},t.resetSVGClipId=Ng,t.resetSVGDefIds=function(){Ng(),Op=0},t.responseType=la,t.runtimeContext=MB,t.sampleCurve=Is,t.sampleLogNormal=ms,t.sampleNormal=cs,t.sampleUniform=ws,t.scale=Xd,t.sceneEqual=F_,t.sceneFromJSON=qy,t.scenePickVisit=Fm,t.sceneToJSON=Ly,t.sceneVisit=Cm,t.sceneZOrder=Dm,t.scheme=dp,t.serializeXML=Wv,t.setHybridRendererOptions=function(t){h_.svgMarkTypes=t.svgMarkTypes??[\"text\"],h_.svgOnTop=t.svgOnTop??!0,h_.debug=t.debug??!1},t.setRandom=function(e){t.random=e},t.span=Dt,t.splitAccessPath=u,t.stringValue=Ct,t.textMetrics=py,t.timeBin=Jr,t.timeFloor=wr,t.timeFormatLocale=No,t.timeInterval=Cr,t.timeOffset=$r,t.timeSequence=zr,t.timeUnitSpecifier=rr,t.timeUnits=er,t.toBoolean=Ft,t.toDate=$t,t.toNumber=S,t.toSet=Bt,t.toString=Tt,t.transform=Ka,t.transforms=Za,t.truncate=zt,t.truthy=p,t.tupleid=ya,t.typeParsers=Zo,t.utcFloor=Mr,t.utcInterval=Fr,t.utcOffset=Tr,t.utcSequence=Nr,t.utcdayofyear=hr,t.utcquarter=G,t.utcweek=dr,t.version=\"5.27.0\",t.visitArray=Nt,t.week=sr,t.writeConfig=D,t.zero=h,t.zoomLinear=j,t.zoomLog=I,t.zoomPow=W,t.zoomSymlog=H}));\n//# sourceMappingURL=vega.min.js.map\n" }, { "path": "docs/autogen_rst.py", "content": "import logging\nimport os\nimport shutil\nfrom pathlib import Path\n\nlog = logging.getLogger(os.path.basename(__file__))\n\n\ndef module_template(module_qualname: str):\n module_name = module_qualname.split(\".\")[-1]\n title = module_name.replace(\"_\", r\"\\_\")\n return f\"\"\"{title}\n{\"=\" * len(title)}\n\n.. automodule:: {module_qualname}\n :members:\n :undoc-members:\n\"\"\"\n\n\ndef index_template(package_name: str, doc_references: list[str] | None = None, text_prefix=\"\"):\n doc_references = doc_references or \"\"\n if doc_references:\n doc_references = \"\\n\" + \"\\n\".join(f\"* :doc:`{ref}`\" for ref in doc_references) + \"\\n\"\n\n dirname = package_name.split(\".\")[-1]\n title = dirname.replace(\"_\", r\"\\_\")\n if title == \"tianshou\":\n title = \"Tianshou API Reference\"\n return f\"{title}\\n{'=' * len(title)}\" + text_prefix + doc_references\n\n\ndef write_to_file(content: str, path: str):\n os.makedirs(os.path.dirname(path), exist_ok=True)\n with open(path, \"w\") as f:\n f.write(content)\n os.chmod(path, 0o666)\n\n\n_SUBTITLE = (\n \"\\n Here is the autogenerated documentation of the Tianshou API. \\n \\n \"\n \"The Table of Contents to the left has the same structure as the \"\n \"repository's package code. The links at each page point to the submodules and subpackages. \\n\\n \"\n \"Enjoy scrolling through! \\n\"\n)\n\n\ndef make_rst(src_root, rst_root, clean=False, overwrite=False, package_prefix=\"\"):\n \"\"\"Creates/updates documentation in form of rst files for modules and packages.\n\n Does not delete any existing rst files. Thus, rst files for packages or modules that have been removed or renamed\n should be deleted by hand.\n\n This method should be executed from the project's top-level directory\n\n :param src_root: path to library base directory, typically \"src/\"\n :param clean: whether to completely clean the target directory beforehand, removing any existing .rst files\n :param overwrite: whether to overwrite existing rst files. This should be used with caution as it will delete\n all manual changes to documentation files\n :package_prefix: a prefix to prepend to each module (for the case where the src_root is not the base package),\n which, if not empty, should end with a \".\"\n :return:\n \"\"\"\n rst_root = os.path.abspath(rst_root)\n\n if clean and os.path.isdir(rst_root):\n shutil.rmtree(rst_root)\n\n base_package_name = package_prefix + os.path.basename(src_root)\n\n # TODO: reduce duplication with same logic for subpackages below\n files_in_dir = os.listdir(src_root)\n module_names = [f[:-3] for f in files_in_dir if f.endswith(\".py\") and not f.startswith(\"_\")]\n subdir_refs = [\n f\"{f}/index\"\n for f in files_in_dir\n if os.path.isdir(os.path.join(src_root, f))\n and not f.startswith(\"_\")\n and not f.startswith(\".\")\n ]\n package_index_rst_path = os.path.join(\n rst_root,\n \"index.rst\",\n )\n log.info(f\"Writing {package_index_rst_path}\")\n write_to_file(\n index_template(\n base_package_name,\n doc_references=module_names + subdir_refs,\n text_prefix=_SUBTITLE,\n ),\n package_index_rst_path,\n )\n\n for root, dirnames, filenames in os.walk(src_root):\n if os.path.basename(root).startswith(\"_\"):\n continue\n base_package_relpath = os.path.relpath(root, start=src_root)\n base_package_qualname = package_prefix + os.path.relpath(\n root,\n start=os.path.dirname(src_root),\n ).replace(os.path.sep, \".\")\n\n for dirname in dirnames:\n if dirname.startswith(\"_\"):\n log.debug(f\"Skipping {dirname}\")\n continue\n files_in_dir = os.listdir(os.path.join(root, dirname))\n module_names = [\n f[:-3] for f in files_in_dir if f.endswith(\".py\") and not f.startswith(\"_\")\n ]\n subdir_refs = [\n f\"{f}/index\"\n for f in files_in_dir\n if os.path.isdir(os.path.join(root, dirname, f)) and not f.startswith(\"_\")\n ]\n if not module_names and \"__init__.py\" not in files_in_dir:\n log.debug(f\"Skipping {dirname} as it does not contain any modules or __init__.py\")\n continue\n package_qualname = f\"{base_package_qualname}.{dirname}\"\n package_index_rst_path = os.path.join(\n rst_root,\n base_package_relpath,\n dirname,\n \"index.rst\",\n )\n log.info(f\"Writing {package_index_rst_path}\")\n write_to_file(\n index_template(package_qualname, doc_references=module_names + subdir_refs),\n package_index_rst_path,\n )\n\n for filename in filenames:\n base_name, ext = os.path.splitext(filename)\n if ext == \".py\" and not filename.startswith(\"_\"):\n module_qualname = f\"{base_package_qualname}.{filename[:-3]}\"\n\n module_rst_path = os.path.join(rst_root, base_package_relpath, f\"{base_name}.rst\")\n if os.path.exists(module_rst_path) and not overwrite:\n log.debug(f\"{module_rst_path} already exists, skipping it\")\n\n log.info(f\"Writing module documentation to {module_rst_path}\")\n write_to_file(module_template(module_qualname), module_rst_path)\n\n\nif __name__ == \"__main__\":\n logging.basicConfig(level=logging.INFO)\n docs_root = Path(__file__).parent\n make_rst(\n docs_root / \"..\" / \"tianshou\",\n docs_root / \"03_api\",\n clean=True,\n )\n" }, { "path": "docs/bibtex.json", "content": "{\n \"cited\": {\n \"tutorials/dqn\": [\n \"DQN\",\n \"DDPG\",\n \"PPO\"\n ]\n }\n}" }, { "path": "docs/create_toc.py", "content": "import os\nfrom pathlib import Path\n\n# This script provides a platform-independent way of making the jupyter-book call (used in pyproject.toml)\ntoc_file = Path(__file__).parent / \"_toc.yml\"\ncmd = f'jupyter-book toc from-project docs -e .rst -e .md -e .ipynb >\"{toc_file}\"'\nprint(cmd)\nos.system(cmd)\n" }, { "path": "docs/index.rst", "content": "\nWelcome to Tianshou!\n====================\n\n**Tianshou** (`天授 `_) is a reinforcement learning platform based on pure PyTorch. Unlike existing reinforcement learning libraries, which are mainly based on TensorFlow, have many nested classes, unfriendly API, or slow-speed, Tianshou provides a fast-speed framework and pythonic API for building the deep reinforcement learning agent. The supported interface algorithms include:\n\n* :class:`~tianshou.algorithm.modelfree.dqn.DQN` `Deep Q-Network `_\n* :class:`~tianshou.algorithm.modelfree.dqn.DQN` `Double DQN `_\n* :class:`~tianshou.algorithm.modelfree.dqn.DQN` `Dueling DQN `_\n* :class:`~tianshou.algorithm.modelfree.bdqn.BDQN` `Branching DQN `_\n* :class:`~tianshou.algorithm.modelfree.c51.C51` `Categorical DQN `_\n* :class:`~tianshou.algorithm.modelfree.rainbow.RainbowDQN` `Rainbow DQN `_\n* :class:`~tianshou.algorithm.modelfree.qrdqn.QRDQN` `Quantile Regression DQN `_\n* :class:`~tianshou.algorithm.modelfree.iqn.IQN` `Implicit Quantile Network `_\n* :class:`~tianshou.algorithm.modelfree.fqf.FQF` `Fully-parameterized Quantile Function `_\n* :class:`~tianshou.algorithm.modelfree.reinforce.Reinforce` `Reinforce/Vanilla Policy Gradients `_\n* :class:`~tianshou.algorithm.modelfree.npg.NPG` `Natural Policy Gradient `_\n* :class:`~tianshou.algorithm.modelfree.a2c.A2C` `Advantage Actor-Critic `_\n* :class:`~tianshou.algorithm.modelfree.trpo.TRPO` `Trust Region Policy Optimization `_\n* :class:`~tianshou.algorithm.modelfree.ppo.PPO` `Proximal Policy Optimization `_\n* :class:`~tianshou.algorithm.modelfree.ddpg.DDPG` `Deep Deterministic Policy Gradient `_\n* :class:`~tianshou.algorithm.modelfree.td3.TD3` `Twin Delayed DDPG `_\n* :class:`~tianshou.algorithm.modelfree.sac.SAC` `Soft Actor-Critic `_\n* :class:`~tianshou.algorithm.modelfree.redq.REDQ` `Randomized Ensembled Double Q-Learning `_\n* :class:`~tianshou.algorithm.modelfree.discrete_sac.DiscreteSAC` `Discrete Soft Actor-Critic `_\n* :class:`~tianshou.algorithm.imitation.imitation_base.ImitationPolicy` Imitation Learning\n* :class:`~tianshou.algorithm.imitation.bcq.BCQ` `Batch-Constrained deep Q-Learning `_\n* :class:`~tianshou.algorithm.imitation.cql.CQL` `Conservative Q-Learning `_\n* :class:`~tianshou.algorithm.imitation.td3_bc.TD3BC` `Twin Delayed DDPG with Behavior Cloning `_\n* :class:`~tianshou.algorithm.imitation.discrete_cql.DiscreteCQL` `Discrete Conservative Q-Learning `_\n* :class:`~tianshou.algorithm.imitation.discrete_bcq.DiscreteBCQ` `Discrete Batch-Constrained deep Q-Learning `_\n* :class:`~tianshou.algorithm.imitation.discrete_crr.DiscreteCRR` `Critic Regularized Regression `_\n* :class:`~tianshou.algorithm.imitation.gail.GAIL` `Generative Adversarial Imitation Learning `_\n* :class:`~tianshou.algorithm.modelbased.psrl.PSRLPolicy` `Posterior Sampling Reinforcement Learning `_\n* :class:`~tianshou.algorithm.modelbased.icm.ICMOffPolicyWrapper`, :class:`~tianshou.algorithm.modelbased.icm.ICMOnPolicyWrapper` `Intrinsic Curiosity Module `_\n* :class:`~tianshou.data.buffer.prio.PrioritizedReplayBuffer` `Prioritized Experience Replay `_\n* :meth:`~tianshou.algorithm.algorithm_base.Algorithm.compute_episodic_return` `Generalized Advantage Estimator `_\n* :class:`~tianshou.data.buffer.her.HERReplayBuffer` `Hindsight Experience Replay `_\n\n\nInstallation\n------------\n\nTianshou is available through `PyPI `_.\nNew releases require Python >= 3.11.\n\nInstall Tianshou with the following command:\n\n.. code-block:: bash\n\n $ pip install tianshou\n\nAlternatively, install the current version on GitHub:\n\n.. code-block:: bash\n\n $ pip install git+https://github.com/thu-ml/tianshou.git@master --upgrade\n\nAfter installation, open your python console and type\n::\n\n import tianshou\n print(tianshou.__version__)\n\nIf no error occurs, you have successfully installed Tianshou.\n\n\nIndices and tables\n------------------\n\n* :ref:`genindex`\n* :ref:`modindex`\n* :ref:`search`\n" }, { "path": "docs/nbstripout.py", "content": "\"\"\"Implements a platform-independent way of calling nbstripout (used in pyproject.toml).\"\"\"\n\nimport glob\nimport os\nfrom pathlib import Path\n\nif __name__ == \"__main__\":\n docs_dir = Path(__file__).parent\n for path in glob.glob(str(docs_dir / \"02_notebooks\" / \"*.ipynb\")):\n cmd = f\"nbstripout {path}\"\n os.system(cmd)\n" }, { "path": "docs/refs.bib", "content": "@article{DQN,\n author = {Volodymyr Mnih and\n Koray Kavukcuoglu and\n David Silver and\n Andrei A. Rusu and\n Joel Veness and\n Marc G. Bellemare and\n Alex Graves and\n Martin A. Riedmiller and\n Andreas Fidjeland and\n Georg Ostrovski and\n Stig Petersen and\n Charles Beattie and\n Amir Sadik and\n Ioannis Antonoglou and\n Helen King and\n Dharshan Kumaran and\n Daan Wierstra and\n Shane Legg and\n Demis Hassabis},\n title = {Human-level control through deep reinforcement learning},\n journal = {Nature},\n volume = {518},\n number = {7540},\n pages = {529--533},\n year = {2015},\n url = {https://doi.org/10.1038/nature14236},\n doi = {10.1038/nature14236},\n timestamp = {Wed, 14 Nov 2018 10:30:43 +0100},\n biburl = {https://dblp.org/rec/journals/nature/MnihKSRVBGRFOPB15.bib},\n bibsource = {dblp computer science bibliography, https://dblp.org}\n}\n\n@inproceedings{DDPG,\n author = {Timothy P. Lillicrap and\n Jonathan J. Hunt and\n Alexander Pritzel and\n Nicolas Heess and\n Tom Erez and\n Yuval Tassa and\n David Silver and\n Daan Wierstra},\n title = {Continuous control with deep reinforcement learning},\n booktitle = {4th International Conference on Learning Representations, {ICLR} 2016,\n San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings},\n year = {2016},\n url = {http://arxiv.org/abs/1509.02971},\n timestamp = {Thu, 25 Jul 2019 14:25:37 +0200},\n biburl = {https://dblp.org/rec/journals/corr/LillicrapHPHETS15.bib},\n bibsource = {dblp computer science bibliography, https://dblp.org}\n}\n\n@article{PPO,\n author = {John Schulman and\n Filip Wolski and\n Prafulla Dhariwal and\n Alec Radford and\n Oleg Klimov},\n title = {Proximal Policy Optimization Algorithms},\n journal = {CoRR},\n volume = {abs/1707.06347},\n year = {2017},\n url = {http://arxiv.org/abs/1707.06347},\n archivePrefix = {arXiv},\n eprint = {1707.06347},\n timestamp = {Mon, 13 Aug 2018 16:47:34 +0200},\n biburl = {https://dblp.org/rec/journals/corr/SchulmanWDRK17.bib},\n bibsource = {dblp computer science bibliography, https://dblp.org}\n}\n" }, { "path": "examples/__init__.py", "content": "" }, { "path": "examples/atari/README.md", "content": "# Atari Environment\n\n## EnvPool\n\nWe highly recommend using envpool to run the following experiments. To install, in a linux machine, type:\n\n```bash\npip install envpool\n```\n\nAfter that, `atari_wrapper` will automatically switch to envpool's Atari env. EnvPool's implementation is much faster (\nabout 2\\~3x faster for pure execution speed, 1.5x for overall RL training pipeline) than python vectorized env\nimplementation, and it's behavior is consistent to that approach (OpenAI wrapper), which will describe below.\n\nFor more information, please refer to\nEnvPool's [GitHub](https://github.com/sail-sg/envpool/), [Docs](https://envpool.readthedocs.io/en/latest/api/atari.html),\nand [3rd-party report](https://iclr-blog-track.github.io/2022/03/25/ppo-implementation-details/#solving-pong-in-5-minutes-with-ppo--envpool).\n\n## ALE-py\n\nThe sample speed is \\~3000 env step per second (\\~12000 Atari frame per second in fact since we use frame_stack=4) under\nthe normal mode (use a CNN policy and a collector, also storing data into the buffer).\n\nThe env wrapper is a crucial thing. Without wrappers, the agent cannot perform well enough on Atari games. Many existing\nRL codebases use [OpenAI wrapper](https://github.com/openai/baselines/blob/master/baselines/common/atari_wrappers.py),\nbut it is not the original DeepMind version ([related issue](https://github.com/openai/baselines/issues/240)). Dopamine\nhas a different [wrapper](https://github.com/google/dopamine/blob/master/dopamine/discrete_domains/atari_lib.py) but\nunfortunately it cannot work very well in our codebase.\n\n# DQN (single run)\n\nOne epoch here is equal to 100,000 env step, 100 epochs stand for 10M.\n\n| task | best reward | reward curve | parameters | time cost |\n|-----------------------------|-------------|---------------------------------------|---------------------------------------------------------------------------------|---------------------|\n| PongNoFrameskip-v4 | 20 | ![](results/dqn/Pong_rew.png) | `python3 atari_dqn.py --task \"PongNoFrameskip-v4\" --batch_size 64` | ~30 min (~15 epoch) |\n| BreakoutNoFrameskip-v4 | 316 | ![](results/dqn/Breakout_rew.png) | `python3 atari_dqn.py --task \"BreakoutNoFrameskip-v4\" --num_test_envs 100` | 3~4h (100 epoch) |\n| EnduroNoFrameskip-v4 | 670 | ![](results/dqn/Enduro_rew.png) | `python3 atari_dqn.py --task \"EnduroNoFrameskip-v4 \" --num_test_envs 100` | 3~4h (100 epoch) |\n| QbertNoFrameskip-v4 | 7307 | ![](results/dqn/Qbert_rew.png) | `python3 atari_dqn.py --task \"QbertNoFrameskip-v4\" --num_test_envs 100` | 3~4h (100 epoch) |\n| MsPacmanNoFrameskip-v4 | 2107 | ![](results/dqn/MsPacman_rew.png) | `python3 atari_dqn.py --task \"MsPacmanNoFrameskip-v4\" --num_test_envs 100` | 3~4h (100 epoch) |\n| SeaquestNoFrameskip-v4 | 2088 | ![](results/dqn/Seaquest_rew.png) | `python3 atari_dqn.py --task \"SeaquestNoFrameskip-v4\" --num_test_envs 100` | 3~4h (100 epoch) |\n| SpaceInvadersNoFrameskip-v4 | 812.2 | ![](results/dqn/SpaceInvader_rew.png) | `python3 atari_dqn.py --task \"SpaceInvadersNoFrameskip-v4\" --num_test_envs 100` | 3~4h (100 epoch) |\n\nNote: The `eps_train_final` and `eps_test` in the original DQN paper is 0.1 and 0.01,\nbut [some works](https://github.com/google/dopamine/tree/master/baselines) found that smaller eps helps improve the\nperformance. Also, a large batchsize (say 64 instead of 32) will help faster convergence but will slow down the training\nspeed.\n\nWe haven't tuned this result to the best, so have fun with playing these hyperparameters!\n\n# C51 (single run)\n\nOne epoch here is equal to 100,000 env step, 100 epochs stand for 10M.\n\n| task | best reward | reward curve | parameters |\n|-----------------------------|-------------|---------------------------------------|--------------------------------------------------------------------|\n| PongNoFrameskip-v4 | 20 | ![](results/c51/Pong_rew.png) | `python3 atari_c51.py --task \"PongNoFrameskip-v4\" --batch_size 64` |\n| BreakoutNoFrameskip-v4 | 536.6 | ![](results/c51/Breakout_rew.png) | `python3 atari_c51.py --task \"BreakoutNoFrameskip-v4\" --n-step 1` |\n| EnduroNoFrameskip-v4 | 1032 | ![](results/c51/Enduro_rew.png) | `python3 atari_c51.py --task \"EnduroNoFrameskip-v4 \" ` |\n| QbertNoFrameskip-v4 | 16245 | ![](results/c51/Qbert_rew.png) | `python3 atari_c51.py --task \"QbertNoFrameskip-v4\"` |\n| MsPacmanNoFrameskip-v4 | 3133 | ![](results/c51/MsPacman_rew.png) | `python3 atari_c51.py --task \"MsPacmanNoFrameskip-v4\"` |\n| SeaquestNoFrameskip-v4 | 6226 | ![](results/c51/Seaquest_rew.png) | `python3 atari_c51.py --task \"SeaquestNoFrameskip-v4\"` |\n| SpaceInvadersNoFrameskip-v4 | 988.5 | ![](results/c51/SpaceInvader_rew.png) | `python3 atari_c51.py --task \"SpaceInvadersNoFrameskip-v4\"` |\n\nNote: The selection of `n_step` is based on Figure 6 in the [Rainbow](https://arxiv.org/abs/1710.02298) paper.\n\n# QRDQN (single run)\n\nOne epoch here is equal to 100,000 env step, 100 epochs stand for 10M.\n\n| task | best reward | reward curve | parameters |\n|-----------------------------|-------------|-----------------------------------------|----------------------------------------------------------------------|\n| PongNoFrameskip-v4 | 20 | ![](results/qrdqn/Pong_rew.png) | `python3 atari_qrdqn.py --task \"PongNoFrameskip-v4\" --batch_size 64` |\n| BreakoutNoFrameskip-v4 | 409.2 | ![](results/qrdqn/Breakout_rew.png) | `python3 atari_qrdqn.py --task \"BreakoutNoFrameskip-v4\" --n-step 1` |\n| EnduroNoFrameskip-v4 | 1055.9 | ![](results/qrdqn/Enduro_rew.png) | `python3 atari_qrdqn.py --task \"EnduroNoFrameskip-v4\"` |\n| QbertNoFrameskip-v4 | 14990 | ![](results/qrdqn/Qbert_rew.png) | `python3 atari_qrdqn.py --task \"QbertNoFrameskip-v4\"` |\n| MsPacmanNoFrameskip-v4 | 2886 | ![](results/qrdqn/MsPacman_rew.png) | `python3 atari_qrdqn.py --task \"MsPacmanNoFrameskip-v4\"` |\n| SeaquestNoFrameskip-v4 | 5676 | ![](results/qrdqn/Seaquest_rew.png) | `python3 atari_qrdqn.py --task \"SeaquestNoFrameskip-v4\"` |\n| SpaceInvadersNoFrameskip-v4 | 938 | ![](results/qrdqn/SpaceInvader_rew.png) | `python3 atari_qrdqn.py --task \"SpaceInvadersNoFrameskip-v4\"` |\n\n# IQN (single run)\n\nOne epoch here is equal to 100,000 env step, 100 epochs stand for 10M.\n\n| task | best reward | reward curve | parameters |\n|-----------------------------|-------------|----------------------------------------|--------------------------------------------------------------------|\n| PongNoFrameskip-v4 | 20.3 | ![](results/iqn/Pong_rew.png) | `python3 atari_iqn.py --task \"PongNoFrameskip-v4\" --batch_size 64` |\n| BreakoutNoFrameskip-v4 | 496.7 | ![](results/iqn/Breakout_rew.png) | `python3 atari_iqn.py --task \"BreakoutNoFrameskip-v4\" --n-step 1` |\n| EnduroNoFrameskip-v4 | 1545 | ![](results/iqn/Enduro_rew.png) | `python3 atari_iqn.py --task \"EnduroNoFrameskip-v4\"` |\n| QbertNoFrameskip-v4 | 15342.5 | ![](results/iqn/Qbert_rew.png) | `python3 atari_iqn.py --task \"QbertNoFrameskip-v4\"` |\n| MsPacmanNoFrameskip-v4 | 2915 | ![](results/iqn/MsPacman_rew.png) | `python3 atari_iqn.py --task \"MsPacmanNoFrameskip-v4\"` |\n| SeaquestNoFrameskip-v4 | 4874 | ![](results/iqn/Seaquest_rew.png) | `python3 atari_iqn.py --task \"SeaquestNoFrameskip-v4\"` |\n| SpaceInvadersNoFrameskip-v4 | 1498.5 | ![](results/iqn/SpaceInvaders_rew.png) | `python3 atari_iqn.py --task \"SpaceInvadersNoFrameskip-v4\"` |\n\n# FQF (single run)\n\nOne epoch here is equal to 100,000 env step, 100 epochs stand for 10M.\n\n| task | best reward | reward curve | parameters |\n|-----------------------------|-------------|----------------------------------------|--------------------------------------------------------------------|\n| PongNoFrameskip-v4 | 20.7 | ![](results/fqf/Pong_rew.png) | `python3 atari_fqf.py --task \"PongNoFrameskip-v4\" --batch_size 64` |\n| BreakoutNoFrameskip-v4 | 517.3 | ![](results/fqf/Breakout_rew.png) | `python3 atari_fqf.py --task \"BreakoutNoFrameskip-v4\" --n-step 1` |\n| EnduroNoFrameskip-v4 | 2240.5 | ![](results/fqf/Enduro_rew.png) | `python3 atari_fqf.py --task \"EnduroNoFrameskip-v4\"` |\n| QbertNoFrameskip-v4 | 16172.5 | ![](results/fqf/Qbert_rew.png) | `python3 atari_fqf.py --task \"QbertNoFrameskip-v4\"` |\n| MsPacmanNoFrameskip-v4 | 2429 | ![](results/fqf/MsPacman_rew.png) | `python3 atari_fqf.py --task \"MsPacmanNoFrameskip-v4\"` |\n| SeaquestNoFrameskip-v4 | 10775 | ![](results/fqf/Seaquest_rew.png) | `python3 atari_fqf.py --task \"SeaquestNoFrameskip-v4\"` |\n| SpaceInvadersNoFrameskip-v4 | 2482 | ![](results/fqf/SpaceInvaders_rew.png) | `python3 atari_fqf.py --task \"SpaceInvadersNoFrameskip-v4\"` |\n\n# Rainbow (single run)\n\nOne epoch here is equal to 100,000 env step, 100 epochs stand for 10M.\n\n| task | best reward | reward curve | parameters |\n|-----------------------------|-------------|--------------------------------------------|------------------------------------------------------------------------|\n| PongNoFrameskip-v4 | 21 | ![](results/rainbow/Pong_rew.png) | `python3 atari_rainbow.py --task \"PongNoFrameskip-v4\" --batch_size 64` |\n| BreakoutNoFrameskip-v4 | 684.6 | ![](results/rainbow/Breakout_rew.png) | `python3 atari_rainbow.py --task \"BreakoutNoFrameskip-v4\" --n-step 1` |\n| EnduroNoFrameskip-v4 | 1625.9 | ![](results/rainbow/Enduro_rew.png) | `python3 atari_rainbow.py --task \"EnduroNoFrameskip-v4\"` |\n| QbertNoFrameskip-v4 | 16192.5 | ![](results/rainbow/Qbert_rew.png) | `python3 atari_rainbow.py --task \"QbertNoFrameskip-v4\"` |\n| MsPacmanNoFrameskip-v4 | 3101 | ![](results/rainbow/MsPacman_rew.png) | `python3 atari_rainbow.py --task \"MsPacmanNoFrameskip-v4\"` |\n| SeaquestNoFrameskip-v4 | 2126 | ![](results/rainbow/Seaquest_rew.png) | `python3 atari_rainbow.py --task \"SeaquestNoFrameskip-v4\"` |\n| SpaceInvadersNoFrameskip-v4 | 1794.5 | ![](results/rainbow/SpaceInvaders_rew.png) | `python3 atari_rainbow.py --task \"SpaceInvadersNoFrameskip-v4\"` |\n\n# PPO (single run)\n\nOne epoch here is equal to 100,000 env step, 100 epochs stand for 10M.\n\n| task | best reward | reward curve | parameters |\n|-----------------------------|-------------|----------------------------------------|------------------------------------------------------------------|\n| PongNoFrameskip-v4 | 20.2 | ![](results/ppo/Pong_rew.png) | `python3 atari_ppo.py --task \"PongNoFrameskip-v4\"` |\n| BreakoutNoFrameskip-v4 | 441.8 | ![](results/ppo/Breakout_rew.png) | `python3 atari_ppo.py --task \"BreakoutNoFrameskip-v4\"` |\n| EnduroNoFrameskip-v4 | 1245.4 | ![](results/ppo/Enduro_rew.png) | `python3 atari_ppo.py --task \"EnduroNoFrameskip-v4\"` |\n| QbertNoFrameskip-v4 | 17395 | ![](results/ppo/Qbert_rew.png) | `python3 atari_ppo.py --task \"QbertNoFrameskip-v4\"` |\n| MsPacmanNoFrameskip-v4 | 2098 | ![](results/ppo/MsPacman_rew.png) | `python3 atari_ppo.py --task \"MsPacmanNoFrameskip-v4\"` |\n| SeaquestNoFrameskip-v4 | 882 | ![](results/ppo/Seaquest_rew.png) | `python3 atari_ppo.py --task \"SeaquestNoFrameskip-v4\" --lr 1e-4` |\n| SpaceInvadersNoFrameskip-v4 | 1340.5 | ![](results/ppo/SpaceInvaders_rew.png) | `python3 atari_ppo.py --task \"SpaceInvadersNoFrameskip-v4\"` |\n\n# SAC (single run)\n\nOne epoch here is equal to 100,000 env step, 100 epochs stand for 10M.\n\n| task | best reward | reward curve | parameters |\n|-----------------------------|-------------|-------------------------------------------------|----------------------------------------------------------------------------------------------------|\n| PongNoFrameskip-v4 | 20.1 | ![](results/discrete_sac/Pong_rew.png) | `python3 atari_sac.py --task \"PongNoFrameskip-v4\"` |\n| BreakoutNoFrameskip-v4 | 211.2 | ![](results/discrete_sac/Breakout_rew.png) | `python3 atari_sac.py --task \"BreakoutNoFrameskip-v4\" --n-step 1 --actor-lr 1e-4 --critic-lr 1e-4` |\n| EnduroNoFrameskip-v4 | 1290.7 | ![](results/discrete_sac/Enduro_rew.png) | `python3 atari_sac.py --task \"EnduroNoFrameskip-v4\"` |\n| QbertNoFrameskip-v4 | 13157.5 | ![](results/discrete_sac/Qbert_rew.png) | `python3 atari_sac.py --task \"QbertNoFrameskip-v4\"` |\n| MsPacmanNoFrameskip-v4 | 3836 | ![](results/discrete_sac/MsPacman_rew.png) | `python3 atari_sac.py --task \"MsPacmanNoFrameskip-v4\"` |\n| SeaquestNoFrameskip-v4 | 1772 | ![](results/discrete_sac/Seaquest_rew.png) | `python3 atari_sac.py --task \"SeaquestNoFrameskip-v4\"` |\n| SpaceInvadersNoFrameskip-v4 | 649 | ![](results/discrete_sac/SpaceInvaders_rew.png) | `python3 atari_sac.py --task \"SpaceInvadersNoFrameskip-v4\"` |\n" }, { "path": "examples/atari/__init__.py", "content": "" }, { "path": "examples/atari/atari_c51.py", "content": "#!/usr/bin/env python3\n\nimport datetime\nimport os\nimport pprint\nimport sys\n\nimport numpy as np\nimport torch\nfrom sensai.util import logging\n\nfrom tianshou.algorithm import C51\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.c51 import C51Policy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env.atari.atari_network import C51Net\nfrom tianshou.env.atari.atari_wrapper import make_atari_env\nfrom tianshou.highlevel.logger import LoggerFactoryDefault\nfrom tianshou.trainer import OffPolicyTrainerParams\n\nlog = logging.getLogger(__name__)\n\n\ndef main(\n task: str = \"PongNoFrameskip-v4\",\n seed: int = 0,\n scale_obs: int = 0,\n eps_test: float = 0.005,\n eps_train: float = 1.0,\n eps_train_final: float = 0.05,\n buffer_size: int = 100000,\n lr: float = 0.0001,\n gamma: float = 0.99,\n num_atoms: int = 51,\n v_min: float = -10.0,\n v_max: float = 10.0,\n n_step: int = 3,\n target_update_freq: int = 500,\n epoch: int = 100,\n epoch_num_steps: int = 100000,\n collection_step_num_env_steps: int = 10,\n update_per_step: float = 0.1,\n batch_size: int = 32,\n num_training_envs: int = 10,\n num_test_envs: int = 10,\n persistence_base_dir: str = \"log\",\n render: float = 0.0,\n device: str | None = None,\n frames_stack: int = 4,\n resume_path: str | None = None,\n resume_id: str | None = None,\n logger_type: str = \"tensorboard\",\n wandb_project: str = \"atari.benchmark\",\n watch: bool = False,\n save_buffer_name: str | None = None,\n) -> None:\n # Set defaults for mutable arguments\n if device is None:\n device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\n # Get all local variables as config (excluding internal/temporary ones)\n params_log_info = locals()\n log.info(f\"Starting training with config:\\n{params_log_info}\")\n\n env, training_envs, test_envs = make_atari_env(\n task,\n seed,\n num_training_envs,\n num_test_envs,\n scale=scale_obs,\n frame_stack=frames_stack,\n )\n state_shape = env.observation_space.shape or env.observation_space.n # type: ignore\n action_shape = env.action_space.shape or env.action_space.n # type: ignore\n # should be N_FRAMES x H x W\n log.info(f\"Observations shape: {state_shape}\")\n log.info(f\"Actions shape: {action_shape}\")\n # seed\n np.random.seed(seed)\n torch.manual_seed(seed)\n\n # define model\n c, h, w = state_shape\n net = C51Net(c=c, h=h, w=w, action_shape=action_shape, num_atoms=num_atoms)\n\n # define policy and algorithm\n optim = AdamOptimizerFactory(lr=lr)\n policy = C51Policy(\n model=net,\n action_space=env.action_space,\n num_atoms=num_atoms,\n v_min=v_min,\n v_max=v_max,\n eps_training=eps_train,\n eps_inference=eps_test,\n )\n algorithm: C51 = C51(\n policy=policy,\n optim=optim,\n gamma=gamma,\n n_step_return_horizon=n_step,\n target_update_freq=target_update_freq,\n ).to(device)\n\n # load a previous model\n if resume_path:\n algorithm.load_state_dict(torch.load(resume_path, map_location=device))\n log.info(f\"Loaded agent from: {resume_path}\")\n\n # replay buffer: `save_last_obs` and `stack_num` can be removed together\n # when you have enough RAM\n buffer = VectorReplayBuffer(\n buffer_size,\n buffer_num=len(training_envs),\n ignore_obs_next=True,\n save_only_last_obs=True,\n stack_num=frames_stack,\n )\n\n # collectors\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buffer, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n algo_name = \"c51\"\n log_name = os.path.join(task, algo_name, str(seed), now)\n log_path = os.path.join(persistence_base_dir, log_name)\n\n # logger\n logger_factory = LoggerFactoryDefault()\n if logger_type == \"wandb\":\n logger_factory.logger_type = \"wandb\"\n logger_factory.wandb_project = wandb_project\n else:\n logger_factory.logger_type = \"tensorboard\"\n\n logger = logger_factory.create_logger(\n log_dir=log_path,\n experiment_name=log_name,\n run_id=resume_id,\n config_dict=params_log_info,\n )\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n if env.spec.reward_threshold: # type: ignore\n return mean_rewards >= env.spec.reward_threshold # type: ignore\n if \"Pong\" in task:\n return mean_rewards >= 20\n return False\n\n def train_fn(epoch: int, env_step: int) -> None:\n # nature DQN setting, linear decay in the first 1M steps\n if env_step <= 1e6:\n eps = eps_train - env_step / 1e6 * (eps_train - eps_train_final)\n else:\n eps = eps_train_final\n policy.set_eps_training(eps)\n if env_step % 1000 == 0:\n logger.write(\"train/env_step\", env_step, {\"train/eps\": eps})\n\n def watch_fn() -> None:\n log.info(\"Setup test envs ...\")\n test_envs.seed(seed)\n if save_buffer_name:\n log.info(f\"Generate buffer with size {buffer_size}\")\n buffer = VectorReplayBuffer(\n buffer_size,\n buffer_num=len(test_envs),\n ignore_obs_next=True,\n save_only_last_obs=True,\n stack_num=frames_stack,\n )\n collector = Collector[CollectStats](\n algorithm, test_envs, buffer, exploration_noise=True\n )\n result = collector.collect(n_step=buffer_size, reset_before_collect=True)\n log.info(f\"Save buffer into {save_buffer_name}\")\n # Unfortunately, pickle will cause oom with 1M buffer size\n buffer.save_hdf5(save_buffer_name)\n else:\n log.info(\"Testing agent ...\")\n test_collector.reset()\n result = test_collector.collect(n_episode=num_test_envs, render=render)\n result.pprint_asdict()\n\n if watch:\n watch_fn()\n sys.exit(0)\n\n # test training_collector and start filling replay buffer\n training_collector.reset()\n training_collector.collect(n_step=batch_size * num_training_envs)\n # trainer\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n collection_step_num_env_steps=collection_step_num_env_steps,\n test_step_num_episodes=num_test_envs,\n batch_size=batch_size,\n training_fn=train_fn,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n update_step_num_gradient_steps_per_sample=update_per_step,\n test_in_training=False,\n )\n )\n\n pprint.pprint(result)\n watch_fn()\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/atari/atari_dqn.py", "content": "#!/usr/bin/env python3\n\nimport datetime\nimport os\nimport pprint\nimport sys\n\nimport numpy as np\nimport torch\nfrom sensai.util import logging\n\nfrom tianshou.algorithm import DQN\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelbased.icm import ICMOffPolicyWrapper\nfrom tianshou.algorithm.modelfree.dqn import DiscreteQLearningPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env.atari.atari_network import DQNet\nfrom tianshou.env.atari.atari_wrapper import make_atari_env\nfrom tianshou.highlevel.logger import LoggerFactoryDefault\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils.net.discrete import IntrinsicCuriosityModule\n\nlog = logging.getLogger(__name__)\n\n\ndef main(\n task: str = \"PongNoFrameskip-v4\",\n seed: int = 0,\n scale_obs: int = 0,\n eps_test: float = 0.005,\n eps_train: float = 1.0,\n eps_train_final: float = 0.05,\n buffer_size: int = 100000,\n lr: float = 0.0001,\n gamma: float = 0.99,\n n_step: int = 3,\n target_update_freq: int = 500,\n epoch: int = 100,\n epoch_num_steps: int = 100000,\n collection_step_num_env_steps: int = 10,\n update_per_step: float = 0.1,\n batch_size: int = 32,\n num_training_envs: int = 10,\n num_test_envs: int = 10,\n persistence_base_dir: str = \"log\",\n render: float = 0.0,\n device: str | None = None,\n frames_stack: int = 4,\n resume_path: str | None = None,\n resume_id: str | None = None,\n logger_type: str = \"tensorboard\",\n wandb_project: str = \"atari.benchmark\",\n watch: bool = False,\n save_buffer_name: str | None = None,\n icm_lr_scale: float = 0.0,\n icm_reward_scale: float = 0.01,\n icm_forward_loss_weight: float = 0.2,\n) -> None:\n # Set defaults for mutable arguments\n if device is None:\n device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\n # Get all local variables as config (excluding internal/temporary ones)\n params_log_info = locals()\n log.info(f\"Starting training with config:\\n{params_log_info}\")\n\n state_shape: tuple[int, ...] | int\n action_shape: int\n\n env, training_envs, test_envs = make_atari_env(\n task,\n seed,\n num_training_envs,\n num_test_envs,\n scale=scale_obs,\n frame_stack=frames_stack,\n )\n state_shape = env.observation_space.shape or env.observation_space.n # type: ignore\n action_shape = env.action_space.shape or env.action_space.n # type: ignore\n # should be N_FRAMES x H x W\n log.info(f\"Observations shape: {state_shape}\")\n log.info(f\"Actions shape: {action_shape}\")\n\n # seed\n np.random.seed(seed)\n torch.manual_seed(seed)\n\n # define model\n c, h, w = state_shape\n net = DQNet(c=c, h=h, w=w, action_shape=action_shape).to(device)\n optim = AdamOptimizerFactory(lr=lr)\n\n # define policy and algorithm\n policy = DiscreteQLearningPolicy(\n model=net,\n action_space=env.action_space,\n eps_training=eps_train,\n eps_inference=eps_test,\n )\n algorithm: DQN | ICMOffPolicyWrapper\n algorithm = DQN(\n policy=policy,\n optim=optim,\n gamma=gamma,\n n_step_return_horizon=n_step,\n target_update_freq=target_update_freq,\n )\n if icm_lr_scale > 0:\n c, h, w = state_shape\n feature_net = DQNet(c=c, h=h, w=w, action_shape=action_shape, features_only=True)\n action_dim = int(np.prod(action_shape))\n feature_dim = feature_net.output_dim\n icm_net = IntrinsicCuriosityModule(\n feature_net=feature_net.net,\n feature_dim=feature_dim,\n action_dim=action_dim,\n hidden_sizes=[512],\n )\n icm_optim = AdamOptimizerFactory(lr=lr)\n algorithm = ICMOffPolicyWrapper(\n wrapped_algorithm=algorithm,\n model=icm_net,\n optim=icm_optim,\n lr_scale=icm_lr_scale,\n reward_scale=icm_reward_scale,\n forward_loss_weight=icm_forward_loss_weight,\n ).to(device)\n\n # load a previous model\n if resume_path:\n algorithm.load_state_dict(torch.load(resume_path, map_location=device))\n log.info(f\"Loaded agent from: {resume_path}\")\n\n # replay buffer: `save_last_obs` and `stack_num` can be removed together\n # when you have enough RAM\n buffer = VectorReplayBuffer(\n buffer_size,\n buffer_num=len(training_envs),\n ignore_obs_next=True,\n save_only_last_obs=True,\n stack_num=frames_stack,\n )\n\n # collectors\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buffer, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n algo_name = \"dqn_icm\" if icm_lr_scale > 0 else \"dqn\"\n log_name = os.path.join(task, algo_name, str(seed), now)\n log_path = os.path.join(persistence_base_dir, log_name)\n\n # logger\n logger_factory = LoggerFactoryDefault()\n if logger_type == \"wandb\":\n logger_factory.logger_type = \"wandb\"\n logger_factory.wandb_project = wandb_project\n else:\n logger_factory.logger_type = \"tensorboard\"\n\n logger = logger_factory.create_logger(\n log_dir=log_path,\n experiment_name=log_name,\n run_id=resume_id,\n config_dict=params_log_info,\n )\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n if env.spec.reward_threshold: # type: ignore\n return mean_rewards >= env.spec.reward_threshold # type: ignore\n if \"Pong\" in task:\n return mean_rewards >= 20\n return False\n\n def train_fn(epoch: int, env_step: int) -> None:\n # nature DQN setting, linear decay in the first 1M steps\n if env_step <= 1e6:\n eps = eps_train - env_step / 1e6 * (eps_train - eps_train_final)\n else:\n eps = eps_train_final\n policy.set_eps_training(eps)\n if env_step % 1000 == 0:\n logger.write(\"train/env_step\", env_step, {\"train/eps\": eps})\n\n def save_checkpoint_fn(epoch: int, env_step: int, gradient_step: int) -> str:\n # see also: https://pytorch.org/tutorials/beginner/saving_loading_models.html\n ckpt_path = os.path.join(log_path, f\"checkpoint_{epoch}.pth\")\n torch.save({\"model\": algorithm.state_dict()}, ckpt_path)\n return ckpt_path\n\n def watch_fn() -> None:\n log.info(\"Setup test envs ...\")\n test_envs.seed(seed)\n if save_buffer_name:\n log.info(f\"Generate buffer with size {buffer_size}\")\n buffer = VectorReplayBuffer(\n buffer_size,\n buffer_num=len(test_envs),\n ignore_obs_next=True,\n save_only_last_obs=True,\n stack_num=frames_stack,\n )\n collector = Collector[CollectStats](\n algorithm, test_envs, buffer, exploration_noise=True\n )\n result = collector.collect(n_step=buffer_size, reset_before_collect=True)\n log.info(f\"Save buffer into {save_buffer_name}\")\n # Unfortunately, pickle will cause oom with 1M buffer size\n buffer.save_hdf5(save_buffer_name)\n else:\n log.info(\"Testing agent ...\")\n test_collector.reset()\n result = test_collector.collect(n_episode=num_test_envs, render=render)\n result.pprint_asdict()\n\n if watch:\n watch_fn()\n sys.exit(0)\n\n # test training_collector and start filling replay buffer\n training_collector.reset()\n training_collector.collect(n_step=batch_size * num_training_envs)\n\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n collection_step_num_env_steps=collection_step_num_env_steps,\n test_step_num_episodes=num_test_envs,\n batch_size=batch_size,\n training_fn=train_fn,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n update_step_num_gradient_steps_per_sample=update_per_step,\n test_in_training=False,\n resume_from_log=resume_id is not None,\n save_checkpoint_fn=save_checkpoint_fn,\n )\n )\n\n pprint.pprint(result)\n watch_fn()\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/atari/atari_dqn_hl.py", "content": "#!/usr/bin/env python3\n\nimport os\nfrom typing import Literal\n\nfrom sensai.util import logging\n\nfrom tianshou.env.atari.atari_network import (\n IntermediateModuleFactoryAtariDQN,\n)\nfrom tianshou.env.atari.atari_wrapper import AtariEnvFactory, AtariEpochStopCallback\nfrom tianshou.highlevel.config import OffPolicyTrainingConfig\nfrom tianshou.highlevel.experiment import (\n DQNExperimentBuilder,\n ExperimentConfig,\n)\nfrom tianshou.highlevel.params.algorithm_params import DQNParams\nfrom tianshou.highlevel.trainer import (\n EpochTestCallbackDQNSetEps,\n EpochTrainCallbackDQNEpsLinearDecay,\n)\n\n\ndef main(\n task: str = \"PongNoFrameskip-v4\",\n persistence_base_dir: str = \"log\",\n num_experiments: int = 1,\n experiment_launcher: Literal[\"sequential\", \"joblib\"] = \"sequential\",\n max_epochs: int = 100,\n epoch_num_steps: int = 100000,\n) -> None:\n \"\"\"\n Train an agent using DQN on a specified Atari task, potentially running multiple experiments with different seeds\n and evaluating the results using rliable.\n\n :param task: the Atari task to train on.\n :param persistence_base_dir: the base directory for logging and saving experiment data,\n the task name will be appended to it.\n :param num_experiments: the number of experiments to run. The experiments differ exclusively in the seeds.\n :param experiment_launcher: the type of experiment launcher to use, only has an effect if `num_experiments>1`.\n You can use \"joblib\" for parallel execution of whole experiments.\n :param max_epochs: the maximum number of training epochs.\n :param epoch_num_steps: the number of environment steps per epoch.\n \"\"\"\n persistence_base_dir = os.path.abspath(os.path.join(persistence_base_dir, task))\n experiment_config = ExperimentConfig(persistence_base_dir=persistence_base_dir, watch=False)\n\n training_config = OffPolicyTrainingConfig(\n max_epochs=max_epochs,\n epoch_num_steps=epoch_num_steps,\n batch_size=32,\n num_training_envs=10,\n num_test_envs=10,\n buffer_size=100000,\n collection_step_num_env_steps=10,\n update_step_num_gradient_steps_per_sample=0.1,\n replay_buffer_stack_num=4,\n replay_buffer_ignore_obs_next=True,\n replay_buffer_save_only_last_obs=True,\n )\n\n env_factory = AtariEnvFactory(task, 4, scale=False)\n\n experiment_builder = (\n DQNExperimentBuilder(env_factory, experiment_config, training_config)\n .with_dqn_params(\n DQNParams(\n gamma=0.99,\n n_step_return_horizon=3,\n lr=0.0001,\n target_update_freq=500,\n ),\n )\n .with_model_factory(IntermediateModuleFactoryAtariDQN())\n .with_epoch_train_callback(\n EpochTrainCallbackDQNEpsLinearDecay(1.0, 0.05),\n )\n .with_epoch_test_callback(EpochTestCallbackDQNSetEps(0.005))\n .with_epoch_stop_callback(AtariEpochStopCallback(task))\n )\n\n experiment_builder.build_and_run(num_experiments=num_experiments, launcher=experiment_launcher)\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/atari/atari_fqf.py", "content": "#!/usr/bin/env python3\n\nimport datetime\nimport os\nimport pprint\nimport sys\n\nimport numpy as np\nimport torch\nfrom sensai.util import logging\n\nfrom tianshou.algorithm import FQF\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.fqf import FQFPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory, RMSpropOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env.atari.atari_network import DQNet\nfrom tianshou.env.atari.atari_wrapper import make_atari_env\nfrom tianshou.highlevel.logger import LoggerFactoryDefault\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils.net.discrete import FractionProposalNetwork, FullQuantileFunction\n\nlog = logging.getLogger(__name__)\n\n\ndef main(\n task: str = \"PongNoFrameskip-v4\",\n seed: int = 3128,\n scale_obs: int = 0,\n eps_test: float = 0.005,\n eps_train: float = 1.0,\n eps_train_final: float = 0.05,\n buffer_size: int = 100000,\n lr: float = 5e-5,\n fraction_lr: float = 2.5e-9,\n gamma: float = 0.99,\n num_fractions: int = 32,\n num_cosines: int = 64,\n ent_coef: float = 10.0,\n hidden_sizes: list | None = None,\n n_step: int = 3,\n target_update_freq: int = 500,\n epoch: int = 100,\n epoch_num_steps: int = 100000,\n collection_step_num_env_steps: int = 10,\n update_per_step: float = 0.1,\n batch_size: int = 32,\n num_training_envs: int = 10,\n num_test_envs: int = 10,\n persistence_base_dir: str = \"log\",\n render: float = 0.0,\n device: str | None = None,\n frames_stack: int = 4,\n resume_path: str | None = None,\n resume_id: str | None = None,\n logger_type: str = \"tensorboard\",\n wandb_project: str = \"atari.benchmark\",\n watch: bool = False,\n save_buffer_name: str | None = None,\n) -> None:\n # Set defaults for mutable arguments\n if hidden_sizes is None:\n hidden_sizes = [512]\n if device is None:\n device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\n # Get all local variables as config (excluding internal/temporary ones)\n params_log_info = locals()\n log.info(f\"Starting training with config:\\n{params_log_info}\")\n\n env, training_envs, test_envs = make_atari_env(\n task,\n seed,\n num_training_envs,\n num_test_envs,\n scale=scale_obs,\n frame_stack=frames_stack,\n )\n state_shape = env.observation_space.shape or env.observation_space.n # type: ignore\n action_shape = env.action_space.shape or env.action_space.n # type: ignore\n\n # should be N_FRAMES x H x W\n log.info(f\"Observations shape: {state_shape}\")\n log.info(f\"Actions shape: {action_shape}\")\n\n # seed\n np.random.seed(seed)\n torch.manual_seed(seed)\n\n # define model\n c, h, w = state_shape\n feature_net = DQNet(c=c, h=h, w=w, action_shape=action_shape, features_only=True)\n net = FullQuantileFunction(\n preprocess_net=feature_net,\n action_shape=action_shape,\n hidden_sizes=hidden_sizes,\n num_cosines=num_cosines,\n ).to(device)\n optim = AdamOptimizerFactory(lr=lr)\n fraction_net = FractionProposalNetwork(num_fractions, net.input_dim)\n fraction_optim = RMSpropOptimizerFactory(lr=fraction_lr)\n\n # define policy and algorithm\n policy = FQFPolicy(\n model=net,\n fraction_model=fraction_net,\n action_space=env.action_space,\n )\n algorithm: FQF = FQF(\n policy=policy,\n optim=optim,\n fraction_optim=fraction_optim,\n gamma=gamma,\n num_fractions=num_fractions,\n ent_coef=ent_coef,\n n_step_return_horizon=n_step,\n target_update_freq=target_update_freq,\n ).to(device)\n\n # load a previous policy\n if resume_path:\n algorithm.load_state_dict(torch.load(resume_path, map_location=device))\n log.info(f\"Loaded agent from: {resume_path}\")\n\n # replay buffer: `save_last_obs` and `stack_num` can be removed together\n # when you have enough RAM\n buffer = VectorReplayBuffer(\n buffer_size,\n buffer_num=len(training_envs),\n ignore_obs_next=True,\n save_only_last_obs=True,\n stack_num=frames_stack,\n )\n\n # collectors\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buffer, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n algo_name = \"fqf\"\n log_name = os.path.join(task, algo_name, str(seed), now)\n log_path = os.path.join(persistence_base_dir, log_name)\n\n # logger\n logger_factory = LoggerFactoryDefault()\n if logger_type == \"wandb\":\n logger_factory.logger_type = \"wandb\"\n logger_factory.wandb_project = wandb_project\n else:\n logger_factory.logger_type = \"tensorboard\"\n\n logger = logger_factory.create_logger(\n log_dir=log_path,\n experiment_name=log_name,\n run_id=resume_id,\n config_dict=params_log_info,\n )\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n if env.spec.reward_threshold: # type: ignore\n return mean_rewards >= env.spec.reward_threshold # type: ignore\n if \"Pong\" in task:\n return mean_rewards >= 20\n return False\n\n def train_fn(epoch: int, env_step: int) -> None:\n # nature DQN setting, linear decay in the first 1M steps\n if env_step <= 1e6:\n eps = eps_train - env_step / 1e6 * (eps_train - eps_train_final)\n else:\n eps = eps_train_final\n policy.set_eps_training(eps)\n if env_step % 1000 == 0:\n logger.write(\"train/env_step\", env_step, {\"train/eps\": eps})\n\n def test_fn(epoch: int, env_step: int | None) -> None:\n policy.set_eps_training(eps_test)\n\n def watch_fn() -> None:\n log.info(\"Setup test envs ...\")\n policy.set_eps_training(eps_test)\n test_envs.seed(seed)\n if save_buffer_name:\n log.info(f\"Generate buffer with size {buffer_size}\")\n buffer = VectorReplayBuffer(\n buffer_size,\n buffer_num=len(test_envs),\n ignore_obs_next=True,\n save_only_last_obs=True,\n stack_num=frames_stack,\n )\n collector = Collector[CollectStats](\n algorithm, test_envs, buffer, exploration_noise=True\n )\n result = collector.collect(n_step=buffer_size, reset_before_collect=True)\n log.info(f\"Save buffer into {save_buffer_name}\")\n # Unfortunately, pickle will cause oom with 1M buffer size\n buffer.save_hdf5(save_buffer_name)\n else:\n log.info(\"Testing agent ...\")\n test_collector.reset()\n result = test_collector.collect(n_episode=num_test_envs, render=render)\n result.pprint_asdict()\n\n if watch:\n watch_fn()\n sys.exit(0)\n\n # test training_collector and start filling replay buffer\n training_collector.reset()\n training_collector.collect(n_step=batch_size * num_training_envs)\n\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n collection_step_num_env_steps=collection_step_num_env_steps,\n test_step_num_episodes=num_test_envs,\n batch_size=batch_size,\n training_fn=train_fn,\n test_fn=test_fn,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n update_step_num_gradient_steps_per_sample=update_per_step,\n test_in_training=False,\n )\n )\n\n pprint.pprint(result)\n watch_fn()\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/atari/atari_iqn.py", "content": "#!/usr/bin/env python3\n\nimport datetime\nimport os\nimport pprint\nimport sys\n\nimport numpy as np\nimport torch\nfrom sensai.util import logging\n\nfrom tianshou.algorithm import IQN\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.iqn import IQNPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env.atari.atari_network import DQNet\nfrom tianshou.env.atari.atari_wrapper import make_atari_env\nfrom tianshou.highlevel.logger import LoggerFactoryDefault\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils.net.discrete import ImplicitQuantileNetwork\n\nlog = logging.getLogger(__name__)\n\n\ndef main(\n task: str = \"PongNoFrameskip-v4\",\n seed: int = 1234,\n scale_obs: int = 0,\n eps_test: float = 0.005,\n eps_train: float = 1.0,\n eps_train_final: float = 0.05,\n buffer_size: int = 100000,\n lr: float = 0.0001,\n gamma: float = 0.99,\n sample_size: int = 32,\n online_sample_size: int = 8,\n target_sample_size: int = 8,\n num_cosines: int = 64,\n hidden_sizes: list | None = None,\n n_step: int = 3,\n target_update_freq: int = 500,\n epoch: int = 100,\n epoch_num_steps: int = 100000,\n collection_step_num_env_steps: int = 10,\n update_per_step: float = 0.1,\n batch_size: int = 32,\n num_training_envs: int = 10,\n num_test_envs: int = 10,\n persistence_base_dir: str = \"log\",\n render: float = 0.0,\n device: str | None = None,\n frames_stack: int = 4,\n resume_path: str | None = None,\n resume_id: str | None = None,\n logger_type: str = \"tensorboard\",\n wandb_project: str = \"atari.benchmark\",\n watch: bool = False,\n save_buffer_name: str | None = None,\n) -> None:\n # Set defaults for mutable arguments\n if hidden_sizes is None:\n hidden_sizes = [512]\n if device is None:\n device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\n # Get all local variables as config (excluding internal/temporary ones)\n params_log_info = locals()\n log.info(f\"Starting training with config:\\n{params_log_info}\")\n\n env, training_envs, test_envs = make_atari_env(\n task,\n seed,\n num_training_envs,\n num_test_envs,\n scale=scale_obs,\n frame_stack=frames_stack,\n )\n state_shape = env.observation_space.shape or env.observation_space.n # type: ignore\n action_shape = env.action_space.shape or env.action_space.n # type: ignore\n # should be N_FRAMES x H x W\n log.info(f\"Observations shape: {state_shape}\")\n log.info(f\"Actions shape: {action_shape}\")\n\n # seed\n np.random.seed(seed)\n torch.manual_seed(seed)\n\n # define model\n c, h, w = state_shape\n feature_net = DQNet(c=c, h=h, w=w, action_shape=action_shape, features_only=True)\n net = ImplicitQuantileNetwork(\n preprocess_net=feature_net,\n action_shape=action_shape,\n hidden_sizes=hidden_sizes,\n num_cosines=num_cosines,\n ).to(device)\n optim = AdamOptimizerFactory(lr=lr)\n\n # define policy and algorithm\n policy = IQNPolicy(\n model=net,\n action_space=env.action_space,\n sample_size=sample_size,\n online_sample_size=online_sample_size,\n target_sample_size=target_sample_size,\n eps_training=eps_train,\n eps_inference=eps_test,\n )\n algorithm: IQN = IQN(\n policy=policy,\n optim=optim,\n gamma=gamma,\n n_step_return_horizon=n_step,\n target_update_freq=target_update_freq,\n ).to(device)\n\n # load previous model\n if resume_path:\n algorithm.load_state_dict(torch.load(resume_path, map_location=device))\n log.info(f\"Loaded agent from: {resume_path}\")\n\n # replay buffer: `save_last_obs` and `stack_num` can be removed together\n # when you have enough RAM\n buffer = VectorReplayBuffer(\n buffer_size,\n buffer_num=len(training_envs),\n ignore_obs_next=True,\n save_only_last_obs=True,\n stack_num=frames_stack,\n )\n # collector\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buffer, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n algo_name = \"iqn\"\n log_name = os.path.join(task, algo_name, str(seed), now)\n log_path = os.path.join(persistence_base_dir, log_name)\n\n # logger\n logger_factory = LoggerFactoryDefault()\n if logger_type == \"wandb\":\n logger_factory.logger_type = \"wandb\"\n logger_factory.wandb_project = wandb_project\n else:\n logger_factory.logger_type = \"tensorboard\"\n\n logger = logger_factory.create_logger(\n log_dir=log_path,\n experiment_name=log_name,\n run_id=resume_id,\n config_dict=params_log_info,\n )\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n if env.spec.reward_threshold: # type: ignore\n return mean_rewards >= env.spec.reward_threshold # type: ignore\n if \"Pong\" in task:\n return mean_rewards >= 20\n return False\n\n def train_fn(epoch: int, env_step: int) -> None:\n # nature DQN setting, linear decay in the first 1M steps\n if env_step <= 1e6:\n eps = eps_train - env_step / 1e6 * (eps_train - eps_train_final)\n else:\n eps = eps_train_final\n policy.set_eps_training(eps)\n if env_step % 1000 == 0:\n logger.write(\"train/env_step\", env_step, {\"train/eps\": eps})\n\n # watch agent's performance\n def watch_fn() -> None:\n log.info(\"Setup test envs ...\")\n test_envs.seed(seed)\n if save_buffer_name:\n log.info(f\"Generate buffer with size {buffer_size}\")\n buffer = VectorReplayBuffer(\n buffer_size,\n buffer_num=len(test_envs),\n ignore_obs_next=True,\n save_only_last_obs=True,\n stack_num=frames_stack,\n )\n collector = Collector[CollectStats](\n algorithm, test_envs, buffer, exploration_noise=True\n )\n result = collector.collect(n_step=buffer_size, reset_before_collect=True)\n log.info(f\"Save buffer into {save_buffer_name}\")\n # Unfortunately, pickle will cause oom with 1M buffer size\n buffer.save_hdf5(save_buffer_name)\n else:\n log.info(\"Testing agent ...\")\n test_collector.reset()\n result = test_collector.collect(n_episode=num_test_envs, render=render)\n result.pprint_asdict()\n\n if watch:\n watch_fn()\n sys.exit(0)\n\n # test training_collector and start filling replay buffer\n training_collector.reset()\n training_collector.collect(n_step=batch_size * num_training_envs)\n\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n collection_step_num_env_steps=collection_step_num_env_steps,\n test_step_num_episodes=num_test_envs,\n batch_size=batch_size,\n training_fn=train_fn,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n update_step_num_gradient_steps_per_sample=update_per_step,\n test_in_training=False,\n )\n )\n\n pprint.pprint(result)\n watch_fn()\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/atari/atari_iqn_hl.py", "content": "#!/usr/bin/env python3\n\nimport os\nfrom typing import Literal\n\nfrom sensai.util import logging\n\nfrom tianshou.env.atari.atari_network import (\n IntermediateModuleFactoryAtariDQN,\n)\nfrom tianshou.env.atari.atari_wrapper import AtariEnvFactory, AtariEpochStopCallback\nfrom tianshou.highlevel.config import OffPolicyTrainingConfig\nfrom tianshou.highlevel.experiment import (\n ExperimentConfig,\n IQNExperimentBuilder,\n)\nfrom tianshou.highlevel.params.algorithm_params import IQNParams\nfrom tianshou.highlevel.trainer import (\n EpochTrainCallbackDQNEpsLinearDecay,\n)\n\n\ndef main(\n task: str = \"PongNoFrameskip-v4\",\n persistence_base_dir: str = \"log\",\n num_experiments: int = 1,\n experiment_launcher: Literal[\"sequential\", \"joblib\"] = \"sequential\",\n max_epochs: int = 100,\n epoch_num_steps: int = 100000,\n) -> None:\n \"\"\"\n Train an agent using IQN on a specified Atari task, potentially running multiple experiments with different seeds\n and evaluating the results using rliable.\n\n :param task: the Atari task to train on.\n :param persistence_base_dir: the base directory for logging and saving experiment data,\n the task name will be appended to it.\n :param num_experiments: the number of experiments to run. The experiments differ exclusively in the seeds.\n :param experiment_launcher: the type of experiment launcher to use, only has an effect if `num_experiments>1`.\n You can use \"joblib\" for parallel execution of whole experiments.\n :param max_epochs: the maximum number of training epochs.\n :param epoch_num_steps: the number of environment steps per epoch.\n \"\"\"\n persistence_base_dir = os.path.abspath(os.path.join(persistence_base_dir, task))\n experiment_config = ExperimentConfig(persistence_base_dir=persistence_base_dir, watch=False)\n\n training_config = OffPolicyTrainingConfig(\n max_epochs=max_epochs,\n epoch_num_steps=epoch_num_steps,\n batch_size=32,\n num_training_envs=10,\n num_test_envs=10,\n buffer_size=100000,\n collection_step_num_env_steps=10,\n update_step_num_gradient_steps_per_sample=0.1,\n replay_buffer_stack_num=4,\n replay_buffer_ignore_obs_next=True,\n replay_buffer_save_only_last_obs=True,\n )\n\n env_factory = AtariEnvFactory(task, 4, scale=False)\n\n experiment_builder = (\n IQNExperimentBuilder(env_factory, experiment_config, training_config)\n .with_iqn_params(\n IQNParams(\n gamma=0.99,\n n_step_return_horizon=3,\n lr=0.0001,\n sample_size=32,\n online_sample_size=8,\n target_update_freq=500,\n target_sample_size=8,\n hidden_sizes=(512,),\n num_cosines=64,\n eps_training=1.0,\n eps_inference=0.005,\n ),\n )\n .with_preprocess_network_factory(IntermediateModuleFactoryAtariDQN(features_only=True))\n .with_epoch_train_callback(\n EpochTrainCallbackDQNEpsLinearDecay(1.0, 0.05),\n )\n .with_epoch_stop_callback(AtariEpochStopCallback(task))\n )\n\n experiment_builder.build_and_run(num_experiments=num_experiments, launcher=experiment_launcher)\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/atari/atari_ppo.py", "content": "#!/usr/bin/env python3\n\nimport datetime\nimport os\nimport pprint\nimport sys\nfrom collections.abc import Sequence\nfrom typing import cast\n\nimport numpy as np\nimport torch\nfrom sensai.util import logging\n\nfrom tianshou.algorithm import PPO\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelbased.icm import ICMOnPolicyWrapper\nfrom tianshou.algorithm.modelfree.reinforce import DiscreteActorPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory, LRSchedulerFactoryLinear\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env.atari.atari_network import (\n DQNet,\n ScaledObsInputActionReprNet,\n layer_init,\n)\nfrom tianshou.env.atari.atari_wrapper import make_atari_env\nfrom tianshou.highlevel.logger import LoggerFactoryDefault\nfrom tianshou.trainer import OnPolicyTrainerParams\nfrom tianshou.utils.net.discrete import (\n DiscreteActor,\n DiscreteCritic,\n IntrinsicCuriosityModule,\n)\n\nlog = logging.getLogger(__name__)\n\n\ndef main(\n task: str = \"PongNoFrameskip-v4\",\n seed: int = 4213,\n scale_obs: int = 1,\n buffer_size: int = 100000,\n lr: float = 2.5e-4,\n gamma: float = 0.99,\n epoch: int = 100,\n epoch_num_steps: int = 100000,\n collection_step_num_env_steps: int = 1000,\n update_step_num_repetitions: int = 4,\n batch_size: int = 256,\n hidden_size: int = 512,\n num_training_envs: int = 10,\n num_test_envs: int = 10,\n return_scaling: bool = False,\n vf_coef: float = 0.25,\n ent_coef: float = 0.01,\n gae_lambda: float = 0.95,\n lr_decay: int = True,\n max_grad_norm: float = 0.5,\n eps_clip: float = 0.1,\n dual_clip: float | None = None,\n value_clip: bool = True,\n advantage_normalization: bool = True,\n recompute_adv: bool = False,\n persistence_base_dir: str = \"log\",\n render: float = 0.0,\n device: str | None = None,\n frames_stack: int = 4,\n resume_path: str | None = None,\n resume_id: str | None = None,\n logger_type: str = \"tensorboard\",\n wandb_project: str = \"atari.benchmark\",\n watch: bool = False,\n save_buffer_name: str | None = None,\n icm_lr_scale: float = 0.0,\n icm_reward_scale: float = 0.01,\n icm_forward_loss_weight: float = 0.2,\n) -> None:\n # Set defaults for mutable arguments\n if device is None:\n device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\n # Get all local variables as config (excluding internal/temporary ones)\n params_log_info = locals()\n log.info(f\"Starting training with config:\\n{params_log_info}\")\n\n state_shape: tuple[int, ...] | int\n action_shape: Sequence[int] | int\n\n env, training_envs, test_envs = make_atari_env(\n task,\n seed,\n num_training_envs,\n num_test_envs,\n scale=scale_obs,\n frame_stack=frames_stack,\n )\n state_shape = cast(tuple[int, ...], env.observation_space.shape)\n action_shape = cast(Sequence[int] | int, env.action_space.shape or env.action_space.n) # type: ignore\n # should be N_FRAMES x H x W\n log.info(f\"Observations shape: {state_shape}\")\n log.info(f\"Actions shape: {action_shape}\")\n # seed\n np.random.seed(seed)\n torch.manual_seed(seed)\n # define model\n c, h, w = state_shape\n net: ScaledObsInputActionReprNet | DQNet\n net = DQNet(\n c=c,\n h=h,\n w=w,\n action_shape=action_shape,\n features_only=True,\n output_dim_added_layer=hidden_size,\n layer_init=layer_init,\n )\n if scale_obs:\n net = ScaledObsInputActionReprNet(net)\n actor = DiscreteActor(preprocess_net=net, action_shape=action_shape, softmax_output=False)\n critic = DiscreteCritic(preprocess_net=net)\n optim = AdamOptimizerFactory(lr=lr, eps=1e-5)\n\n if lr_decay:\n optim.with_lr_scheduler_factory(\n LRSchedulerFactoryLinear(\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n collection_step_num_env_steps=collection_step_num_env_steps,\n )\n )\n\n # define algorithm\n policy = DiscreteActorPolicy(\n actor=actor,\n action_space=env.action_space,\n )\n algorithm: PPO = PPO(\n policy=policy,\n critic=critic,\n optim=optim,\n gamma=gamma,\n gae_lambda=gae_lambda,\n max_grad_norm=max_grad_norm,\n vf_coef=vf_coef,\n ent_coef=ent_coef,\n return_scaling=return_scaling,\n eps_clip=eps_clip,\n value_clip=value_clip,\n dual_clip=dual_clip,\n advantage_normalization=advantage_normalization,\n recompute_advantage=recompute_adv,\n ).to(device)\n if icm_lr_scale > 0:\n c, h, w = state_shape\n feature_net = DQNet(c=c, h=h, w=w, action_shape=action_shape, features_only=True)\n action_dim = int(np.prod(action_shape))\n feature_dim = feature_net.output_dim\n icm_net = IntrinsicCuriosityModule(\n feature_net=feature_net.net,\n feature_dim=feature_dim,\n action_dim=action_dim,\n hidden_sizes=[hidden_size],\n )\n icm_optim = AdamOptimizerFactory(lr=lr)\n algorithm = ICMOnPolicyWrapper( # type: ignore[assignment]\n wrapped_algorithm=algorithm,\n model=icm_net,\n optim=icm_optim,\n lr_scale=icm_lr_scale,\n reward_scale=icm_reward_scale,\n forward_loss_weight=icm_forward_loss_weight,\n ).to(device)\n # load a previous policy\n if resume_path:\n algorithm.load_state_dict(torch.load(resume_path, map_location=device))\n log.info(f\"Loaded agent from: {resume_path}\")\n # replay buffer: `save_last_obs` and `stack_num` can be removed together\n # when you have enough RAM\n buffer = VectorReplayBuffer(\n buffer_size,\n buffer_num=len(training_envs),\n ignore_obs_next=True,\n save_only_last_obs=True,\n stack_num=frames_stack,\n )\n # collector\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buffer, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n algo_name = \"ppo_icm\" if icm_lr_scale > 0 else \"ppo\"\n log_name = os.path.join(task, algo_name, str(seed), now)\n log_path = os.path.join(persistence_base_dir, log_name)\n\n # logger\n logger_factory = LoggerFactoryDefault()\n if logger_type == \"wandb\":\n logger_factory.logger_type = \"wandb\"\n logger_factory.wandb_project = wandb_project\n else:\n logger_factory.logger_type = \"tensorboard\"\n\n logger = logger_factory.create_logger(\n log_dir=log_path,\n experiment_name=log_name,\n run_id=resume_id,\n config_dict=params_log_info,\n )\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n if env.spec.reward_threshold: # type: ignore\n return mean_rewards >= env.spec.reward_threshold # type: ignore\n if \"Pong\" in task:\n return mean_rewards >= 20\n return False\n\n def save_checkpoint_fn(epoch: int, env_step: int, gradient_step: int) -> str:\n # see also: https://pytorch.org/tutorials/beginner/saving_loading_models.html\n ckpt_path = os.path.join(log_path, f\"checkpoint_{epoch}.pth\")\n torch.save({\"model\": algorithm.state_dict()}, ckpt_path)\n return ckpt_path\n\n def watch_fn() -> None:\n log.info(\"Setup test envs ...\")\n test_envs.seed(seed)\n if save_buffer_name:\n log.info(f\"Generate buffer with size {buffer_size}\")\n buffer = VectorReplayBuffer(\n buffer_size,\n buffer_num=len(test_envs),\n ignore_obs_next=True,\n save_only_last_obs=True,\n stack_num=frames_stack,\n )\n collector = Collector[CollectStats](\n algorithm, test_envs, buffer, exploration_noise=True\n )\n result = collector.collect(n_step=buffer_size, reset_before_collect=True)\n log.info(f\"Save buffer into {save_buffer_name}\")\n # Unfortunately, pickle will cause oom with 1M buffer size\n buffer.save_hdf5(save_buffer_name)\n else:\n log.info(\"Testing agent ...\")\n test_collector.reset()\n result = test_collector.collect(n_episode=num_test_envs, render=render)\n result.pprint_asdict()\n\n if watch:\n watch_fn()\n sys.exit(0)\n\n # test training_collector and start filling replay buffer\n training_collector.reset()\n training_collector.collect(n_step=batch_size * num_training_envs)\n\n # train\n result = algorithm.run_training(\n OnPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n update_step_num_repetitions=update_step_num_repetitions,\n test_step_num_episodes=num_test_envs,\n batch_size=batch_size,\n collection_step_num_env_steps=collection_step_num_env_steps,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=False,\n resume_from_log=resume_id is not None,\n save_checkpoint_fn=save_checkpoint_fn,\n )\n )\n\n pprint.pprint(result)\n watch_fn()\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/atari/atari_ppo_hl.py", "content": "#!/usr/bin/env python3\n\nimport os\nfrom typing import Literal\n\nfrom sensai.util import logging\n\nfrom tianshou.env.atari.atari_network import (\n ActorFactoryAtariDQN,\n)\nfrom tianshou.env.atari.atari_wrapper import AtariEnvFactory, AtariEpochStopCallback\nfrom tianshou.highlevel.config import OnPolicyTrainingConfig\nfrom tianshou.highlevel.experiment import (\n ExperimentConfig,\n PPOExperimentBuilder,\n)\nfrom tianshou.highlevel.params.algorithm_params import PPOParams\nfrom tianshou.highlevel.params.lr_scheduler import LRSchedulerFactoryFactoryLinear\n\n\ndef main(\n task: str = \"PongNoFrameskip-v4\",\n persistence_base_dir: str = \"log\",\n num_experiments: int = 1,\n experiment_launcher: Literal[\"sequential\", \"joblib\"] = \"sequential\",\n max_epochs: int = 100,\n epoch_num_steps: int = 100000,\n) -> None:\n \"\"\"\n Train an agent using PPO on a specified Atari task, potentially running multiple experiments with different seeds\n and evaluating the results using rliable.\n\n :param task: the Atari task to train on.\n :param persistence_base_dir: the base directory for logging and saving experiment data,\n the task name will be appended to it.\n :param num_experiments: the number of experiments to run. The experiments differ exclusively in the seeds.\n :param experiment_launcher: the type of experiment launcher to use, only has an effect if `num_experiments>1`.\n You can use \"joblib\" for parallel execution of whole experiments.\n :param max_epochs: the maximum number of training epochs.\n :param epoch_num_steps: the number of environment steps per epoch.\n \"\"\"\n persistence_base_dir = os.path.abspath(os.path.join(persistence_base_dir, task))\n experiment_config = ExperimentConfig(persistence_base_dir=persistence_base_dir, watch=False)\n\n training_config = OnPolicyTrainingConfig(\n max_epochs=max_epochs,\n epoch_num_steps=epoch_num_steps,\n batch_size=256,\n num_training_envs=10,\n num_test_envs=10,\n buffer_size=100000,\n collection_step_num_env_steps=1000,\n update_step_num_repetitions=4,\n replay_buffer_stack_num=4,\n replay_buffer_ignore_obs_next=True,\n replay_buffer_save_only_last_obs=True,\n )\n\n env_factory = AtariEnvFactory(task, 4, scale=True)\n\n experiment_builder = (\n PPOExperimentBuilder(env_factory, experiment_config, training_config)\n .with_ppo_params(\n PPOParams(\n gamma=0.99,\n gae_lambda=0.95,\n return_scaling=False,\n ent_coef=0.01,\n vf_coef=0.25,\n max_grad_norm=0.5,\n value_clip=True,\n advantage_normalization=True,\n eps_clip=0.1,\n dual_clip=None,\n recompute_advantage=False,\n lr=2.5e-4,\n lr_scheduler=LRSchedulerFactoryFactoryLinear(training_config),\n ),\n )\n .with_actor_factory(ActorFactoryAtariDQN(scale_obs=True, features_only=True))\n .with_critic_factory_use_actor()\n .with_epoch_stop_callback(AtariEpochStopCallback(task))\n )\n\n experiment_builder.build_and_run(num_experiments=num_experiments, launcher=experiment_launcher)\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/atari/atari_qrdqn.py", "content": "#!/usr/bin/env python3\n\nimport datetime\nimport os\nimport pprint\nimport sys\n\nimport numpy as np\nimport torch\nfrom sensai.util import logging\n\nfrom tianshou.algorithm import QRDQN\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.qrdqn import QRDQNPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env.atari.atari_network import QRDQNet\nfrom tianshou.env.atari.atari_wrapper import make_atari_env\nfrom tianshou.highlevel.logger import LoggerFactoryDefault\nfrom tianshou.trainer import OffPolicyTrainerParams\n\nlog = logging.getLogger(__name__)\n\n\ndef main(\n task: str = \"PongNoFrameskip-v4\",\n seed: int = 0,\n scale_obs: int = 0,\n eps_test: float = 0.005,\n eps_train: float = 1.0,\n eps_train_final: float = 0.05,\n buffer_size: int = 100000,\n lr: float = 0.0001,\n gamma: float = 0.99,\n num_quantiles: int = 200,\n n_step: int = 3,\n target_update_freq: int = 500,\n epoch: int = 100,\n epoch_num_steps: int = 100000,\n collection_step_num_env_steps: int = 10,\n update_per_step: float = 0.1,\n batch_size: int = 32,\n num_training_envs: int = 10,\n num_test_envs: int = 10,\n persistence_base_dir: str = \"log\",\n render: float = 0.0,\n device: str | None = None,\n frames_stack: int = 4,\n resume_path: str | None = None,\n resume_id: str | None = None,\n logger_type: str = \"tensorboard\",\n wandb_project: str = \"atari.benchmark\",\n watch: bool = False,\n save_buffer_name: str | None = None,\n) -> None:\n # Set defaults for mutable arguments\n if device is None:\n device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\n # Get all local variables as config (excluding internal/temporary ones)\n params_log_info = locals()\n log.info(f\"Starting training with config:\\n{params_log_info}\")\n\n env, training_envs, test_envs = make_atari_env(\n task,\n seed,\n num_training_envs,\n num_test_envs,\n scale=scale_obs,\n frame_stack=frames_stack,\n )\n state_shape = env.observation_space.shape or env.observation_space.n # type: ignore\n action_shape = env.action_space.shape or env.action_space.n # type: ignore\n\n # should be N_FRAMES x H x W\n log.info(f\"Observations shape: {state_shape}\")\n log.info(f\"Actions shape: {action_shape}\")\n\n # seed\n np.random.seed(seed)\n torch.manual_seed(seed)\n\n # define model\n c, h, w = state_shape\n net = QRDQNet(\n c=c,\n h=h,\n w=w,\n action_shape=action_shape,\n num_quantiles=num_quantiles,\n )\n\n # define policy and algorithm\n optim = AdamOptimizerFactory(lr=lr)\n policy = QRDQNPolicy(\n model=net,\n action_space=env.action_space,\n eps_training=eps_train,\n eps_inference=eps_test,\n )\n algorithm: QRDQN = QRDQN(\n policy=policy,\n optim=optim,\n gamma=gamma,\n num_quantiles=num_quantiles,\n n_step_return_horizon=n_step,\n target_update_freq=target_update_freq,\n ).to(device)\n\n # load a previous policy\n if resume_path:\n algorithm.load_state_dict(torch.load(resume_path, map_location=device))\n log.info(f\"Loaded agent from: {resume_path}\")\n # replay buffer: `save_last_obs` and `stack_num` can be removed together\n # when you have enough RAM\n buffer = VectorReplayBuffer(\n buffer_size,\n buffer_num=len(training_envs),\n ignore_obs_next=True,\n save_only_last_obs=True,\n stack_num=frames_stack,\n )\n\n # collector\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buffer, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n algo_name = \"qrdqn\"\n log_name = os.path.join(task, algo_name, str(seed), now)\n log_path = os.path.join(persistence_base_dir, log_name)\n\n # logger\n logger_factory = LoggerFactoryDefault()\n if logger_type == \"wandb\":\n logger_factory.logger_type = \"wandb\"\n logger_factory.wandb_project = wandb_project\n else:\n logger_factory.logger_type = \"tensorboard\"\n\n logger = logger_factory.create_logger(\n log_dir=log_path,\n experiment_name=log_name,\n run_id=resume_id,\n config_dict=params_log_info,\n )\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n if env.spec.reward_threshold: # type: ignore\n return mean_rewards >= env.spec.reward_threshold # type: ignore\n if \"Pong\" in task:\n return mean_rewards >= 20\n return False\n\n def train_fn(epoch: int, env_step: int) -> None:\n # nature DQN setting, linear decay in the first 1M steps\n if env_step <= 1e6:\n eps = eps_train - env_step / 1e6 * (eps_train - eps_train_final)\n else:\n eps = eps_train_final\n policy.set_eps_training(eps)\n if env_step % 1000 == 0:\n logger.write(\"train/env_step\", env_step, {\"train/eps\": eps})\n\n # watch agent's performance\n def watch_fn() -> None:\n log.info(\"Setup test envs ...\")\n test_envs.seed(seed)\n if save_buffer_name:\n log.info(f\"Generate buffer with size {buffer_size}\")\n buffer = VectorReplayBuffer(\n buffer_size,\n buffer_num=len(test_envs),\n ignore_obs_next=True,\n save_only_last_obs=True,\n stack_num=frames_stack,\n )\n collector = Collector[CollectStats](\n algorithm, test_envs, buffer, exploration_noise=True\n )\n result = collector.collect(n_step=buffer_size, reset_before_collect=True)\n log.info(f\"Save buffer into {save_buffer_name}\")\n # Unfortunately, pickle will cause oom with 1M buffer size\n buffer.save_hdf5(save_buffer_name)\n else:\n log.info(\"Testing agent ...\")\n test_collector.reset()\n result = test_collector.collect(n_episode=num_test_envs, render=render)\n result.pprint_asdict()\n\n if watch:\n watch_fn()\n sys.exit(0)\n\n # test training_collector and start filling replay buffer\n training_collector.reset()\n training_collector.collect(n_step=batch_size * num_training_envs)\n\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n collection_step_num_env_steps=collection_step_num_env_steps,\n test_step_num_episodes=num_test_envs,\n batch_size=batch_size,\n training_fn=train_fn,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n update_step_num_gradient_steps_per_sample=update_per_step,\n test_in_training=False,\n )\n )\n\n pprint.pprint(result)\n watch_fn()\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/atari/atari_rainbow.py", "content": "#!/usr/bin/env python3\n\nimport datetime\nimport os\nimport pprint\nimport sys\n\nimport numpy as np\nimport torch\nfrom sensai.util import logging\n\nfrom tianshou.algorithm import C51, RainbowDQN\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.c51 import C51Policy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import (\n Collector,\n CollectStats,\n PrioritizedVectorReplayBuffer,\n VectorReplayBuffer,\n)\nfrom tianshou.env.atari.atari_network import RainbowNet\nfrom tianshou.env.atari.atari_wrapper import make_atari_env\nfrom tianshou.highlevel.logger import LoggerFactoryDefault\nfrom tianshou.trainer import OffPolicyTrainerParams\n\nlog = logging.getLogger(__name__)\n\n\ndef main(\n task: str = \"PongNoFrameskip-v4\",\n seed: int = 0,\n scale_obs: int = 0,\n eps_test: float = 0.005,\n eps_train: float = 1.0,\n eps_train_final: float = 0.05,\n buffer_size: int = 100000,\n lr: float = 0.0000625,\n gamma: float = 0.99,\n num_atoms: int = 51,\n v_min: float = -10.0,\n v_max: float = 10.0,\n noisy_std: float = 0.1,\n no_dueling: bool = False,\n no_noisy: bool = False,\n no_priority: bool = False,\n alpha: float = 0.5,\n beta: float = 0.4,\n beta_final: float = 1.0,\n beta_anneal_step: int = 5000000,\n no_weight_norm: bool = False,\n n_step: int = 3,\n target_update_freq: int = 500,\n epoch: int = 100,\n epoch_num_steps: int = 100000,\n collection_step_num_env_steps: int = 10,\n update_per_step: float = 0.1,\n batch_size: int = 32,\n num_training_envs: int = 10,\n num_test_envs: int = 10,\n persistence_base_dir: str = \"log\",\n render: float = 0.0,\n device: str | None = None,\n frames_stack: int = 4,\n resume_path: str | None = None,\n resume_id: str | None = None,\n logger_type: str = \"tensorboard\",\n wandb_project: str = \"atari.benchmark\",\n watch: bool = False,\n save_buffer_name: str | None = None,\n) -> None:\n # Set defaults for mutable arguments\n if device is None:\n device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\n # Get all local variables as config (excluding internal/temporary ones)\n params_log_info = locals()\n log.info(f\"Starting training with config:\\n{params_log_info}\")\n\n env, training_envs, test_envs = make_atari_env(\n task,\n seed,\n num_training_envs,\n num_test_envs,\n scale=scale_obs,\n frame_stack=frames_stack,\n )\n state_shape = env.observation_space.shape or env.observation_space.n # type: ignore\n action_shape = env.action_space.shape or env.action_space.n # type: ignore\n # should be N_FRAMES x H x W\n log.info(f\"Observations shape: {state_shape}\")\n log.info(f\"Actions shape: {action_shape}\")\n\n # seed\n np.random.seed(seed)\n torch.manual_seed(seed)\n\n # define model\n c, h, w = state_shape\n net = RainbowNet(\n c=c,\n h=h,\n w=w,\n action_shape=action_shape,\n num_atoms=num_atoms,\n noisy_std=noisy_std,\n is_dueling=not no_dueling,\n is_noisy=not no_noisy,\n )\n\n # define policy and algorithm\n policy = C51Policy(\n model=net,\n action_space=env.action_space,\n num_atoms=num_atoms,\n v_min=v_min,\n v_max=v_max,\n eps_training=eps_train,\n eps_inference=eps_test,\n )\n optim = AdamOptimizerFactory(lr=lr)\n algorithm: C51 = RainbowDQN(\n policy=policy,\n optim=optim,\n gamma=gamma,\n n_step_return_horizon=n_step,\n target_update_freq=target_update_freq,\n ).to(device)\n\n # load a previous policy\n if resume_path:\n algorithm.load_state_dict(torch.load(resume_path, map_location=device))\n log.info(f\"Loaded agent from: {resume_path}\")\n\n # replay buffer: `save_last_obs` and `stack_num` can be removed together\n # when you have enough RAM\n buffer: VectorReplayBuffer | PrioritizedVectorReplayBuffer\n if no_priority:\n buffer = VectorReplayBuffer(\n buffer_size,\n buffer_num=len(training_envs),\n ignore_obs_next=True,\n save_only_last_obs=True,\n stack_num=frames_stack,\n )\n else:\n buffer = PrioritizedVectorReplayBuffer(\n buffer_size,\n buffer_num=len(training_envs),\n ignore_obs_next=True,\n save_only_last_obs=True,\n stack_num=frames_stack,\n alpha=alpha,\n beta=beta,\n weight_norm=not no_weight_norm,\n )\n\n # collector\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buffer, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n algo_name = \"rainbow\"\n log_name = os.path.join(task, algo_name, str(seed), now)\n log_path = os.path.join(persistence_base_dir, log_name)\n\n # logger\n logger_factory = LoggerFactoryDefault()\n if logger_type == \"wandb\":\n logger_factory.logger_type = \"wandb\"\n logger_factory.wandb_project = wandb_project\n else:\n logger_factory.logger_type = \"tensorboard\"\n\n logger = logger_factory.create_logger(\n log_dir=log_path,\n experiment_name=log_name,\n run_id=resume_id,\n config_dict=params_log_info,\n )\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n if env.spec.reward_threshold: # type: ignore\n return mean_rewards >= env.spec.reward_threshold # type: ignore\n if \"Pong\" in task:\n return mean_rewards >= 20\n return False\n\n def train_fn(epoch: int, env_step: int) -> None:\n # nature DQN setting, linear decay in the first 1M steps\n if env_step <= 1e6:\n eps = eps_train - env_step / 1e6 * (eps_train - eps_train_final)\n else:\n eps = eps_train_final\n policy.set_eps_training(eps)\n if env_step % 1000 == 0:\n logger.write(\"train/env_step\", env_step, {\"train/eps\": eps})\n if not no_priority:\n if env_step <= beta_anneal_step:\n beta_value = beta - env_step / beta_anneal_step * (beta - beta_final)\n else:\n beta_value = beta_final\n buffer.set_beta(beta_value)\n if env_step % 1000 == 0:\n logger.write(\"train/env_step\", env_step, {\"train/beta\": beta_value})\n\n def watch_fn() -> None:\n log.info(\"Setup test envs ...\")\n test_envs.seed(seed)\n if save_buffer_name:\n log.info(f\"Generate buffer with size {buffer_size}\")\n buffer = PrioritizedVectorReplayBuffer(\n buffer_size,\n buffer_num=len(test_envs),\n ignore_obs_next=True,\n save_only_last_obs=True,\n stack_num=frames_stack,\n alpha=alpha,\n beta=beta,\n )\n collector = Collector[CollectStats](\n algorithm, test_envs, buffer, exploration_noise=True\n )\n result = collector.collect(n_step=buffer_size, reset_before_collect=True)\n log.info(f\"Save buffer into {save_buffer_name}\")\n # Unfortunately, pickle will cause oom with 1M buffer size\n buffer.save_hdf5(save_buffer_name)\n else:\n log.info(\"Testing agent ...\")\n test_collector.reset()\n result = test_collector.collect(n_episode=num_test_envs, render=render)\n result.pprint_asdict()\n\n if watch:\n watch_fn()\n sys.exit(0)\n\n # test training_collector and start filling replay buffer\n training_collector.reset()\n training_collector.collect(n_step=batch_size * num_training_envs)\n\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n collection_step_num_env_steps=collection_step_num_env_steps,\n test_step_num_episodes=num_test_envs,\n batch_size=batch_size,\n training_fn=train_fn,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n update_step_num_gradient_steps_per_sample=update_per_step,\n test_in_training=False,\n )\n )\n\n pprint.pprint(result)\n watch_fn()\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/atari/atari_sac.py", "content": "#!/usr/bin/env python3\n\nimport datetime\nimport os\nimport pprint\nimport sys\n\nimport numpy as np\nimport torch\nfrom sensai.util import logging\n\nfrom tianshou.algorithm import DiscreteSAC, ICMOffPolicyWrapper\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.discrete_sac import DiscreteSACPolicy\nfrom tianshou.algorithm.modelfree.sac import AutoAlpha\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env.atari.atari_network import DQNet\nfrom tianshou.env.atari.atari_wrapper import make_atari_env\nfrom tianshou.highlevel.logger import LoggerFactoryDefault\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils.net.discrete import (\n DiscreteActor,\n DiscreteCritic,\n IntrinsicCuriosityModule,\n)\n\nlog = logging.getLogger(__name__)\n\n\ndef main(\n task: str = \"PongNoFrameskip-v4\",\n seed: int = 4213,\n scale_obs: int = 0,\n buffer_size: int = 100000,\n actor_lr: float = 1e-5,\n critic_lr: float = 1e-5,\n gamma: float = 0.99,\n n_step: int = 3,\n tau: float = 0.005,\n alpha: float = 0.05,\n auto_alpha: bool = False,\n alpha_lr: float = 3e-4,\n epoch: int = 100,\n epoch_num_steps: int = 100000,\n collection_step_num_env_steps: int = 10,\n update_per_step: float = 0.1,\n batch_size: int = 64,\n hidden_size: int = 512,\n num_training_envs: int = 10,\n num_test_envs: int = 10,\n return_scaling: int = False,\n persistence_base_dir: str = \"log\",\n render: float = 0.0,\n device: str | None = None,\n frames_stack: int = 4,\n resume_path: str | None = None,\n resume_id: str | None = None,\n logger_type: str = \"tensorboard\",\n wandb_project: str = \"atari.benchmark\",\n watch: bool = False,\n save_buffer_name: str | None = None,\n icm_lr_scale: float = 0.0,\n icm_reward_scale: float = 0.01,\n icm_forward_loss_weight: float = 0.2,\n) -> None:\n # Set defaults for mutable arguments\n if device is None:\n device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\n # Get all local variables as config (excluding internal/temporary ones)\n params_log_info = locals()\n log.info(f\"Starting training with config:\\n{params_log_info}\")\n\n env, training_envs, test_envs = make_atari_env(\n task,\n seed,\n num_training_envs,\n num_test_envs,\n scale=scale_obs,\n frame_stack=frames_stack,\n )\n c, h, w = env.observation_space.shape # type: ignore\n action_shape = env.action_space.n # type: ignore\n\n # should be N_FRAMES x H x W\n log.info(f\"Observations shape: {(c, h, w)}\")\n log.info(f\"Actions shape: {action_shape}\")\n\n # seed\n np.random.seed(seed)\n torch.manual_seed(seed)\n\n # define model\n net = DQNet(\n c,\n h,\n w,\n action_shape=action_shape,\n features_only=True,\n output_dim_added_layer=hidden_size,\n )\n actor = DiscreteActor(preprocess_net=net, action_shape=action_shape, softmax_output=False)\n actor_optim = AdamOptimizerFactory(lr=actor_lr)\n critic1 = DiscreteCritic(preprocess_net=net, last_size=action_shape)\n critic1_optim = AdamOptimizerFactory(lr=critic_lr)\n critic2 = DiscreteCritic(preprocess_net=net, last_size=action_shape)\n critic2_optim = AdamOptimizerFactory(lr=critic_lr)\n\n # define policy and algorithm\n alpha_param: float | AutoAlpha = alpha\n if auto_alpha:\n target_entropy = 0.98 * np.log(np.prod(action_shape))\n log_alpha = 0.0\n alpha_optim = AdamOptimizerFactory(lr=alpha_lr)\n alpha_param = AutoAlpha(target_entropy, log_alpha, alpha_optim)\n algorithm: DiscreteSAC | ICMOffPolicyWrapper\n policy = DiscreteSACPolicy(\n actor=actor,\n action_space=env.action_space,\n )\n algorithm = DiscreteSAC(\n policy=policy,\n policy_optim=actor_optim,\n critic=critic1,\n critic_optim=critic1_optim,\n critic2=critic2,\n critic2_optim=critic2_optim,\n tau=tau,\n gamma=gamma,\n alpha=alpha_param,\n n_step_return_horizon=n_step,\n ).to(device)\n if icm_lr_scale > 0:\n feature_net = DQNet(c=c, h=h, w=w, action_shape=action_shape, features_only=True)\n action_dim = np.prod(action_shape)\n feature_dim = feature_net.output_dim\n icm_net = IntrinsicCuriosityModule(\n feature_net=feature_net.net,\n feature_dim=feature_dim,\n action_dim=int(action_dim),\n hidden_sizes=[hidden_size],\n )\n icm_optim = AdamOptimizerFactory(lr=actor_lr)\n algorithm = ICMOffPolicyWrapper(\n wrapped_algorithm=algorithm,\n model=icm_net,\n optim=icm_optim,\n lr_scale=icm_lr_scale,\n reward_scale=icm_reward_scale,\n forward_loss_weight=icm_forward_loss_weight,\n ).to(device)\n\n # load a previous model\n if resume_path:\n algorithm.load_state_dict(torch.load(resume_path, map_location=device))\n log.info(f\"Loaded agent from: {resume_path}\")\n\n # replay buffer: `save_last_obs` and `stack_num` can be removed together\n # when you have enough RAM\n buffer = VectorReplayBuffer(\n buffer_size,\n buffer_num=len(training_envs),\n ignore_obs_next=True,\n save_only_last_obs=True,\n stack_num=frames_stack,\n )\n # collector\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buffer, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n algo_name = \"discrete_sac_icm\" if icm_lr_scale > 0 else \"discrete_sac\"\n log_name = os.path.join(task, algo_name, str(seed), now)\n log_path = os.path.join(persistence_base_dir, log_name)\n\n # logger\n logger_factory = LoggerFactoryDefault()\n if logger_type == \"wandb\":\n logger_factory.logger_type = \"wandb\"\n logger_factory.wandb_project = wandb_project\n else:\n logger_factory.logger_type = \"tensorboard\"\n\n logger = logger_factory.create_logger(\n log_dir=log_path,\n experiment_name=log_name,\n run_id=resume_id,\n config_dict=params_log_info,\n )\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n if env.spec.reward_threshold: # type: ignore\n return mean_rewards >= env.spec.reward_threshold # type: ignore\n if \"Pong\" in task:\n return mean_rewards >= 20\n return False\n\n def save_checkpoint_fn(epoch: int, env_step: int, gradient_step: int) -> str:\n # see also: https://pytorch.org/tutorials/beginner/saving_loading_models.html\n ckpt_path = os.path.join(log_path, \"checkpoint.pth\")\n torch.save({\"model\": algorithm.state_dict()}, ckpt_path)\n return ckpt_path\n\n def watch_fn() -> None:\n log.info(\"Setup test envs ...\")\n test_envs.seed(seed)\n if save_buffer_name:\n log.info(f\"Generate buffer with size {buffer_size}\")\n buffer = VectorReplayBuffer(\n buffer_size,\n buffer_num=len(test_envs),\n ignore_obs_next=True,\n save_only_last_obs=True,\n stack_num=frames_stack,\n )\n collector = Collector[CollectStats](\n algorithm, test_envs, buffer, exploration_noise=True\n )\n result = collector.collect(n_step=buffer_size, reset_before_collect=True)\n log.info(f\"Save buffer into {save_buffer_name}\")\n # Unfortunately, pickle will cause oom with 1M buffer size\n buffer.save_hdf5(save_buffer_name)\n else:\n log.info(\"Testing agent ...\")\n test_collector.reset()\n result = test_collector.collect(n_episode=num_test_envs, render=render)\n result.pprint_asdict()\n\n if watch:\n watch_fn()\n sys.exit(0)\n\n # test training_collector and start filling replay buffer\n training_collector.reset()\n training_collector.collect(n_step=batch_size * num_training_envs)\n\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n collection_step_num_env_steps=collection_step_num_env_steps,\n test_step_num_episodes=num_test_envs,\n batch_size=batch_size,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n update_step_num_gradient_steps_per_sample=update_per_step,\n test_in_training=False,\n resume_from_log=resume_id is not None,\n save_checkpoint_fn=save_checkpoint_fn,\n )\n )\n\n pprint.pprint(result)\n watch_fn()\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/atari/atari_sac_hl.py", "content": "#!/usr/bin/env python3\n\nimport os\nfrom typing import Literal\n\nfrom sensai.util import logging\n\nfrom tianshou.env.atari.atari_network import (\n ActorFactoryAtariDQN,\n)\nfrom tianshou.env.atari.atari_wrapper import AtariEnvFactory, AtariEpochStopCallback\nfrom tianshou.highlevel.config import OffPolicyTrainingConfig\nfrom tianshou.highlevel.experiment import (\n DiscreteSACExperimentBuilder,\n ExperimentConfig,\n)\nfrom tianshou.highlevel.params.algorithm_params import DiscreteSACParams\n\n\ndef main(\n task: str = \"PongNoFrameskip-v4\",\n persistence_base_dir: str = \"log\",\n num_experiments: int = 1,\n experiment_launcher: Literal[\"sequential\", \"joblib\"] = \"sequential\",\n max_epochs: int = 100,\n epoch_num_steps: int = 100000,\n) -> None:\n \"\"\"\n Train an agent using SAC on a specified Atari task, potentially running multiple experiments with different seeds\n and evaluating the results using rliable.\n\n :param task: the Atari task to train on.\n :param persistence_base_dir: the base directory for logging and saving experiment data,\n the task name will be appended to it.\n :param num_experiments: the number of experiments to run. The experiments differ exclusively in the seeds.\n :param experiment_launcher: the type of experiment launcher to use, only has an effect if `num_experiments>1`.\n You can use \"joblib\" for parallel execution of whole experiments.\n :param max_epochs: the maximum number of training epochs.\n :param epoch_num_steps: the number of environment steps per epoch.\n \"\"\"\n persistence_base_dir = os.path.abspath(os.path.join(persistence_base_dir, task))\n experiment_config = ExperimentConfig(persistence_base_dir=persistence_base_dir, watch=False)\n\n training_config = OffPolicyTrainingConfig(\n max_epochs=max_epochs,\n epoch_num_steps=epoch_num_steps,\n update_step_num_gradient_steps_per_sample=0.1,\n batch_size=64,\n num_training_envs=10,\n num_test_envs=10,\n buffer_size=100000,\n collection_step_num_env_steps=10,\n replay_buffer_stack_num=4,\n replay_buffer_ignore_obs_next=True,\n replay_buffer_save_only_last_obs=True,\n )\n\n env_factory = AtariEnvFactory(task, 4, scale=False)\n\n experiment_builder = (\n DiscreteSACExperimentBuilder(env_factory, experiment_config, training_config)\n .with_sac_params(\n DiscreteSACParams(\n actor_lr=1e-5,\n critic1_lr=1e-5,\n critic2_lr=1e-5,\n gamma=0.99,\n tau=0.005,\n alpha=0.05,\n n_step_return_horizon=3,\n ),\n )\n .with_actor_factory(ActorFactoryAtariDQN(scale_obs=False, features_only=True))\n .with_common_critic_factory_use_actor()\n .with_epoch_stop_callback(AtariEpochStopCallback(task))\n )\n\n experiment_builder.build_and_run(num_experiments=num_experiments, launcher=experiment_launcher)\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/box2d/README.md", "content": "# Bipedal-Hardcore-SAC\n\n- Our default choice: remove the done flag penalty, will soon converge to \\~280 reward within 100 epochs (10M env steps,\n 3~4 hours, see the image below)\n- If the done penalty is not removed, it converges much slower than before, about 200 epochs (20M env steps) to reach\n the same performance (\\~200 reward)\n\n![](results/sac/BipedalHardcore.png)\n\n# BipedalWalker-BDQ\n\n- To demonstrate the cpabilities of the BDQ to scale up to big discrete action spaces, we run it on a discretized\n version of the BipedalWalker-v3 environment, where the number of possible actions in each dimension is 25, for a total\n of 25^4 = 390 625 possible actions. A usaual DQN architecture would use 25^4 output neurons for the Q-network, thus\n scaling exponentially with the number of action space dimensions, while the Branching architecture scales linearly and\n uses only 25*4 output neurons.\n\n![](results/bdq/BipedalWalker.png)" }, { "path": "examples/box2d/acrobot_dualdqn.py", "content": "import argparse\nimport os\nimport pprint\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom tianshou.algorithm import DQN\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.dqn import DiscreteQLearningPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"Acrobot-v1\")\n parser.add_argument(\"--seed\", type=int, default=0)\n parser.add_argument(\"--eps_test\", type=float, default=0.05)\n parser.add_argument(\"--eps_train\", type=float, default=0.5)\n parser.add_argument(\"--buffer_size\", type=int, default=20000)\n parser.add_argument(\"--lr\", type=float, default=1e-3)\n parser.add_argument(\"--gamma\", type=float, default=0.95)\n parser.add_argument(\"--n_step\", type=int, default=3)\n parser.add_argument(\"--target_update_freq\", type=int, default=320)\n parser.add_argument(\"--epoch\", type=int, default=10)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=100000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=100)\n parser.add_argument(\"--update_per_step\", type=float, default=0.01)\n parser.add_argument(\"--batch_size\", type=int, default=64)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[128])\n parser.add_argument(\"--dueling_q_hidden_sizes\", type=int, nargs=\"*\", default=[128, 128])\n parser.add_argument(\"--dueling_v_hidden_sizes\", type=int, nargs=\"*\", default=[128, 128])\n parser.add_argument(\"--num_training_envs\", type=int, default=10)\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n return parser.parse_args()\n\n\ndef test_dqn(args: argparse.Namespace = get_args()) -> None:\n env = gym.make(args.task)\n assert isinstance(env.action_space, gym.spaces.Discrete)\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n # training_envs = gym.make(args.task)\n # you can also use tianshou.env.SubprocVectorEnv\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n # test_envs = gym.make(args.task)\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n # model\n Q_param = {\"hidden_sizes\": args.dueling_q_hidden_sizes}\n V_param = {\"hidden_sizes\": args.dueling_v_hidden_sizes}\n net = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n dueling_param=(Q_param, V_param),\n )\n optim = AdamOptimizerFactory(lr=args.lr)\n policy = DiscreteQLearningPolicy(\n model=net,\n action_space=env.action_space,\n eps_training=args.eps_train,\n eps_inference=args.eps_test,\n )\n algorithm: DQN = DQN(\n policy=policy,\n optim=optim,\n gamma=args.gamma,\n n_step_return_horizon=args.n_step,\n target_update_freq=args.target_update_freq,\n ).to(args.device)\n # collector\n training_collector = Collector[CollectStats](\n algorithm,\n training_envs,\n VectorReplayBuffer(args.buffer_size, len(training_envs)),\n exploration_noise=True,\n )\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n training_collector.reset()\n training_collector.collect(n_step=args.batch_size * args.num_training_envs)\n # log\n log_path = os.path.join(args.logdir, args.task, \"dqn\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n if env.spec:\n if not env.spec.reward_threshold:\n return False\n else:\n return mean_rewards >= env.spec.reward_threshold\n return False\n\n def train_fn(epoch: int, env_step: int) -> None:\n if env_step <= 100000:\n policy.set_eps_training(args.eps_train)\n elif env_step <= 500000:\n eps = args.eps_train - (env_step - 100000) / 400000 * (0.5 * args.eps_train)\n policy.set_eps_training(eps)\n else:\n policy.set_eps_training(0.5 * args.eps_train)\n\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n update_step_num_gradient_steps_per_sample=args.update_per_step,\n training_fn=train_fn,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=True,\n )\n )\n\n assert stop_fn(result.best_reward)\n if __name__ == \"__main__\":\n pprint.pprint(result)\n # Let's watch its performance!\n test_envs.seed(args.seed)\n test_collector.reset()\n collector_stats = test_collector.collect(n_episode=args.num_test_envs, render=args.render)\n print(collector_stats)\n\n\nif __name__ == \"__main__\":\n test_dqn(get_args())\n" }, { "path": "examples/box2d/bipedal_bdq.py", "content": "import argparse\nimport datetime\nimport os\nimport pprint\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom tianshou.algorithm import BDQN\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.bdqn import BDQNPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env import ContinuousToDiscrete, SubprocVectorEnv\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import BranchingNet\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n # task\n parser.add_argument(\"--task\", type=str, default=\"BipedalWalker-v3\")\n # network architecture\n parser.add_argument(\"--common_hidden_sizes\", type=int, nargs=\"*\", default=[512, 256])\n parser.add_argument(\"--action_hidden_sizes\", type=int, nargs=\"*\", default=[128])\n parser.add_argument(\"--value_hidden_sizes\", type=int, nargs=\"*\", default=[128])\n parser.add_argument(\"--action_per_branch\", type=int, default=25)\n # training hyperparameters\n parser.add_argument(\"--seed\", type=int, default=0)\n parser.add_argument(\"--eps_test\", type=float, default=0.0)\n parser.add_argument(\"--eps_train\", type=float, default=0.73)\n parser.add_argument(\"--eps_decay\", type=float, default=5e-6)\n parser.add_argument(\"--buffer_size\", type=int, default=100000)\n parser.add_argument(\"--lr\", type=float, default=1e-4)\n parser.add_argument(\"--gamma\", type=float, default=0.99)\n parser.add_argument(\"--target_update_freq\", type=int, default=1000)\n parser.add_argument(\"--epoch\", type=int, default=25)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=80000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=16)\n parser.add_argument(\"--update_per_step\", type=float, default=0.0625)\n parser.add_argument(\"--batch_size\", type=int, default=512)\n parser.add_argument(\"--num_training_envs\", type=int, default=20)\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\n # other\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n return parser.parse_args()\n\n\ndef run_bdq(args: argparse.Namespace = get_args()) -> None:\n env = gym.make(args.task)\n env = ContinuousToDiscrete(env, args.action_per_branch)\n\n assert isinstance(env.action_space, gym.spaces.MultiDiscrete)\n assert isinstance(\n env.observation_space,\n gym.spaces.Box,\n ) # BipedalWalker-v3 has `Box` observation space by design\n args.state_shape = env.observation_space.shape\n args.action_shape = env.action_space.shape\n args.num_branches = args.action_shape[0]\n\n print(\"Observations shape:\", args.state_shape)\n print(\"Num branches:\", args.num_branches)\n print(\"Actions per branch:\", args.action_per_branch)\n\n # training_envs = ContinuousToDiscrete(gym.make(args.task), args.action_per_branch)\n # you can also use tianshou.env.SubprocVectorEnv\n training_envs = SubprocVectorEnv(\n [\n lambda: ContinuousToDiscrete(gym.make(args.task), args.action_per_branch)\n for _ in range(args.num_training_envs)\n ],\n )\n # test_envs = ContinuousToDiscrete(gym.make(args.task), args.action_per_branch)\n test_envs = SubprocVectorEnv(\n [\n lambda: ContinuousToDiscrete(gym.make(args.task), args.action_per_branch)\n for _ in range(args.num_test_envs)\n ],\n )\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n # model\n net = BranchingNet(\n state_shape=args.state_shape,\n num_branches=args.num_branches,\n action_per_branch=args.action_per_branch,\n common_hidden_sizes=args.common_hidden_sizes,\n value_hidden_sizes=args.value_hidden_sizes,\n action_hidden_sizes=args.action_hidden_sizes,\n ).to(args.device)\n optim = AdamOptimizerFactory(lr=args.lr)\n policy = BDQNPolicy(\n model=net,\n # TODO: should `BranchingDQNPolicy` support also `MultiDiscrete` action spaces?\n action_space=env.action_space, # type: ignore[arg-type]\n eps_training=args.eps_train,\n eps_inference=args.eps_test,\n )\n algorithm: BDQN = BDQN(\n policy=policy,\n optim=optim,\n gamma=args.gamma,\n target_update_freq=args.target_update_freq,\n )\n # collector\n training_collector = Collector[CollectStats](\n algorithm,\n training_envs,\n VectorReplayBuffer(args.buffer_size, len(training_envs)),\n exploration_noise=True,\n )\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=False)\n training_collector.reset()\n training_collector.collect(n_step=args.batch_size * args.num_training_envs)\n # log\n current_time = datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\")\n log_path = os.path.join(args.logdir, \"bdq\", args.task, current_time)\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n if env.spec and env.spec.reward_threshold:\n return mean_rewards >= env.spec.reward_threshold\n return False\n\n def train_fn(epoch: int, env_step: int) -> None: # exp decay\n eps = max(args.eps_train * (1 - args.eps_decay) ** env_step, args.eps_test)\n policy.set_eps_training(eps)\n\n # trainer\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n update_step_num_gradient_steps_per_sample=args.update_per_step,\n stop_fn=stop_fn,\n training_fn=train_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=True,\n )\n )\n\n assert stop_fn(result.best_reward)\n if __name__ == \"__main__\":\n pprint.pprint(result)\n # Let's watch its performance!\n policy.set_eps_training(args.eps_test)\n test_envs.seed(args.seed)\n test_collector.reset()\n collector_stats = test_collector.collect(n_episode=args.num_test_envs, render=args.render)\n print(collector_stats)\n\n\nif __name__ == \"__main__\":\n run_bdq(get_args())\n" }, { "path": "examples/box2d/bipedal_hardcore_sac.py", "content": "import argparse\nimport os\nimport pprint\nfrom typing import Any\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom gymnasium.core import WrapperActType, WrapperObsType\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom tianshou.algorithm import SAC\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.sac import AutoAlpha, SACPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env import SubprocVectorEnv\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.continuous import ContinuousActorProbabilistic, ContinuousCritic\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"BipedalWalkerHardcore-v3\")\n parser.add_argument(\"--seed\", type=int, default=0)\n parser.add_argument(\"--buffer_size\", type=int, default=1000000)\n parser.add_argument(\"--actor_lr\", type=float, default=3e-4)\n parser.add_argument(\"--critic_lr\", type=float, default=1e-3)\n parser.add_argument(\"--gamma\", type=float, default=0.99)\n parser.add_argument(\"--tau\", type=float, default=0.005)\n parser.add_argument(\"--alpha\", type=float, default=0.1)\n parser.add_argument(\"--auto_alpha\", type=int, default=1)\n parser.add_argument(\"--alpha_lr\", type=float, default=3e-4)\n parser.add_argument(\"--epoch\", type=int, default=100)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=100000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=10)\n parser.add_argument(\"--update_per_step\", type=float, default=0.1)\n parser.add_argument(\"--batch_size\", type=int, default=128)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[128, 128])\n parser.add_argument(\"--num_training_envs\", type=int, default=10)\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\"--n_step\", type=int, default=4)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n parser.add_argument(\"--resume_path\", type=str, default=None)\n return parser.parse_args()\n\n\nclass Wrapper(gym.Wrapper):\n \"\"\"Env wrapper for reward scale, action repeat and removing done penalty.\"\"\"\n\n def __init__(\n self,\n env: gym.Env,\n action_repeat: int = 3,\n reward_scale: int = 5,\n rm_done: bool = True,\n ) -> None:\n super().__init__(env)\n self.action_repeat = action_repeat\n self.reward_scale = reward_scale\n self.rm_done = rm_done\n\n def step(\n self,\n action: WrapperActType,\n ) -> tuple[WrapperObsType, float, bool, bool, dict[str, Any]]:\n rew_sum = 0.0\n for _ in range(self.action_repeat):\n obs, rew, terminated, truncated, info = self.env.step(action)\n done = terminated | truncated\n # remove done reward penalty\n if not done or not self.rm_done:\n rew_sum = rew_sum + float(rew)\n if done:\n break\n # scale reward\n return obs, self.reward_scale * rew_sum, terminated, truncated, info\n\n\ndef test_sac_bipedal(args: argparse.Namespace = get_args()) -> None:\n env = Wrapper(gym.make(args.task))\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n args.max_action = space_info.action_info.max_action\n training_envs = SubprocVectorEnv(\n [lambda: Wrapper(gym.make(args.task)) for _ in range(args.num_training_envs)],\n )\n # test_envs = gym.make(args.task)\n test_envs = SubprocVectorEnv(\n [\n lambda: Wrapper(gym.make(args.task), reward_scale=1, rm_done=False)\n for _ in range(args.num_test_envs)\n ],\n )\n\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n\n # model\n net_a = Net(state_shape=args.state_shape, hidden_sizes=args.hidden_sizes)\n actor = ContinuousActorProbabilistic(\n preprocess_net=net_a,\n action_shape=args.action_shape,\n unbounded=True,\n ).to(args.device)\n actor_optim = AdamOptimizerFactory(lr=args.actor_lr)\n\n net_c1 = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n concat=True,\n )\n critic1 = ContinuousCritic(preprocess_net=net_c1).to(args.device)\n critic1_optim = AdamOptimizerFactory(lr=args.critic_lr)\n\n net_c2 = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n concat=True,\n )\n critic2 = ContinuousCritic(preprocess_net=net_c2).to(args.device)\n critic2_optim = AdamOptimizerFactory(lr=args.critic_lr)\n\n action_dim = space_info.action_info.action_dim\n if args.auto_alpha:\n target_entropy = -action_dim\n log_alpha = 0.0\n alpha_optim = AdamOptimizerFactory(lr=args.alpha_lr)\n args.alpha = AutoAlpha(target_entropy, log_alpha, alpha_optim).to(args.device)\n\n policy = SACPolicy(\n actor=actor,\n action_space=env.action_space,\n )\n algorithm: SAC = SAC(\n policy=policy,\n policy_optim=actor_optim,\n critic=critic1,\n critic_optim=critic1_optim,\n critic2=critic2,\n critic2_optim=critic2_optim,\n tau=args.tau,\n gamma=args.gamma,\n alpha=args.alpha,\n n_step_return_horizon=args.n_step,\n )\n # load a previous policy\n if args.resume_path:\n algorithm.load_state_dict(torch.load(args.resume_path))\n print(\"Loaded agent from: \", args.resume_path)\n\n # collector\n training_collector = Collector[CollectStats](\n algorithm,\n training_envs,\n VectorReplayBuffer(args.buffer_size, len(training_envs)),\n exploration_noise=True,\n )\n test_collector = Collector[CollectStats](algorithm, test_envs)\n # training_collector.collect(n_step=args.buffer_size)\n # log\n log_path = os.path.join(args.logdir, args.task, \"sac\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n if env.spec:\n if not env.spec.reward_threshold:\n return False\n else:\n return mean_rewards >= env.spec.reward_threshold\n return False\n\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n update_step_num_gradient_steps_per_sample=args.update_per_step,\n test_in_training=False,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n )\n )\n\n if __name__ == \"__main__\":\n pprint.pprint(result)\n # Let's watch its performance!\n test_envs.seed(args.seed)\n test_collector.reset()\n collector_stats = test_collector.collect(n_episode=args.num_test_envs, render=args.render)\n print(collector_stats)\n\n\nif __name__ == \"__main__\":\n test_sac_bipedal()\n" }, { "path": "examples/box2d/lunarlander_dqn.py", "content": "import argparse\nimport os\nimport pprint\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom tianshou.algorithm import DQN\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.dqn import DiscreteQLearningPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env import DummyVectorEnv, SubprocVectorEnv\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n # the parameters are found by Optuna\n parser.add_argument(\"--task\", type=str, default=\"LunarLander-v2\")\n parser.add_argument(\"--seed\", type=int, default=0)\n parser.add_argument(\"--eps_test\", type=float, default=0.01)\n parser.add_argument(\"--eps_train\", type=float, default=0.73)\n parser.add_argument(\"--buffer_size\", type=int, default=100000)\n parser.add_argument(\"--lr\", type=float, default=0.013)\n parser.add_argument(\"--gamma\", type=float, default=0.99)\n parser.add_argument(\"--n_step\", type=int, default=4)\n parser.add_argument(\"--target_update_freq\", type=int, default=500)\n parser.add_argument(\"--epoch\", type=int, default=10)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=80000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=16)\n parser.add_argument(\"--update_per_step\", type=float, default=0.0625)\n parser.add_argument(\"--batch_size\", type=int, default=128)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[128, 128])\n parser.add_argument(\"--dueling_q_hidden_sizes\", type=int, nargs=\"*\", default=[128, 128])\n parser.add_argument(\"--dueling_v_hidden_sizes\", type=int, nargs=\"*\", default=[128, 128])\n parser.add_argument(\"--num_training_envs\", type=int, default=16)\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n return parser.parse_args()\n\n\ndef test_dqn(args: argparse.Namespace = get_args()) -> None:\n env = gym.make(args.task)\n assert isinstance(env.action_space, gym.spaces.Discrete)\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n args.max_action = space_info.action_info.max_action\n # training_envs = gym.make(args.task)\n # you can also use tianshou.env.SubprocVectorEnv\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n # test_envs = gym.make(args.task)\n test_envs = SubprocVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n # model\n Q_param = {\"hidden_sizes\": args.dueling_q_hidden_sizes}\n V_param = {\"hidden_sizes\": args.dueling_v_hidden_sizes}\n net = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n dueling_param=(Q_param, V_param),\n ).to(args.device)\n optim = AdamOptimizerFactory(lr=args.lr)\n policy = DiscreteQLearningPolicy(\n model=net,\n action_space=env.action_space,\n eps_training=args.eps_train,\n eps_inference=args.eps_test,\n )\n algorithm: DQN = DQN(\n policy=policy,\n optim=optim,\n gamma=args.gamma,\n n_step_return_horizon=args.n_step,\n target_update_freq=args.target_update_freq,\n )\n # collector\n training_collector = Collector[CollectStats](\n algorithm,\n training_envs,\n VectorReplayBuffer(args.buffer_size, len(training_envs)),\n exploration_noise=True,\n )\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n training_collector.reset()\n training_collector.collect(n_step=args.batch_size * args.num_training_envs)\n # log\n log_path = os.path.join(args.logdir, args.task, \"dqn\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n if env.spec:\n if not env.spec.reward_threshold:\n return False\n else:\n return mean_rewards >= env.spec.reward_threshold\n return False\n\n def train_fn(epoch: int, env_step: int) -> None: # exp decay\n eps = max(args.eps_train * (1 - 5e-6) ** env_step, args.eps_test)\n policy.set_eps_training(eps)\n\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n update_step_num_gradient_steps_per_sample=args.update_per_step,\n stop_fn=stop_fn,\n training_fn=train_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=True,\n )\n )\n\n assert stop_fn(result.best_reward)\n if __name__ == \"__main__\":\n pprint.pprint(result)\n # Let's watch its performance!\n test_envs.seed(args.seed)\n test_collector.reset()\n collector_stats = test_collector.collect(n_episode=args.num_test_envs, render=args.render)\n print(collector_stats)\n\n\nif __name__ == \"__main__\":\n test_dqn(get_args())\n" }, { "path": "examples/box2d/mcc_sac.py", "content": "import argparse\nimport os\nimport pprint\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom tianshou.algorithm import SAC\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.sac import AutoAlpha, SACPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.exploration import OUNoise\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.continuous import ContinuousActorProbabilistic, ContinuousCritic\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"MountainCarContinuous-v0\")\n parser.add_argument(\"--seed\", type=int, default=1626)\n parser.add_argument(\"--buffer_size\", type=int, default=50000)\n parser.add_argument(\"--actor_lr\", type=float, default=3e-4)\n parser.add_argument(\"--critic_lr\", type=float, default=3e-4)\n parser.add_argument(\"--alpha_lr\", type=float, default=3e-4)\n parser.add_argument(\"--noise_std\", type=float, default=1.2)\n parser.add_argument(\"--gamma\", type=float, default=0.99)\n parser.add_argument(\"--tau\", type=float, default=0.005)\n parser.add_argument(\"--auto_alpha\", type=int, default=1)\n parser.add_argument(\"--alpha\", type=float, default=0.2)\n parser.add_argument(\"--epoch\", type=int, default=20)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=12000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=5)\n parser.add_argument(\"--update_per_step\", type=float, default=0.2)\n parser.add_argument(\"--batch_size\", type=int, default=128)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[128, 128])\n parser.add_argument(\"--num_training_envs\", type=int, default=5)\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n return parser.parse_args()\n\n\ndef test_sac(args: argparse.Namespace = get_args()) -> None:\n env = gym.make(args.task)\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n args.max_action = space_info.action_info.max_action\n # training_envs = gym.make(args.task)\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n # test_envs = gym.make(args.task)\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n # model\n net = Net(state_shape=args.state_shape, hidden_sizes=args.hidden_sizes)\n actor = ContinuousActorProbabilistic(\n preprocess_net=net, action_shape=args.action_shape, unbounded=True\n ).to(args.device)\n actor_optim = AdamOptimizerFactory(lr=args.actor_lr)\n net_c1 = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n concat=True,\n )\n critic1 = ContinuousCritic(preprocess_net=net_c1).to(args.device)\n critic1_optim = AdamOptimizerFactory(lr=args.critic_lr)\n net_c2 = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n concat=True,\n )\n critic2 = ContinuousCritic(preprocess_net=net_c2).to(args.device)\n critic2_optim = AdamOptimizerFactory(lr=args.critic_lr)\n\n action_dim = space_info.action_info.action_dim\n if args.auto_alpha:\n target_entropy = -action_dim\n log_alpha = 0.0\n alpha_optim = AdamOptimizerFactory(lr=args.alpha_lr)\n args.alpha = AutoAlpha(target_entropy, log_alpha, alpha_optim).to(args.device)\n\n policy = SACPolicy(\n actor=actor,\n exploration_noise=OUNoise(0.0, args.noise_std),\n action_space=env.action_space,\n )\n algorithm: SAC = SAC(\n policy=policy,\n policy_optim=actor_optim,\n critic=critic1,\n critic_optim=critic1_optim,\n critic2=critic2,\n critic2_optim=critic2_optim,\n tau=args.tau,\n gamma=args.gamma,\n alpha=args.alpha,\n )\n # collector\n training_collector = Collector[CollectStats](\n algorithm,\n training_envs,\n VectorReplayBuffer(args.buffer_size, len(training_envs)),\n exploration_noise=True,\n )\n test_collector = Collector[CollectStats](algorithm, test_envs)\n # training_collector.collect(n_step=args.buffer_size)\n # log\n log_path = os.path.join(args.logdir, args.task, \"sac\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n if env.spec:\n if not env.spec.reward_threshold:\n return False\n else:\n return mean_rewards >= env.spec.reward_threshold\n return False\n\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n update_step_num_gradient_steps_per_sample=args.update_per_step,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=True,\n )\n )\n\n assert stop_fn(result.best_reward)\n if __name__ == \"__main__\":\n pprint.pprint(result)\n # Let's watch its performance!\n test_envs.seed(args.seed)\n test_collector.reset()\n collector_stats = test_collector.collect(n_episode=args.num_test_envs, render=args.render)\n print(collector_stats)\n\n\nif __name__ == \"__main__\":\n test_sac()\n" }, { "path": "examples/discrete/discrete_dqn.py", "content": "import gymnasium as gym\nfrom torch.utils.tensorboard import SummaryWriter\n\nimport tianshou as ts\nfrom tianshou.algorithm.modelfree.dqn import DiscreteQLearningPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import CollectStats\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef main() -> None:\n task = \"CartPole-v1\"\n lr, epoch, batch_size = 1e-3, 10, 64\n num_training_envs, num_test_envs = 10, 100\n gamma, n_step, target_freq = 0.9, 3, 320\n buffer_size = 20000\n eps_train, eps_test = 0.1, 0.05\n epoch_num_steps, collection_step_num_env_steps = 10000, 10\n\n logger = ts.utils.TensorboardLogger(SummaryWriter(\"log/dqn\")) # TensorBoard is supported!\n # For other loggers, see https://tianshou.readthedocs.io/en/master/tutorials/logger.html\n\n # Create the environments\n # You can also try SubprocVectorEnv, which will use parallelization\n training_envs = ts.env.DummyVectorEnv(\n [lambda: gym.make(task) for _ in range(num_training_envs)]\n )\n test_envs = ts.env.DummyVectorEnv([lambda: gym.make(task) for _ in range(num_test_envs)])\n\n # Create the network and optimizer\n # Note: You can easily define other networks.\n # See https://tianshou.readthedocs.io/en/master/01_tutorials/00_dqn.html#build-the-network\n env = gym.make(task, render_mode=\"human\")\n assert isinstance(env.action_space, gym.spaces.Discrete)\n space_info = SpaceInfo.from_env(env)\n state_shape = space_info.observation_info.obs_shape\n action_shape = space_info.action_info.action_shape\n net = Net(state_shape=state_shape, action_shape=action_shape, hidden_sizes=[128, 128, 128])\n optim = AdamOptimizerFactory(lr=lr)\n\n policy = DiscreteQLearningPolicy(\n model=net,\n action_space=env.action_space,\n eps_training=eps_train,\n eps_inference=eps_test,\n )\n algorithm = ts.algorithm.DQN(\n policy=policy,\n optim=optim,\n gamma=gamma,\n n_step_return_horizon=n_step,\n target_update_freq=target_freq,\n )\n training_collector = ts.data.Collector[CollectStats](\n algorithm,\n training_envs,\n ts.data.VectorReplayBuffer(buffer_size, num_training_envs),\n exploration_noise=True,\n )\n test_collector = ts.data.Collector[CollectStats](\n algorithm,\n test_envs,\n exploration_noise=True,\n ) # because DQN uses epsilon-greedy method\n\n def stop_fn(mean_rewards: float) -> bool:\n if env.spec:\n if not env.spec.reward_threshold:\n return False\n else:\n return mean_rewards >= env.spec.reward_threshold\n return False\n\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n collection_step_num_env_steps=collection_step_num_env_steps,\n test_step_num_episodes=num_test_envs,\n batch_size=batch_size,\n update_step_num_gradient_steps_per_sample=1 / collection_step_num_env_steps,\n stop_fn=stop_fn,\n logger=logger,\n test_in_training=True,\n )\n )\n print(f\"Finished training in {result.timing.total_time} seconds\")\n\n # watch performance\n collector = ts.data.Collector[CollectStats](algorithm, env, exploration_noise=True)\n collector.collect(n_episode=100, render=1 / 35)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "examples/discrete/discrete_dqn_hl.py", "content": "#!/usr/bin/env python3\n\nimport os\nfrom typing import Literal\n\nfrom sensai.util import logging\n\nfrom tianshou.highlevel.config import OffPolicyTrainingConfig\nfrom tianshou.highlevel.env import (\n EnvFactoryRegistered,\n VectorEnvType,\n)\nfrom tianshou.highlevel.experiment import DQNExperimentBuilder, ExperimentConfig\nfrom tianshou.highlevel.params.algorithm_params import DQNParams\n\n\ndef main(\n task: str = \"CartPole-v1\",\n persistence_base_dir: str = \"log\",\n num_experiments: int = 1,\n experiment_launcher: Literal[\"sequential\", \"joblib\"] = \"sequential\",\n) -> None:\n \"\"\"\n Train an agent using DQN on a specified discrete task, potentially running multiple experiments with different seeds\n and evaluating the results using rliable.\n\n :param task: the discrete task to train on.\n :param persistence_base_dir: the base directory for logging and saving experiment data,\n the task name will be appended to it.\n :param num_experiments: the number of experiments to run. The experiments differ exclusively in the seeds.\n :param experiment_launcher: the type of experiment launcher to use, only has an effect if `num_experiments>1`.\n You can use \"joblib\" for parallel execution of whole experiments.\n \"\"\"\n persistence_base_dir = os.path.abspath(os.path.join(persistence_base_dir, task))\n experiment_config = ExperimentConfig(persistence_base_dir=persistence_base_dir, watch=False)\n\n training_config = OffPolicyTrainingConfig(\n max_epochs=10,\n epoch_num_steps=10000,\n num_training_envs=10,\n num_test_envs=100,\n buffer_size=20000,\n batch_size=64,\n collection_step_num_env_steps=10,\n update_step_num_gradient_steps_per_sample=1 / 10,\n start_timesteps=0,\n start_timesteps_random=False,\n )\n\n env_factory = EnvFactoryRegistered(\n task=task, venv_type=VectorEnvType.DUMMY, training_seed=0, test_seed=10\n )\n\n hidden_sizes = (64, 64)\n experiment_builder = (\n DQNExperimentBuilder(env_factory, experiment_config, training_config)\n .with_dqn_params(\n DQNParams(\n lr=1e-3,\n gamma=0.9,\n n_step_return_horizon=3,\n target_update_freq=320,\n eps_training=0.3,\n eps_inference=0.0,\n ),\n )\n .with_model_factory_default(hidden_sizes=hidden_sizes)\n )\n\n experiment_builder.build_and_run(num_experiments=num_experiments, launcher=experiment_launcher)\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/inverse/README.md", "content": "# Inverse Reinforcement Learning\n\nIn inverse reinforcement learning setting, the agent learns a policy from interaction with an environment without reward\nand a fixed dataset which is collected with an expert policy.\n\n## Continuous control\n\nOnce the dataset is collected, it will not be changed during training. We\nuse [d4rl](https://github.com/rail-berkeley/d4rl) datasets to train agent for continuous control. You can refer\nto [d4rl](https://github.com/rail-berkeley/d4rl) to see how to use d4rl datasets.\n\nWe provide implementation of GAIL algorithm for continuous control.\n\n### Train\n\nYou can parse d4rl datasets into a `ReplayBuffer` , and set it as the parameter `expert_buffer` of `GAILPolicy`.\n`irl_gail.py` is an example of inverse RL using the d4rl dataset.\n\nTo train an agent with BCQ algorithm:\n\n```bash\npython irl_gail.py --task HalfCheetah-v2 --expert-data-task halfcheetah-expert-v2\n```\n\n## GAIL (single run)\n\n| task | best reward | reward curve | parameters |\n|----------------|-------------|------------------------------------------|------------------------------------------------------------------------------------------|\n| HalfCheetah-v2 | 5177.07 | ![](results/gail/HalfCheetah-v2_rew.png) | `python3 irl_gail.py --task \"HalfCheetah-v2\" --expert-data-task \"halfcheetah-expert-v2\"` |\n| Hopper-v2 | 1761.44 | ![](results/gail/Hopper-v2_rew.png) | `python3 irl_gail.py --task \"Hopper-v2\" --expert-data-task \"hopper-expert-v2\"` |\n| Walker2d-v2 | 2020.77 | ![](results/gail/Walker2d-v2_rew.png) | `python3 irl_gail.py --task \"Walker2d-v2\" --expert-data-task \"walker2d-expert-v2\"` |\n" }, { "path": "examples/inverse/irl_gail.py", "content": "#!/usr/bin/env python3\n\nimport argparse\nimport datetime\nimport os\nimport pprint\nfrom typing import SupportsFloat, cast\n\nimport d4rl\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch import nn\nfrom torch.distributions import Distribution, Independent, Normal\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom tianshou.algorithm import GAIL\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.reinforce import ProbabilisticActorPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory, LRSchedulerFactoryLinear\nfrom tianshou.data import (\n Batch,\n Collector,\n CollectStats,\n ReplayBuffer,\n VectorReplayBuffer,\n)\nfrom tianshou.data.types import RolloutBatchProtocol\nfrom tianshou.env import SubprocVectorEnv, VectorEnvNormObs\nfrom tianshou.trainer import OnPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.continuous import ContinuousActorProbabilistic, ContinuousCritic\nfrom tianshou.utils.space_info import SpaceInfo\n\n\nclass NoRewardEnv(gym.RewardWrapper):\n \"\"\"sets the reward to 0.\n\n :param gym.Env env: the environment to wrap.\n \"\"\"\n\n def __init__(self, env: gym.Env) -> None:\n super().__init__(env)\n\n def reward(self, reward: SupportsFloat) -> np.ndarray:\n \"\"\"Set reward to 0.\"\"\"\n return np.zeros_like(reward)\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"HalfCheetah-v2\")\n parser.add_argument(\"--seed\", type=int, default=0)\n parser.add_argument(\"--expert_data_task\", type=str, default=\"halfcheetah-expert-v2\")\n parser.add_argument(\"--buffer_size\", type=int, default=4096)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[64, 64])\n parser.add_argument(\"--lr\", type=float, default=3e-4)\n parser.add_argument(\"--disc_lr\", type=float, default=2.5e-5)\n parser.add_argument(\"--gamma\", type=float, default=0.99)\n parser.add_argument(\"--epoch\", type=int, default=100)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=30000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=2048)\n parser.add_argument(\"--update_step_num_repetitions\", type=int, default=10)\n parser.add_argument(\"--disc_update_num\", type=int, default=2)\n parser.add_argument(\"--batch_size\", type=int, default=64)\n parser.add_argument(\"--num_training_envs\", type=int, default=64)\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\n # ppo special\n parser.add_argument(\"--return_scaling\", type=int, default=True)\n # In theory, `vf-coef` will not make any difference if using Adam optimizer.\n parser.add_argument(\"--vf_coef\", type=float, default=0.25)\n parser.add_argument(\"--ent_coef\", type=float, default=0.001)\n parser.add_argument(\"--gae_lambda\", type=float, default=0.95)\n parser.add_argument(\"--bound_action_method\", type=str, default=\"clip\")\n parser.add_argument(\"--lr_decay\", type=int, default=True)\n parser.add_argument(\"--max_grad_norm\", type=float, default=0.5)\n parser.add_argument(\"--eps_clip\", type=float, default=0.2)\n parser.add_argument(\"--dual_clip\", type=float, default=None)\n parser.add_argument(\"--value_clip\", type=int, default=0)\n parser.add_argument(\"--advantage_normalization\", type=int, default=0)\n parser.add_argument(\"--recompute_adv\", type=int, default=1)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n parser.add_argument(\"--resume_path\", type=str, default=None)\n parser.add_argument(\n \"--watch\",\n default=False,\n action=\"store_true\",\n help=\"watch the play of pre-trained policy only\",\n )\n return parser.parse_args()\n\n\ndef test_gail(args: argparse.Namespace = get_args()) -> None:\n env = gym.make(args.task)\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n args.max_action = space_info.action_info.max_action\n print(\"Observations shape:\", args.state_shape)\n print(\"Actions shape:\", args.action_shape)\n print(\"Action range:\", args.min_action, args.max_action)\n # training_envs = gym.make(args.task)\n training_envs = SubprocVectorEnv(\n [lambda: NoRewardEnv(gym.make(args.task)) for _ in range(args.num_training_envs)],\n )\n training_envs = VectorEnvNormObs(training_envs)\n # test_envs = gym.make(args.task)\n test_envs = SubprocVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n test_envs = VectorEnvNormObs(test_envs, update_obs_rms=False)\n test_envs.set_obs_rms(training_envs.get_obs_rms())\n\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n # model\n net_a = Net(\n state_shape=args.state_shape,\n hidden_sizes=args.hidden_sizes,\n activation=nn.Tanh,\n )\n actor = ContinuousActorProbabilistic(\n preprocess_net=net_a,\n action_shape=args.action_shape,\n unbounded=True,\n ).to(args.device)\n net_c = Net(\n state_shape=args.state_shape,\n hidden_sizes=args.hidden_sizes,\n activation=nn.Tanh,\n )\n critic = ContinuousCritic(preprocess_net=net_c).to(args.device)\n torch.nn.init.constant_(actor.sigma_param, -0.5)\n for m in list(actor.modules()) + list(critic.modules()):\n if isinstance(m, torch.nn.Linear):\n # orthogonal initialization\n torch.nn.init.orthogonal_(m.weight, gain=np.sqrt(2))\n torch.nn.init.zeros_(m.bias)\n # do last policy layer scaling, this will make initial actions have (close to)\n # 0 mean and std, and will help boost performances,\n # see https://arxiv.org/abs/2006.05990, Fig.24 for details\n for m in actor.mu.modules():\n if isinstance(m, torch.nn.Linear):\n torch.nn.init.zeros_(m.bias)\n m.weight.data.copy_(0.01 * m.weight.data)\n\n optim = AdamOptimizerFactory(lr=args.lr)\n # discriminator\n net_d = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n activation=nn.Tanh,\n concat=True,\n )\n disc_net = ContinuousCritic(preprocess_net=net_d).to(args.device)\n for m in disc_net.modules():\n if isinstance(m, torch.nn.Linear):\n # orthogonal initialization\n torch.nn.init.orthogonal_(m.weight, gain=np.sqrt(2))\n torch.nn.init.zeros_(m.bias)\n disc_optim = AdamOptimizerFactory(lr=args.disc_lr)\n\n if args.lr_decay:\n optim.with_lr_scheduler_factory(\n LRSchedulerFactoryLinear(\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n )\n )\n\n def dist(loc_scale: tuple[torch.Tensor, torch.Tensor]) -> Distribution:\n loc, scale = loc_scale\n return Independent(Normal(loc, scale), 1)\n\n # expert replay buffer\n dataset = d4rl.qlearning_dataset(gym.make(args.expert_data_task))\n dataset_size = dataset[\"rewards\"].size\n\n print(\"dataset_size\", dataset_size)\n expert_buffer = ReplayBuffer(dataset_size)\n\n for i in range(dataset_size):\n expert_buffer.add(\n cast(\n RolloutBatchProtocol,\n Batch(\n obs=dataset[\"observations\"][i],\n act=dataset[\"actions\"][i],\n rew=dataset[\"rewards\"][i],\n done=dataset[\"terminals\"][i],\n obs_next=dataset[\"next_observations\"][i],\n ),\n ),\n )\n print(\"dataset loaded\")\n\n policy = ProbabilisticActorPolicy(\n actor=actor,\n dist_fn=dist,\n action_scaling=True,\n action_bound_method=args.bound_action_method,\n action_space=env.action_space,\n )\n algorithm: GAIL = GAIL(\n policy=policy,\n critic=critic,\n optim=optim,\n expert_buffer=expert_buffer,\n disc_net=disc_net,\n disc_optim=disc_optim,\n disc_update_num=args.disc_update_num,\n gamma=args.gamma,\n gae_lambda=args.gae_lambda,\n max_grad_norm=args.max_grad_norm,\n vf_coef=args.vf_coef,\n ent_coef=args.ent_coef,\n return_scaling=args.return_scaling,\n eps_clip=args.eps_clip,\n value_clip=args.value_clip,\n dual_clip=args.dual_clip,\n advantage_normalization=args.advantage_normalization,\n recompute_advantage=args.recompute_adv,\n )\n\n # load a previous policy\n if args.resume_path:\n algorithm.load_state_dict(torch.load(args.resume_path, map_location=args.device))\n print(\"Loaded agent from: \", args.resume_path)\n\n # collector\n buffer: ReplayBuffer\n if args.num_training_envs > 1:\n buffer = VectorReplayBuffer(args.buffer_size, len(training_envs))\n else:\n buffer = ReplayBuffer(args.buffer_size)\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buffer, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs)\n # log\n t0 = datetime.datetime.now().strftime(\"%m%d_%H%M%S\")\n log_file = f\"seed_{args.seed}_{t0}-{args.task.replace('-', '_')}_gail\"\n log_path = os.path.join(args.logdir, args.task, \"gail\", log_file)\n writer = SummaryWriter(log_path)\n writer.add_text(\"args\", str(args))\n logger = TensorboardLogger(writer, update_interval=100, training_interval=100)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n if not args.watch:\n # train\n result = algorithm.run_training(\n OnPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n update_step_num_repetitions=args.update_step_num_repetitions,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=False,\n )\n )\n pprint.pprint(result)\n\n # Let's watch its performance!\n test_envs.seed(args.seed)\n test_collector.reset()\n collector_stats = test_collector.collect(n_episode=args.num_test_envs, render=args.render)\n print(collector_stats)\n\n\nif __name__ == \"__main__\":\n test_gail()\n" }, { "path": "examples/modelbased/README.md", "content": "# PSRL\n\n`NChain-v0`: `python3 psrl.py --task NChain-v0 --epoch_num_steps 10 --rew-mean-prior 0 --rew-std-prior 1`\n\n`FrozenLake-v0`:\n`python3 psrl.py --task FrozenLake-v0 --epoch_num_steps 1000 --rew-mean-prior 0 --rew-std-prior 1 --add-done-loop --epoch 20`\n\n`Taxi-v3`: `python3 psrl.py --task Taxi-v3 --epoch_num_steps 1000 --rew-mean-prior 0 --rew-std-prior 2 --epoch 20`\n" }, { "path": "examples/mujoco/README.md", "content": "# Tianshou's Mujoco Benchmark\n\nWe benchmarked Tianshou algorithm implementations in 9 out of 13 environments from the MuJoCo Gym task\nsuite[[1]](#footnote1).\n\nFor each supported algorithm and supported mujoco environments, we provide:\n\n- Default hyperparameters used for benchmark and scripts to reproduce the benchmark;\n- A comparison of performance (or code level details) with other open source implementations or classic papers;\n- Graphs and raw data that can be used for research purposes[[2]](#footnote2);\n- Log details obtained during training[[2]](#footnote2);\n- Pretrained agents[[2]](#footnote2);\n- Some hints on how to tune the algorithm.\n\nSupported algorithms are listed below:\n\n- [Deep Deterministic Policy Gradient (DDPG)](https://arxiv.org/pdf/1509.02971.pdf), [commit id](https://github.com/thu-ml/tianshou/tree/e605bdea942b408126ef4fbc740359773259c9ec)\n- [Twin Delayed DDPG (TD3)](https://arxiv.org/pdf/1802.09477.pdf), [commit id](https://github.com/thu-ml/tianshou/tree/e605bdea942b408126ef4fbc740359773259c9ec)\n- [Soft Actor-Critic (SAC)](https://arxiv.org/pdf/1812.05905.pdf), [commit id](https://github.com/thu-ml/tianshou/tree/e605bdea942b408126ef4fbc740359773259c9ec)\n- [REINFORCE algorithm](https://papers.nips.cc/paper/1999/file/464d828b85b0bed98e80ade0a5c43b0f-Paper.pdf), [commit id](https://github.com/thu-ml/tianshou/tree/e27b5a26f330de446fe15388bf81c3777f024fb9)\n- [Natural Policy Gradient](https://proceedings.neurips.cc/paper/2001/file/4b86abe48d358ecf194c56c69108433e-Paper.pdf), [commit id](https://github.com/thu-ml/tianshou/tree/844d7703c313009c4c364edb4018c91de93439ca)\n- [Advantage Actor-Critic (A2C)](https://openai.com/blog/baselines-acktr-a2c/), [commit id](https://github.com/thu-ml/tianshou/tree/1730a9008ad6bb67cac3b21347bed33b532b17bc)\n- [Proximal Policy Optimization (PPO)](https://arxiv.org/pdf/1707.06347.pdf), [commit id](https://github.com/thu-ml/tianshou/tree/6426a39796db052bafb7cabe85c764db20a722b0)\n- [Trust Region Policy Optimization (TRPO)](https://arxiv.org/pdf/1502.05477.pdf), [commit id](https://github.com/thu-ml/tianshou/tree/5057b5c89e6168220272c9c28a15b758a72efc32)\n- [Hindsight Experience Replay (HER)](https://arxiv.org/abs/1707.01495)\n\n## EnvPool\n\nWe highly recommend using envpool to run the following experiments. To install, in a linux machine, type:\n\n```bash\npip install envpool\n```\n\nAfter that, `make_mujoco_env` will automatically switch to envpool's Mujoco env. EnvPool's implementation is much\nfaster (about 2\\~3x faster for pure execution speed, 1.5x for overall RL training pipeline in average) than python\nvectorized env implementation, and it's behavior is consistent to gym's Mujoco env.\n\nFor more information, please refer to EnvPool's [GitHub](https://github.com/sail-sg/envpool/)\nand [Docs](https://envpool.readthedocs.io/en/latest/api/mujoco.html).\n\n## Usage\n\nRun\n\n```bash\n$ python mujoco_sac.py --task Ant-v3\n```\n\nLogs is saved in `./log/` and can be monitored with tensorboard.\n\n```bash\n$ tensorboard --logdir log\n```\n\nYou can also reproduce the benchmark (e.g. SAC in Ant-v3) with the example script we provide under `examples/mujoco/`:\n\n```bash\n$ ./run_experiments.sh Ant-v3 sac\n```\n\nThis will start 10 experiments with different seeds.\n\nNow that all the experiments are finished, we can convert all tfevent files into csv files and then try plotting the\nresults.\n\n```bash\n# generate csv\n$ ./tools.py --root-dir ./results/Ant-v3/sac\n# generate figures\n$ ./plotter.py --root-dir ./results/Ant-v3 --shaded-std --legend-pattern \"\\\\w+\"\n# generate numerical result (support multiple groups: `--root-dir ./` instead of single dir)\n$ ./analysis.py --root-dir ./results --norm\n```\n\n## Example benchmark\n\n\n\nOther graphs can be found under `examples/mujuco/benchmark/`\n\nFor pretrained agents, detailed graphs (single agent, single game) and log details, please refer\nto [https://cloud.tsinghua.edu.cn/d/f45fcfc5016043bc8fbc/](https://cloud.tsinghua.edu.cn/d/f45fcfc5016043bc8fbc/).\n\n## Offpolicy algorithms\n\n### Notes\n\n1. In offpolicy algorithms (DDPG, TD3, SAC), the shared hyperparameters are almost the same, and unless otherwise\n stated, hyperparameters are consistent with those used for benchmark in SpinningUp's implementations (e.g. we use\n batchsize 256 in DDPG/TD3/SAC while SpinningUp use 100. Minor difference also lies with `start-timesteps`, data loop\n method `collection_step_num_env_steps`, method to deal with/bootstrap truncated steps because of timelimit and\n unfinished/collecting episodes (contribute to performance improvement), etc.).\n2. By comparison to both classic literature and open source implementations (e.g.,\n SpinningUp)[[1]](#footnote1)[[2]](#footnote2), Tianshou's implementations of DDPG, TD3, and SAC\n are roughly at-parity with or better than the best reported results for these algorithms, so you can definitely use\n Tianshou's benchmark for research purposes.\n3. We didn't compare offpolicy algorithms to OpenAI\n baselines [benchmark](https://github.com/openai/baselines/blob/master/benchmarks_mujoco1M.htm), because for now it\n seems that they haven't provided benchmark for offpolicy algorithms, but\n in [SpinningUp docs](https://spinningup.openai.com/en/latest/spinningup/bench.html) they stated that \"SpinningUp\n implementations of DDPG, TD3, and SAC are roughly at-parity with the best-reported results for these algorithms\", so\n we think lack of comparisons with OpenAI baselines is okay.\n\n### DDPG\n\n| Environment | Tianshou (1M) | [Spinning Up (PyTorch)](https://spinningup.openai.com/en/latest/spinningup/bench.html) | [TD3 paper (DDPG)](https://arxiv.org/abs/1802.09477) | [TD3 paper (OurDDPG)](https://arxiv.org/abs/1802.09477) |\n|:----------------------:|:-----------------:|:--------------------------------------------------------------------------------------:|:----------------------------------------------------:|:-------------------------------------------------------:|\n| Ant | 990.4±4.3 | ~840 | **1005.3** | 888.8 |\n| HalfCheetah | **11718.7±465.6** | ~11000 | 3305.6 | 8577.3 |\n| Hopper | **2197.0±971.6** | ~1800 | **2020.5** | 1860.0 |\n| Walker2d | 1400.6±905.0 | ~1950 | 1843.6 | **3098.1** |\n| Swimmer | **144.1±6.5** | ~137 | N | N |\n| Humanoid | **177.3±77.6** | N | N | N |\n| Reacher | **-3.3±0.3** | N | -6.51 | -4.01 |\n| InvertedPendulum | **1000.0±0.0** | N | **1000.0** | **1000.0** |\n| InvertedDoublePendulum | 8364.3±2778.9 | N | **9355.5** | 8370.0 |\n\n\\* details[[4]](#footnote4)[[5]](#footnote5)[[6]](#footnote6)\n\n### TD3\n\n| Environment | Tianshou (1M) | [Spinning Up (PyTorch)](https://spinningup.openai.com/en/latest/spinningup/bench.html) | [TD3 paper](https://arxiv.org/abs/1802.09477) |\n|:----------------------:|:-----------------:|:--------------------------------------------------------------------------------------:|:---------------------------------------------:|\n| Ant | **5116.4±799.9** | ~3800 | 4372.4±1000.3 |\n| HalfCheetah | **10201.2±772.8** | ~9750 | 9637.0±859.1 |\n| Hopper | 3472.2±116.8 | ~2860 | **3564.1±114.7** |\n| Walker2d | 3982.4±274.5 | ~4000 | **4682.8±539.6** |\n| Swimmer | **104.2±34.2** | ~78 | N |\n| Humanoid | **5189.5±178.5** | N | N |\n| Reacher | **-2.7±0.2** | N | -3.6±0.6 |\n| InvertedPendulum | **1000.0±0.0** | N | **1000.0±0.0** |\n| InvertedDoublePendulum | **9349.2±14.3** | N | **9337.5±15.0** |\n\n\\* details[[4]](#footnote4)[[5]](#footnote5)[[6]](#footnote6)\n\n#### Hints for TD3\n\n1. TD3's learning rate is set to 3e-4 while it is 1e-3 for DDPG/SAC. However, there is NO enough evidence to support our\n choice of such hyperparameters (we simply choose them because SpinningUp do so) and you can try playing with those\n hyperparameters to see if you can improve performance. Do tell us if you can!\n\n### SAC\n\n| Environment | Tianshou (1M) | [Spinning Up (PyTorch)](https://spinningup.openai.com/en/latest/spinningup/bench.html) | [SAC paper](https://arxiv.org/abs/1801.01290) |\n|:----------------------:|:------------------:|:--------------------------------------------------------------------------------------:|:---------------------------------------------:|\n| Ant | **5850.2±475.7** | ~3980 | ~3720 |\n| HalfCheetah | **12138.8±1049.3** | ~11520 | ~10400 |\n| Hopper | **3542.2±51.5** | ~3150 | ~3370 |\n| Walker2d | **5007.0±251.5** | ~4250 | ~3740 |\n| Swimmer | **44.4±0.5** | ~41.7 | N |\n| Humanoid | **5488.5±81.2** | N | ~5200 |\n| Reacher | **-2.6±0.2** | N | N |\n| InvertedPendulum | **1000.0±0.0** | N | N |\n| InvertedDoublePendulum | **9359.5±0.4** | N | N |\n\n\\* details[[4]](#footnote4)[[5]](#footnote5)\n\n#### Hints for SAC\n\n1. SAC's start-timesteps is set to 10000 by default while it is 25000 is DDPG/TD3. However, there is NO enough evidence\n to support our choice of such hyperparameters (we simply choose them because SpinningUp do so) and you can try\n playing with those hyperparameters to see if you can improve performance. Do tell us if you can!\n2. DO NOT share the same network with two critic networks.\n3. The sigma (of the Gaussian policy) should be conditioned on input.\n4. The deterministic evaluation helps a lot :)\n\n## Onpolicy Algorithms\n\n### Notes\n\n1. In A2C and PPO, unless otherwise stated, most hyperparameters are consistent with those used for benchmark\n in [ikostrikov/pytorch-a2c-ppo-acktr-gail](https://github.com/ikostrikov/pytorch-a2c-ppo-acktr-gail).\n2. Gernally speaking, by comparison to both classic literature and open source implementations (e.g., OPENAI\n Baselines)[[1]](#footnote1)[[2]](#footnote2), Tianshou's implementations of REINFORCE, A2C, PPO\n are better than the best reported results for these algorithms, so you can definitely use Tianshou's benchmark for\n research purposes.\n\n### REINFORCE\n\n| Environment | Tianshou (10M) |\n|:----------------------:|:-----------------:|\n| Ant | **1108.1±323.1** |\n| HalfCheetah | **1138.8±104.7** |\n| Hopper | **416.0±104.7** |\n| Walker2d | **440.9±148.2** |\n| Swimmer | **35.6±2.6** |\n| Humanoid | **464.3±58.4** |\n| Reacher | **-5.5±0.2** |\n| InvertedPendulum | **1000.0±0.0** |\n| InvertedDoublePendulum | **7726.2±1287.3** |\n\n| Environment | Tianshou (3M) | [Spinning Up (VPG PyTorch)](https://spinningup.openai.com/en/latest/spinningup/bench_vpg.html)[[7]](#footnote7) |\n|:----------------------:|:-----------------:|:--------------------------------------------------------------------------------------------------------------------------:|\n| Ant | **474.9+-133.5** | ~5 |\n| HalfCheetah | **884.0+-41.0** | ~600 |\n| Hopper | 395.8+-64.5\\* | **~800** |\n| Walker2d | 412.0+-52.4 | **~460** |\n| Swimmer | 35.3+-1.4 | **~51** |\n| Humanoid | **438.2+-47.8** | N |\n| Reacher | **-10.5+-0.7** | N |\n| InvertedPendulum | **999.2+-2.4** | N |\n| InvertedDoublePendulum | **1059.7+-307.7** | N |\n\n\\* details[[4]](#footnote4)[[5]](#footnote5)\n\n### Hints for REINFORCE\n\n1. Following [Andrychowicz, Marcin, et al](https://arxiv.org/abs/2006.05990), we downscale last layer of policy network\n by a factor of 0.01 after orthogonal initialization.\n2. We choose \"tanh\" function to squash sampled action from range (-inf, inf) to (-1, 1) rather than usually used\n clipping method (As in StableBaselines3). We did full scale ablation studies and results show that tanh squashing\n performs a tiny little bit better than clipping overall, and is much better than no action bounding. However, \"clip\"\n method is still a very good method, considering its simplicity.\n3. We use global observation normalization and global rew-to-go (value) normalization by default. Both are crucial to\n good performance of REINFORCE algorithm. Since we minus mean when doing rew-to-go normalization, you can treat global\n mean of rew-to-go as a naive version of \"baseline\".\n4. Since we do not have a value estimator, we use global rew-to-go mean to bootstrap truncated steps because of\n timelimit and unfinished collecting, while most other implementations use 0. We feel this would help because mean is\n more likely a better estimate than 0 (no ablation study has been done).\n5. We have done full scale ablation study on learning rate and lr decay strategy. We experiment with lr of 3e-4, 5e-4,\n 1e-3, each have 2 options: no lr decay or linear decay to 0. Experiments show that 3e-4 learning rate will cause\n slowly learning and make agent step in local optima easily for certain environments like InvertedDoublePendulum, Ant,\n HalfCheetah, and 1e-3 lr helps a lot. However, after training agents with lr 1e-3 for 5M steps or so, agents in\n certain environments like InvertedPendulum will become unstable. Conclusion is that we should start with a large\n learning rate and linearly decay it, but for a small initial learning rate or if you only train agents for limited\n timesteps, DO NOT decay it.\n6. We didn't tune `step-per-collect` option and `training-num` option. Default values are finetuned with PPO algorithm\n so we assume they are also good for REINFORCE. You can play with them if you want, but remember that `buffer-size`\n should always be larger than `step-per-collect`, and if `step-per-collect` is too small and `training-num` too large,\n episodes will be truncated and bootstrapped very often, which will harm performance. If `training-num` is too small (\n e.g., less than 8), speed will go down.\n7. Sigma of action is not fixed (normally seen in other implementation) or conditioned on observation, but is an\n independent parameter which can be updated by gradient descent. We choose this setting because it works well in PPO,\n and is recommended by [Andrychowicz, Marcin, et al](https://arxiv.org/abs/2006.05990). See Fig. 23.\n\n### A2C\n\n| Environment | Tianshou (3M) | [Spinning Up (PyTorch)](https://spinningup.openai.com/en/latest/spinningup/bench_vpg.html) |\n|:----------------------:|:------------------:|:------------------------------------------------------------------------------------------:|\n| Ant | **5236.8+-236.7** | ~5 |\n| HalfCheetah | **2377.3+-1363.7** | ~600 |\n| Hopper | **1608.6+-529.5** | ~800 |\n| Walker2d | **1805.4+-1055.9** | ~460 |\n| Swimmer | 40.2+-1.8 | **~51** |\n| Humanoid | **5316.6+-554.8** | N |\n| Reacher | **-5.2+-0.5** | N |\n| InvertedPendulum | **1000.0+-0.0** | N |\n| InvertedDoublePendulum | **9351.3+-12.8** | N |\n\n| Environment | Tianshou (1M) | [PPO paper](https://arxiv.org/abs/1707.06347) A2C | [PPO paper](https://arxiv.org/abs/1707.06347) A2C + Trust Region |\n|:----------------------:|:------------------:|:-------------------------------------------------:|:----------------------------------------------------------------:|\n| Ant | **3485.4+-433.1** | N | N |\n| HalfCheetah | **1829.9+-1068.3** | ~1000 | ~930 |\n| Hopper | **1253.2+-458.0** | ~900 | ~1220 |\n| Walker2d | **1091.6+-709.2** | ~850 | ~700 |\n| Swimmer | **36.6+-2.1** | ~31 | **~36** |\n| Humanoid | **1726.0+-1070.1** | N | N |\n| Reacher | **-6.7+-2.3** | ~-24 | ~-27 |\n| InvertedPendulum | **1000.0+-0.0** | **~1000** | **~1000** |\n| InvertedDoublePendulum | **9257.7+-277.4** | ~7100 | ~8100 |\n\n\\* details[[4]](#footnote4)[[5]](#footnote5)\n\n#### Hints for A2C\n\n1. We choose `clip` action method in A2C instead of `tanh` option as used in REINFORCE simply to be consistent with\n original implementation. `tanh` may be better or equally well but we didn't have a try.\n2. (Initial) learning rate, lr_decay, `step-per-collect` and `training-num` affect the performance of A2C to a great\n extend. These 4 hyperparameters also affect each other and should be tuned together. We have done full scale ablation\n studies on these 4 hyperparameters (more than 800 agents have been trained). Below are our findings.\n3. `step-per-collect` / `training-num` are equal to `bootstrap-lenghth`, which is the max length of an \"episode\" used in\n GAE estimator and 80/16=5 in default settings. When `bootstrap-lenghth` is small, (maybe) because GAE can look\n forward at most 5 steps and use bootstrap strategy very often, the critic is less well-trained leading the actor to a\n not very high score. However, if we increase `step-per-collect` to increase `bootstrap-lenghth` (e.g. 256/16=16),\n actor/critic will be updated less often, resulting in low sample efficiency and slow training process. To conclude,\n If you don't restrict env timesteps, you can try using larger `bootstrap-lenghth` and train with more steps to get a\n better converged score. Train slower, achieve higher.\n4. The learning rate 7e-4 with decay strategy is appropriate for `step-per-collect=80` and `training-num=16`. But if you\n use a larger `step-per-collect`(e.g. 256 - 2048), 7e-4 is a little bit small for `lr` because each update will have\n more data, less noise and thus smaller deviation in this case. So it is more appropriate to use a higher learning\n rate (e.g. 1e-3) to boost performance in this setting. If plotting results arise fast in early stages and become\n unstable later, consider lr decay first before decreasing lr.\n5. `max-grad-norm` didn't really help in our experiments. We simply keep it for consistency with other open-source\n implementations (e.g. SB3).\n6. Although original paper of A3C uses RMSprop optimizer, we found that Adam with the same learning rate worked equally\n well. We use RMSprop anyway. Again, for consistency.\n7. We noticed that the implementation of A2C in SB3 sets `gae-lambda` to 1 by default for no reason, and our experiments\n showed better results overall when `gae-lambda` was set to 0.95.\n8. We found out that `step-per-collect=256` and `training-num=8` are also good settings. You can have a try.\n\n### PPO\n\n| Environment | Tianshou (1M) | [ikostrikov/pytorch-a2c-ppo-acktr-gail](https://github.com/ikostrikov/pytorch-a2c-ppo-acktr-gail) | [PPO paper](https://arxiv.org/pdf/1707.06347.pdf) | [OpenAI Baselines](https://github.com/openai/baselines/blob/master/benchmarks_mujoco1M.htm) | [Spinning Up (PyTorch)](https://spinningup.openai.com/en/latest/spinningup/bench_ppo.html) |\n|:----------------------:|:------------------:|:-------------------------------------------------------------------------------------------------:|:-------------------------------------------------:|:-------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------:|\n| Ant | **3258.4+-1079.3** | N | N | N | ~650 |\n| HalfCheetah | **5783.9+-1244.0** | ~3120 | ~1800 | ~1700 | ~1670 |\n| Hopper | **2609.3+-700.8** | ~2300 | ~2330 | ~2400 | ~1850 |\n| Walker2d | 3588.5+-756.6 | **~4000** | ~3460 | ~3510 | ~1230 |\n| Swimmer | 66.7+-99.1 | N | ~108 | ~111 | **~120** |\n| Humanoid | **787.1+-193.5** | N | N | N | N |\n| Reacher | **-4.1+-0.3** | ~-5 | ~-7 | ~-6 | N |\n| InvertedPendulum | **1000.0+-0.0** | N | **~1000** | ~940 | N |\n| InvertedDoublePendulum | **9231.3+-270.4** | N | ~8000 | ~7350 | N |\n\n| Environment | Tianshou (3M) | [Spinning Up (PyTorch)](https://spinningup.openai.com/en/latest/spinningup/bench_ppo.html) |\n|:----------------------:|:------------------:|:------------------------------------------------------------------------------------------:|\n| Ant | **4079.3+-880.2** | ~3000 |\n| HalfCheetah | **7337.4+-1508.2** | ~3130 |\n| Hopper | **3127.7+-413.0** | ~2460 |\n| Walker2d | **4895.6+-704.3** | ~2600 |\n| Swimmer | 81.4+-96.0 | **~120** |\n| Humanoid | **1359.7+-572.7** | N |\n| Reacher | **-3.7+-0.3** | N |\n| InvertedPendulum | **1000.0+-0.0** | N |\n| InvertedDoublePendulum | **9231.3+-270.4** | N |\n\n\\* details[[4]](#footnote4)[[5]](#footnote5)\n\n#### Hints for PPO\n\n1. Following [Andrychowicz, Marcin, et al](https://arxiv.org/abs/2006.05990) Sec 3.5, we use \"recompute advantage\"\n strategy, which contributes a lot to our SOTA benchmark. However, I personally don't quite agree with their\n explanation about why \"recompute advantage\" helps. They stated that it's because old strategy \"makes it impossible to\n compute advantages as the temporal structure is broken\", but PPO's update equation is designed to learn from\n slightly-outdated advantages. I think the only reason \"recompute advantage\" works is that it update the critic\n several times rather than just one time per update, which leads to a better value function estimation.\n2. We have done full scale ablation studies of PPO algorithm's hyperparameters. Here are our findings: In Mujoco\n settings, `value-clip` and `norm-adv` may help a litte bit in some games (e.g. `norm-adv` helps stabilize training in\n InvertedPendulum-v2), but they make no difference to overall performance. So in our benchmark we do not use such\n tricks. We validate that setting `ent-coef` to 0.0 rather than 0.01 will increase overall performance in mujoco\n environments. `max-grad-norm` still offers no help for PPO algorithm, but we still keep it for consistency.\n3. [Andrychowicz, Marcin, et al](https://arxiv.org/abs/2006.05990)'s work indicates that using `gae-lambda` 0.9 and\n changing policy network's width based on which game you play (e.g. use [16, 16] `hidden-sizes` for `actor` network in\n HalfCheetah and [256, 256] for Ant) may help boost performance. Our ablation studies say otherwise: both options may\n lead to equal or lower performance overall in our experiments. We are not confident about this claim because we\n didn't change learning rate and other maybe-correlated factors in our experiments. So if you want, you can still have\n a try.\n4. `batch-size` 128 and 64 (default) work equally well. Changing `training-num` alone slightly (maybe in range [8, 128])\n won't affect performance. For bound action method, both `clip` and `tanh` work quite well.\n5. In OPENAI implementations of PPO, they multiply value loss with a factor of 0.5 for no good reason (see\n this [issue](https://github.com/openai/baselines/issues/445#issuecomment-777988738)). We do not do so and therefore\n make our `vf-coef` 0.25 (half of standard 0.5). However, since value loss is only used to optimize `critic` network,\n setting different `vf-coef` should in theory make no difference if using Adam optimizer.\n\n### TRPO\n\n| Environment | Tianshou (1M) | [ACKTR paper](https://arxiv.org/pdf/1708.05144.pdf) | [PPO paper](https://arxiv.org/pdf/1707.06347.pdf) | [OpenAI Baselines](https://github.com/openai/baselines/blob/master/benchmarks_mujoco1M.htm) | [Spinning Up (Tensorflow)](https://spinningup.openai.com/en/latest/spinningup/bench.html) |\n|:----------------------:|:-----------------:|:---------------------------------------------------:|:-------------------------------------------------:|:-------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------:|\n| Ant | **2866.7±707.9** | ~0 | N | N | ~150 |\n| HalfCheetah | **4471.2±804.9** | ~400 | ~0 | ~1350 | ~850 |\n| Hopper | 2046.0±1037.9 | ~1400 | ~2100 | **~2200** | ~1200 |\n| Walker2d | **3826.7±782.7** | ~550 | ~1100 | ~2350 | ~600 |\n| Swimmer | 40.9±19.6 | ~40 | **~121** | ~95 | ~85 |\n| Humanoid | **810.1±126.1** | N | N | N | N |\n| Reacher | **-5.1±0.8** | -8 | ~-115 | **~-5** | N |\n| InvertedPendulum | **1000.0±0.0** | **~1000** | **~1000** | ~910 | N |\n| InvertedDoublePendulum | **8435.2±1073.3** | ~800 | ~200 | ~7000 | N |\n\n\\* details[[4]](#footnote4)[[5]](#footnote5)\n\n#### Hints for TRPO\n\n1. We have tried `step-per-collect` in (80, 1024, 2048, 4096), and `training-num` in (4, 16, 32, 64), and found out 1024\n for `step-per-collect` (same as OpenAI Baselines) and smaller `training-num` (below 16) are good choices. Set\n `training-num` to 4 is actually better but we still use 16 considering the boost of training speed.\n2. Advantage normalization is a standard trick in TRPO, but we found it of minor help, just like in PPO.\n3. Larger `optim-critic-iters` (than 5, as used in OpenAI Baselines) helps in most environments. Smaller lr and lr_decay\n strategy also help a tiny little bit for performance.\n4. `gae-lambda` 0.98 and 0.95 work equally well.\n5. We use GAE returns (GAE advantage + value) as the target of critic network when updating, while people usually tend\n to use reward to go (lambda = 0.) as target. We found that they work equally well although using GAE returns is a\n little bit inaccurate (biased) by math.\n6. Empirically, Swimmer-v3 usually requires larger bootstrap lengths and learning rate. Humanoid-v3 and\n InvertedPendulum-v2, however, are on the opposite.\n7. In contrast, with the statement made in TRPO paper, we found that backtracking in line search is rarely used at least\n in Mujoco settings, which is actually unimportant. This makes TRPO algorithm actually the same as TNPG algorithm (\n described in this [paper](http://proceedings.mlr.press/v48/duan16.html)). This also explains why TNPG and TRPO's\n plotting results look so similar in that paper.\n8. \"recompute advantage\" is helpful in PPO but doesn't help in TRPO.\n\n### NPG\n\n| Environment | Tianshou (1M) |\n|:----------------------:|:----------------:|\n| Ant | **2358.0±517.5** |\n| HalfCheetah | **3485.2±716.6** |\n| Hopper | **1915.2±550.5** |\n| Walker2d | **2503.2±963.3** |\n| Swimmer | **31.5±8.0** |\n| Humanoid | **765.1±91.3** |\n| Reacher | **-4.5±0.5** |\n| InvertedPendulum | **1000.0±0.0** |\n| InvertedDoublePendulum | **9243.2±276.0** |\n\n\\* details[[4]](#footnote4)[[5]](#footnote5)\n\n#### Hints for NPG\n\n1. All shared hyperparameters are exactly the same as TRPO, regarding how similar these two algorithms are.\n2. We found different games in Mujoco may require quite different `actor-step-size`: Reacher/Swimmer are insensitive to\n step-size in range (0.1~1.0), while InvertedDoublePendulum / InvertedPendulum / Humanoid are quite sensitive to step\n size, and even 0.1 is too large. Other games may require `actor-step-size` in range (0.1~0.4), but aren't that\n sensitive in general.\n\n## Others\n\n### HER\n\n| Environment | DDPG without HER | DDPG with HER |\n|:-----------:|:----------------:|:--------------:|\n| FetchReach | -49.9±0.2. | **-17.6±21.7** |\n\n#### Hints for HER\n\n1. The HER technique is proposed for solving task-based environments, so it cannot be compared with non-task-based\n mujoco benchmarks. The environment used in this evaluation is `FetchReach-v3` which requires an\n extra [installation](https://github.com/Farama-Foundation/Gymnasium-Robotics).\n2. Simple hyperparameters optimizations are done for both settings, DDPG with and without HER. However, since _DDPG\n without HER_ failed in every experiment, the best hyperparameters for _DDPG with HER_ are used in the evaluation of\n both settings.\n3. The scores are the mean reward ± 1 standard deviation of 16 seeds. The minimum reward for `FetchReach-v3` is -50\n which we can imply that _DDPG without HER_ performs as good as a random policy. _DDPG with HER_ although has a better\n mean reward, the standard deviation is quite high. This is because in this setting, the agent will either fail\n completely (-50 reward) or successfully learn the task (close to 0 reward). This means that the agent successfully\n learned in about 70% of the 16 seeds.\n\n## Note\n\n[1] Supported environments include HalfCheetah-v3, Hopper-v3, Swimmer-v3, Walker2d-v3, Ant-v3,\nHumanoid-v3, Reacher-v2, InvertedPendulum-v2 and InvertedDoublePendulum-v2. Pusher, Thrower, Striker and HumanoidStandup\nare not supported because they are not commonly seen in literatures.\n\n[2] Pretrained agents, detailed graphs (single agent, single game) and log details can all be\nfound at [Google Drive](https://drive.google.com/drive/folders/1IycImzTmWcyEeD38viea5JHoboC4zmNP?usp=share_link).\n\n[3] We used the latest version of all mujoco environments in gym (0.17.3 with mujoco==2.0.2.13),\nbut it's not often the case with other benchmarks. Please check for details yourself in the original paper. (Different\nversion's outcomes are usually similar, though)\n\n[4] ~ means the number is approximated from the graph because accurate numbers is not provided\nin the paper. N means graphs not provided.\n\n[5] Reward metric: The meaning of the table value is the max average return over 10 trails (\ndifferent seeds) ± a single standard deviation over trails. Each trial is averaged on another 10 test seeds. Only the\nfirst 1M steps data will be considered, if not otherwise stated. The shaded region on the graph also represents a single\nstandard deviation. It is the same as [TD3 evaluation method](https://github.com/sfujim/TD3/issues/34).\n\n[6] In TD3 paper, shaded region represents only half of standard deviation.\n\n[7] Comparing Tianshou's REINFORCE algorithm with SpinningUp's VPG is quite unfair because\nSpinningUp's VPG uses a generative advantage estimator (GAE) which requires a dnn value predictor (critic network),\nwhich makes so called \"VPG\" more like A2C (advantage actor critic) algorithm. Even so, you can see that we are roughly\nat-parity with each other even if tianshou's REINFORCE do not use a critic or GAE.\n" }, { "path": "examples/mujoco/analysis.py", "content": "#!/usr/bin/env python3\n\nimport argparse\nimport re\nfrom collections import defaultdict\nfrom os import PathLike\n\nimport numpy as np\nfrom tabulate import tabulate\nfrom tools import csv2numpy, find_all_files, group_files\n\n\ndef numerical_analysis(root_dir: str | PathLike, xlim: float, norm: bool = False) -> None:\n file_pattern = re.compile(r\".*/test_reward_\\d+seeds.csv$\")\n norm_group_pattern = re.compile(r\"(/|^)\\w+?\\-v(\\d|$)\")\n output_group_pattern = re.compile(r\".*?(?=(/|^)\\w+?\\-v\\d)\")\n csv_files = find_all_files(root_dir, file_pattern)\n norm_group = group_files(csv_files, norm_group_pattern)\n output_group = group_files(csv_files, output_group_pattern)\n # calculate numerical outcome for each csv_file (y/std integration max_y, final_y)\n results = defaultdict(list)\n for f in csv_files:\n result = csv2numpy(f)\n if norm:\n result = np.stack(\n [\n result[\"env_step\"],\n result[\"reward\"] - result[\"reward\"][0],\n result[\"reward:shaded\"],\n ],\n )\n else:\n result = np.stack([result[\"env_step\"], result[\"reward\"], result[\"reward:shaded\"]])\n\n if result[0, -1] < xlim:\n continue\n\n final_rew = np.interp(xlim, result[0], result[1])\n final_rew_std = np.interp(xlim, result[0], result[2])\n result = result[:, result[0] <= xlim]\n\n if len(result) == 0:\n continue\n\n if result[0, -1] < xlim:\n last_line = np.array([xlim, final_rew, final_rew_std]).reshape(3, 1)\n result = np.concatenate([result, last_line], axis=-1)\n\n max_id = np.argmax(result[1])\n results[\"name\"].append(f)\n results[\"final_reward\"].append(result[1, -1])\n results[\"final_reward_std\"].append(result[2, -1])\n results[\"max_reward\"].append(result[1, max_id])\n results[\"max_std\"].append(result[2, max_id])\n results[\"reward_integration\"].append(np.trapz(result[1], x=result[0]))\n results[\"reward_std_integration\"].append(np.trapz(result[2], x=result[0]))\n\n results = {k: np.array(v) for k, v in results.items()}\n print(tabulate(results, headers=\"keys\"))\n\n if norm:\n # calculate normalized numerical outcome for each csv_file group\n for _, fs in norm_group.items():\n mask = np.isin(results[\"name\"], fs)\n for k, v in results.items():\n if k == \"name\":\n continue\n v[mask] = v[mask] / max(v[mask])\n # Add all numerical results for each outcome group\n group_results = defaultdict(list)\n for g, fs in output_group.items():\n group_results[\"name\"].append(g)\n mask = np.isin(results[\"name\"], fs)\n group_results[\"num\"].append(sum(mask))\n for k in results:\n if k == \"name\":\n continue\n group_results[k + \":norm\"].append(results[k][mask].mean())\n # print all outputs for each csv_file and each outcome group\n print()\n print(tabulate(group_results, headers=\"keys\"))\n\n\nif __name__ == \"__main__\":\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"--xlim\",\n type=int,\n default=1000000,\n help=\"x-axis limitation (default: 1000000)\",\n )\n parser.add_argument(\"--root_dir\", type=str)\n parser.add_argument(\n \"--norm\",\n action=\"store_true\",\n help=\"Normalize all results according to environment.\",\n )\n args = parser.parse_args()\n numerical_analysis(args.root_dir, args.xlim, norm=args.norm)\n" }, { "path": "examples/mujoco/fetch_her_ddpg.py", "content": "#!/usr/bin/env python3\n# isort: skip_file\n\nimport argparse\nimport datetime\nimport os\nimport pprint\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\n\nfrom tianshou.algorithm import DDPG\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.ddpg import ContinuousDeterministicPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import (\n Collector,\n CollectStats,\n HERReplayBuffer,\n HERVectorReplayBuffer,\n ReplayBuffer,\n VectorReplayBuffer,\n)\nfrom tianshou.env import ShmemVectorEnv, TruncatedAsTerminated\nfrom tianshou.env.venvs import BaseVectorEnv\nfrom tianshou.exploration import GaussianNoise\nfrom tianshou.highlevel.logger import LoggerFactoryDefault\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils.net.common import Net, get_dict_state_decorator\nfrom tianshou.utils.net.continuous import ContinuousActorDeterministic, ContinuousCritic\nfrom tianshou.utils.space_info import ActionSpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"FetchReach-v2\")\n parser.add_argument(\"--seed\", type=int, default=0)\n parser.add_argument(\"--buffer_size\", type=int, default=100000)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[256, 256])\n parser.add_argument(\"--actor_lr\", type=float, default=1e-3)\n parser.add_argument(\"--critic_lr\", type=float, default=3e-3)\n parser.add_argument(\"--gamma\", type=float, default=0.99)\n parser.add_argument(\"--tau\", type=float, default=0.005)\n parser.add_argument(\"--exploration_noise\", type=float, default=0.1)\n parser.add_argument(\"--start_timesteps\", type=int, default=25000)\n parser.add_argument(\"--epoch\", type=int, default=10)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=5000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=1)\n parser.add_argument(\"--update_per_step\", type=int, default=1)\n parser.add_argument(\"--n_step\", type=int, default=1)\n parser.add_argument(\"--batch_size\", type=int, default=512)\n parser.add_argument(\"--replay_buffer\", type=str, default=\"her\", choices=[\"normal\", \"her\"])\n parser.add_argument(\"--her_horizon\", type=int, default=50)\n parser.add_argument(\"--her_future_k\", type=int, default=8)\n parser.add_argument(\"--num_training_envs\", type=int, default=1)\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n parser.add_argument(\"--resume_path\", type=str, default=None)\n parser.add_argument(\"--resume_id\", type=str, default=None)\n parser.add_argument(\n \"--logger\",\n type=str,\n default=\"tensorboard\",\n choices=[\"tensorboard\", \"wandb\"],\n )\n parser.add_argument(\"--wandb_project\", type=str, default=\"HER-benchmark\")\n parser.add_argument(\n \"--watch\",\n default=False,\n action=\"store_true\",\n help=\"watch the play of pre-trained policy only\",\n )\n return parser.parse_args()\n\n\ndef make_fetch_env(\n task: str,\n num_training_envs: int,\n num_test_envs: int,\n) -> tuple[gym.Env, BaseVectorEnv, BaseVectorEnv]:\n env = TruncatedAsTerminated(gym.make(task))\n training_envs = ShmemVectorEnv(\n [lambda: TruncatedAsTerminated(gym.make(task)) for _ in range(num_training_envs)],\n )\n test_envs = ShmemVectorEnv(\n [lambda: TruncatedAsTerminated(gym.make(task)) for _ in range(num_test_envs)],\n )\n return env, training_envs, test_envs\n\n\ndef test_ddpg(args: argparse.Namespace = get_args()) -> None:\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n args.algo_name = \"ddpg\"\n log_name = os.path.join(args.task, args.algo_name, str(args.seed), now)\n log_path = os.path.join(args.logdir, log_name)\n\n # logger\n logger_factory = LoggerFactoryDefault()\n if args.logger == \"wandb\":\n logger_factory.logger_type = \"wandb\"\n logger_factory.wandb_project = args.wandb_project\n else:\n logger_factory.logger_type = \"tensorboard\"\n\n logger = logger_factory.create_logger(\n log_dir=log_path,\n experiment_name=log_name,\n run_id=args.resume_id,\n config_dict=vars(args),\n )\n\n env, training_envs, test_envs = make_fetch_env(\n args.task, args.num_training_envs, args.num_test_envs\n )\n # The method HER works with goal-based environments\n if not isinstance(env.observation_space, gym.spaces.Dict):\n raise ValueError(\n \"`env.observation_space` must be of type `gym.spaces.Dict`. Make sure you're using a goal-based environment like `FetchReach-v2`.\",\n )\n if not hasattr(env, \"compute_reward\"):\n raise ValueError(\n \"Atrribute `compute_reward` not found in `env`. \"\n \"HER-based algorithms typically require this attribute. Make sure you're using a goal-based environment like `FetchReach-v2`.\",\n )\n args.state_shape = {\n \"observation\": env.observation_space[\"observation\"].shape,\n \"achieved_goal\": env.observation_space[\"achieved_goal\"].shape,\n \"desired_goal\": env.observation_space[\"desired_goal\"].shape,\n }\n action_info = ActionSpaceInfo.from_space(env.action_space)\n args.action_shape = action_info.action_shape\n args.max_action = action_info.max_action\n\n args.exploration_noise = args.exploration_noise * args.max_action\n print(\"Observations shape:\", args.state_shape)\n print(\"Actions shape:\", args.action_shape)\n print(\"Action range:\", action_info.min_action, action_info.max_action)\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n # model\n dict_state_dec, flat_state_shape = get_dict_state_decorator(\n state_shape=args.state_shape,\n keys=[\"observation\", \"achieved_goal\", \"desired_goal\"],\n )\n net_a = dict_state_dec(Net)(\n flat_state_shape,\n hidden_sizes=args.hidden_sizes,\n device=args.device,\n )\n actor = dict_state_dec(ContinuousActorDeterministic)(\n net_a,\n args.action_shape,\n max_action=args.max_action,\n device=args.device,\n ).to(args.device)\n actor_optim = AdamOptimizerFactory(lr=args.actor_lr)\n net_c = dict_state_dec(Net)(\n flat_state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n concat=True,\n device=args.device,\n )\n critic = dict_state_dec(ContinuousCritic)(net_c, device=args.device).to(args.device)\n critic_optim = AdamOptimizerFactory(lr=args.critic_lr)\n policy = ContinuousDeterministicPolicy(\n actor=actor,\n exploration_noise=GaussianNoise(sigma=args.exploration_noise),\n action_space=env.action_space,\n )\n algorithm: DDPG = DDPG(\n policy=policy,\n policy_optim=actor_optim,\n critic=critic,\n critic_optim=critic_optim,\n tau=args.tau,\n gamma=args.gamma,\n n_step_return_horizon=args.n_step,\n )\n\n # load a previous policy\n if args.resume_path:\n algorithm.load_state_dict(torch.load(args.resume_path, map_location=args.device))\n print(\"Loaded agent from: \", args.resume_path)\n\n # collector\n def compute_reward_fn(ag: np.ndarray, g: np.ndarray) -> np.ndarray:\n return env.compute_reward(ag, g, {})\n\n buffer: VectorReplayBuffer | ReplayBuffer | HERReplayBuffer | HERVectorReplayBuffer\n if args.replay_buffer == \"normal\":\n if args.num_training_envs > 1:\n buffer = VectorReplayBuffer(args.buffer_size, len(training_envs))\n else:\n buffer = ReplayBuffer(args.buffer_size)\n else:\n if args.num_training_envs > 1:\n buffer = HERVectorReplayBuffer(\n args.buffer_size,\n len(training_envs),\n compute_reward_fn=compute_reward_fn,\n horizon=args.her_horizon,\n future_k=args.her_future_k,\n )\n else:\n buffer = HERReplayBuffer(\n args.buffer_size,\n compute_reward_fn=compute_reward_fn,\n horizon=args.her_horizon,\n future_k=args.her_future_k,\n )\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buffer, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs)\n training_collector.reset()\n training_collector.collect(n_step=args.start_timesteps, random=True)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n if not args.watch:\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n save_best_fn=save_best_fn,\n logger=logger,\n update_step_num_gradient_steps_per_sample=args.update_per_step,\n test_in_training=False,\n )\n )\n pprint.pprint(result)\n\n # Let's watch its performance!\n test_envs.seed(args.seed)\n test_collector.reset()\n collector_stats = test_collector.collect(n_episode=args.num_test_envs, render=args.render)\n collector_stats.pprint_asdict()\n\n\nif __name__ == \"__main__\":\n test_ddpg()\n" }, { "path": "examples/mujoco/mujoco_a2c.py", "content": "#!/usr/bin/env python3\n\nimport datetime\nimport os\nimport pprint\nfrom typing import Literal\n\nimport numpy as np\nimport torch\nfrom mujoco_env import make_mujoco_env\nfrom sensai.util import logging\nfrom torch import nn\nfrom torch.distributions import Distribution, Independent, Normal\n\nfrom tianshou.algorithm import A2C\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.reinforce import ProbabilisticActorPolicy\nfrom tianshou.algorithm.optim import LRSchedulerFactoryLinear, RMSpropOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, ReplayBuffer, VectorReplayBuffer\nfrom tianshou.highlevel.logger import LoggerFactoryDefault\nfrom tianshou.trainer import OnPolicyTrainerParams\nfrom tianshou.utils.net.common import ActorCritic, Net\nfrom tianshou.utils.net.continuous import ContinuousActorProbabilistic, ContinuousCritic\n\nlog = logging.getLogger(__name__)\n\n\ndef main(\n task: str = \"Ant-v4\",\n persistence_base_dir: str = \"log\",\n seed: int = 0,\n buffer_size: int = 4096,\n hidden_sizes: list | None = None,\n lr: float = 7e-4,\n gamma: float = 0.99,\n epoch: int = 100,\n epoch_num_steps: int = 30000,\n collection_step_num_env_steps: int = 80,\n update_step_num_repetitions: int = 1,\n batch_size: int | None = None,\n num_training_envs: int = 16,\n num_test_envs: int = 10,\n return_scaling: bool = True,\n vf_coef: float = 0.5,\n ent_coef: float = 0.01,\n gae_lambda: float = 0.95,\n action_bound_method: Literal[\"clip\", \"tanh\"] | None = \"clip\",\n lr_decay: bool = True,\n max_grad_norm: float = 0.5,\n render: float = 0.0,\n device: str | None = None,\n resume_path: str | None = None,\n resume_id: str | None = None,\n logger_type: str = \"tensorboard\",\n wandb_project: str = \"mujoco.benchmark\",\n watch: bool = False,\n) -> None:\n # Set defaults for mutable arguments\n if hidden_sizes is None:\n hidden_sizes = [64, 64]\n if device is None:\n device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\n # Get all local variables as config (excluding internal/temporary ones)\n params_log_info = locals()\n log.info(f\"Starting training with config:\\n{params_log_info}\")\n\n env, training_envs, test_envs = make_mujoco_env(\n task,\n seed,\n num_training_envs,\n num_test_envs,\n obs_norm=True,\n )\n state_shape = env.observation_space.shape or env.observation_space.n\n action_shape = env.action_space.shape or env.action_space.n\n max_action = env.action_space.high[0]\n log.info(f\"Observations shape: {state_shape}\")\n log.info(f\"Actions shape: {action_shape}\")\n log.info(f\"Action range: {np.min(env.action_space.low)}, {np.max(env.action_space.high)}\")\n # seed\n np.random.seed(seed)\n torch.manual_seed(seed)\n # model\n net_a = Net(\n state_shape=state_shape,\n hidden_sizes=hidden_sizes,\n activation=nn.Tanh,\n )\n actor = ContinuousActorProbabilistic(\n preprocess_net=net_a,\n action_shape=action_shape,\n unbounded=True,\n ).to(device)\n net_c = Net(\n state_shape=state_shape,\n hidden_sizes=hidden_sizes,\n activation=nn.Tanh,\n )\n critic = ContinuousCritic(preprocess_net=net_c).to(device)\n actor_critic = ActorCritic(actor, critic)\n\n torch.nn.init.constant_(actor.sigma_param, -0.5)\n for m in actor_critic.modules():\n if isinstance(m, torch.nn.Linear):\n # orthogonal initialization\n torch.nn.init.orthogonal_(m.weight, gain=np.sqrt(2))\n torch.nn.init.zeros_(m.bias)\n # do last policy layer scaling, this will make initial actions have (close to)\n # 0 mean and std, and will help boost performances,\n # see https://arxiv.org/abs/2006.05990, Fig.24 for details\n for m in actor.mu.modules():\n if isinstance(m, torch.nn.Linear):\n torch.nn.init.zeros_(m.bias)\n m.weight.data.copy_(0.01 * m.weight.data)\n\n optim = RMSpropOptimizerFactory(\n lr=lr,\n eps=1e-5,\n alpha=0.99,\n )\n\n if lr_decay:\n optim.with_lr_scheduler_factory(\n LRSchedulerFactoryLinear(\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n collection_step_num_env_steps=collection_step_num_env_steps,\n )\n )\n\n def dist(loc_scale: tuple[torch.Tensor, torch.Tensor]) -> Distribution:\n loc, scale = loc_scale\n return Independent(Normal(loc, scale), 1)\n\n policy = ProbabilisticActorPolicy(\n actor=actor,\n dist_fn=dist,\n action_scaling=True,\n action_bound_method=action_bound_method,\n action_space=env.action_space,\n )\n algorithm: A2C = A2C(\n policy=policy,\n critic=critic,\n optim=optim,\n gamma=gamma,\n gae_lambda=gae_lambda,\n max_grad_norm=max_grad_norm,\n vf_coef=vf_coef,\n ent_coef=ent_coef,\n return_scaling=return_scaling,\n )\n\n # load a previous policy\n if resume_path:\n ckpt = torch.load(resume_path, map_location=device)\n algorithm.load_state_dict(ckpt[\"model\"])\n training_envs.set_obs_rms(ckpt[\"obs_rms\"])\n test_envs.set_obs_rms(ckpt[\"obs_rms\"])\n print(\"Loaded agent from: \", resume_path)\n\n # collector\n buffer: VectorReplayBuffer | ReplayBuffer\n if num_training_envs > 1:\n buffer = VectorReplayBuffer(buffer_size, len(training_envs))\n else:\n buffer = ReplayBuffer(buffer_size)\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buffer, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n algo_name = \"a2c\"\n log_name = os.path.join(task, algo_name, str(seed), now)\n log_path = os.path.join(persistence_base_dir, log_name)\n\n # logger\n logger_factory = LoggerFactoryDefault()\n if logger_type == \"wandb\":\n logger_factory.logger_type = \"wandb\"\n logger_factory.wandb_project = wandb_project\n else:\n logger_factory.logger_type = \"tensorboard\"\n\n logger = logger_factory.create_logger(\n log_dir=log_path,\n experiment_name=log_name,\n run_id=resume_id,\n config_dict=params_log_info,\n )\n\n def save_best_fn(policy: Algorithm) -> None:\n state = {\"model\": policy.state_dict(), \"obs_rms\": training_envs.get_obs_rms()}\n torch.save(state, os.path.join(log_path, \"policy.pth\"))\n\n if not watch:\n # train\n result = algorithm.run_training(\n OnPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n update_step_num_repetitions=update_step_num_repetitions,\n test_step_num_episodes=num_test_envs,\n batch_size=batch_size,\n collection_step_num_env_steps=collection_step_num_env_steps,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=False,\n )\n )\n pprint.pprint(result)\n\n # Let's watch its performance!\n test_envs.seed(seed)\n test_collector.reset()\n collector_stats = test_collector.collect(n_episode=num_test_envs, render=render)\n print(collector_stats)\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/mujoco/mujoco_a2c_hl.py", "content": "#!/usr/bin/env python3\n\nimport os\nfrom typing import Literal\n\nimport torch\nfrom sensai.util import logging\n\nfrom examples.mujoco.mujoco_env import MujocoEnvFactory\nfrom tianshou.highlevel.config import OnPolicyTrainingConfig\nfrom tianshou.highlevel.experiment import (\n A2CExperimentBuilder,\n ExperimentConfig,\n)\nfrom tianshou.highlevel.params.algorithm_params import A2CParams\nfrom tianshou.highlevel.params.lr_scheduler import LRSchedulerFactoryFactoryLinear\nfrom tianshou.highlevel.params.optim import OptimizerFactoryFactoryRMSprop\n\n\ndef main(\n task: str = \"Ant-v4\",\n persistence_base_dir: str = \"log\",\n num_experiments: int = 1,\n experiment_launcher: Literal[\"sequential\", \"joblib\"] = \"sequential\",\n max_epochs: int = 100,\n epoch_num_steps: int = 30000,\n) -> None:\n \"\"\"\n Train an agent using A2C on a specified MuJoCo task, potentially running multiple experiments with different seeds\n and evaluating the results using rliable.\n\n :param task: the MuJoCo task to train on.\n :param persistence_base_dir: the base directory for logging and saving experiment data,\n the task name will be appended to it.\n :param num_experiments: the number of experiments to run. The experiments differ exclusively in the seeds.\n :param experiment_launcher: the type of experiment launcher to use, only has an effect if `num_experiments>1`.\n You can use \"joblib\" for parallel execution of whole experiments.\n :param max_epochs: the maximum number of training epochs.\n :param epoch_num_steps: the number of environment steps per epoch.\n \"\"\"\n persistence_base_dir = os.path.abspath(os.path.join(persistence_base_dir, task))\n experiment_config = ExperimentConfig(persistence_base_dir=persistence_base_dir, watch=False)\n\n training_config = OnPolicyTrainingConfig(\n max_epochs=max_epochs,\n epoch_num_steps=epoch_num_steps,\n batch_size=None,\n num_training_envs=16,\n num_test_envs=10,\n buffer_size=4096,\n collection_step_num_env_steps=80,\n update_step_num_repetitions=1,\n )\n\n env_factory = MujocoEnvFactory(task, obs_norm=True)\n\n hidden_sizes = (64, 64)\n experiment_builder = (\n A2CExperimentBuilder(env_factory, experiment_config, training_config)\n .with_a2c_params(\n A2CParams(\n gamma=0.99,\n gae_lambda=0.95,\n action_bound_method=\"clip\",\n return_scaling=True,\n ent_coef=0.01,\n vf_coef=0.5,\n max_grad_norm=0.5,\n optim=OptimizerFactoryFactoryRMSprop(eps=1e-5, alpha=0.99),\n lr=7e-4,\n lr_scheduler=LRSchedulerFactoryFactoryLinear(training_config),\n ),\n )\n .with_actor_factory_default(hidden_sizes, torch.nn.Tanh, continuous_unbounded=True)\n .with_critic_factory_default(hidden_sizes, torch.nn.Tanh)\n )\n\n experiment_builder.build_and_run(num_experiments=num_experiments, launcher=experiment_launcher)\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/mujoco/mujoco_ddpg.py", "content": "#!/usr/bin/env python3\n\nimport datetime\nimport os\nimport pprint\n\nimport numpy as np\nimport torch\nfrom mujoco_env import make_mujoco_env\nfrom sensai.util import logging\n\nfrom tianshou.algorithm import DDPG\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.ddpg import ContinuousDeterministicPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, ReplayBuffer, VectorReplayBuffer\nfrom tianshou.exploration import GaussianNoise\nfrom tianshou.highlevel.logger import LoggerFactoryDefault\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.continuous import ContinuousActorDeterministic, ContinuousCritic\n\nlog = logging.getLogger(__name__)\n\n\ndef main(\n task: str = \"Ant-v4\",\n persistence_base_dir: str = \"log\",\n seed: int = 0,\n hidden_sizes: list | None = None,\n actor_lr: float = 1e-3,\n critic_lr: float = 1e-3,\n gamma: float = 0.99,\n tau: float = 0.005,\n exploration_noise: float = 0.1,\n start_timesteps: int = 25000,\n epoch: int = 50,\n epoch_num_steps: int = 5000,\n buffer_size: int = 1000000,\n collection_step_num_env_steps: int = 1,\n update_per_step: int = 1,\n n_step: int = 1,\n batch_size: int = 256,\n num_training_envs: int = 1,\n num_test_envs: int = 10,\n device: str | None = None,\n resume_path: str | None = None,\n resume_id: str | None = None,\n logger_type: str = \"tensorboard\",\n wandb_project: str = \"mujoco.benchmark\",\n watch: bool = False,\n render: float = 0.0,\n) -> None:\n # Set defaults for mutable arguments\n if hidden_sizes is None:\n hidden_sizes = [256, 256]\n if device is None:\n device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\n # Get all local variables as config\n params_log_info = locals()\n log.info(f\"Starting training with config:\\n{params_log_info}\")\n\n env, training_envs, test_envs = make_mujoco_env(\n task,\n seed,\n num_training_envs,\n num_test_envs,\n obs_norm=False,\n )\n state_shape = env.observation_space.shape or env.observation_space.n\n action_shape = env.action_space.shape or env.action_space.n\n max_action = env.action_space.high[0]\n exploration_noise = exploration_noise * max_action\n log.info(f\"Observations shape: {state_shape}\")\n log.info(f\"Actions shape: {action_shape}\")\n log.info(f\"Action range: {np.min(env.action_space.low)}, {np.max(env.action_space.high)}\")\n # seed\n np.random.seed(seed)\n torch.manual_seed(seed)\n # model\n net_a = Net(state_shape=state_shape, hidden_sizes=hidden_sizes)\n actor = ContinuousActorDeterministic(\n preprocess_net=net_a, action_shape=action_shape, max_action=max_action\n ).to(device)\n actor_optim = AdamOptimizerFactory(lr=actor_lr)\n net_c = Net(\n state_shape=state_shape,\n action_shape=action_shape,\n hidden_sizes=hidden_sizes,\n concat=True,\n )\n critic = ContinuousCritic(preprocess_net=net_c).to(device)\n critic_optim = AdamOptimizerFactory(lr=critic_lr)\n policy = ContinuousDeterministicPolicy(\n actor=actor,\n exploration_noise=GaussianNoise(sigma=exploration_noise),\n action_space=env.action_space,\n )\n algorithm: DDPG = DDPG(\n policy=policy,\n policy_optim=actor_optim,\n critic=critic,\n critic_optim=critic_optim,\n tau=tau,\n gamma=gamma,\n n_step_return_horizon=n_step,\n )\n\n # load a previous policy\n if resume_path:\n algorithm.load_state_dict(torch.load(resume_path, map_location=device))\n log.info(f\"Loaded agent from: {resume_path}\")\n\n # collector\n buffer: VectorReplayBuffer | ReplayBuffer\n if num_training_envs > 1:\n buffer = VectorReplayBuffer(buffer_size, len(training_envs))\n else:\n buffer = ReplayBuffer(buffer_size)\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buffer, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs)\n training_collector.reset()\n training_collector.collect(n_step=start_timesteps, random=True)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n algo_name = \"ddpg\"\n log_name = os.path.join(task, algo_name, str(seed), now)\n log_path = os.path.join(persistence_base_dir, log_name)\n\n # logger\n logger_factory = LoggerFactoryDefault()\n if logger_type == \"wandb\":\n logger_factory.logger_type = \"wandb\"\n logger_factory.wandb_project = wandb_project\n else:\n logger_factory.logger_type = \"tensorboard\"\n\n logger = logger_factory.create_logger(\n log_dir=log_path,\n experiment_name=log_name,\n run_id=resume_id,\n config_dict=params_log_info,\n )\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n if not watch:\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n collection_step_num_env_steps=collection_step_num_env_steps,\n test_step_num_episodes=num_test_envs,\n batch_size=batch_size,\n save_best_fn=save_best_fn,\n logger=logger,\n update_step_num_gradient_steps_per_sample=update_per_step,\n test_in_training=False,\n )\n )\n pprint.pprint(result)\n\n # Let's watch its performance!\n test_envs.seed(seed)\n test_collector.reset()\n collector_stats = test_collector.collect(n_episode=num_test_envs, render=render)\n log.info(f\"Collector stats: {collector_stats}\")\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/mujoco/mujoco_ddpg_hl.py", "content": "#!/usr/bin/env python3\n\nimport os\nfrom typing import Literal\n\nfrom sensai.util import logging\n\nfrom examples.mujoco.mujoco_env import MujocoEnvFactory\nfrom tianshou.highlevel.config import OffPolicyTrainingConfig\nfrom tianshou.highlevel.experiment import (\n DDPGExperimentBuilder,\n ExperimentConfig,\n)\nfrom tianshou.highlevel.params.algorithm_params import DDPGParams\nfrom tianshou.highlevel.params.noise import MaxActionScaledGaussian\n\n\ndef main(\n task: str = \"Ant-v4\",\n persistence_base_dir: str = \"log\",\n num_experiments: int = 1,\n experiment_launcher: Literal[\"sequential\", \"joblib\"] = \"joblib\",\n max_epochs: int = 50,\n epoch_num_steps: int = 5000,\n) -> None:\n \"\"\"\n Train an agent using DDPG on a specified MuJoCo task, potentially running multiple experiments with different seeds\n and evaluating the results using rliable.\n\n :param task: the MuJoCo task to train on.\n :param persistence_base_dir: the base directory for logging and saving experiment data,\n the task name will be appended to it.\n :param num_experiments: the number of experiments to run. The experiments differ exclusively in the seeds.\n :param experiment_launcher: the type of experiment launcher to use, only has an effect if `num_experiments>1`.\n You can use \"joblib\" for parallel execution of whole experiments.\n :param max_epochs: the maximum number of training epochs.\n :param epoch_num_steps: the number of environment steps per epoch.\n \"\"\"\n persistence_base_dir = os.path.abspath(os.path.join(persistence_base_dir, task))\n experiment_config = ExperimentConfig(persistence_base_dir=persistence_base_dir, watch=False)\n\n training_config = OffPolicyTrainingConfig(\n max_epochs=max_epochs,\n epoch_num_steps=epoch_num_steps,\n num_training_envs=1,\n num_test_envs=10,\n buffer_size=1000000,\n batch_size=256,\n collection_step_num_env_steps=1,\n update_step_num_gradient_steps_per_sample=1,\n start_timesteps=25000,\n start_timesteps_random=True,\n )\n\n env_factory = MujocoEnvFactory(task, obs_norm=False)\n\n hidden_sizes = (256, 256)\n experiment_builder = (\n DDPGExperimentBuilder(env_factory, experiment_config, training_config)\n .with_ddpg_params(\n DDPGParams(\n actor_lr=1e-3,\n critic_lr=1e-3,\n gamma=0.99,\n tau=0.005,\n exploration_noise=MaxActionScaledGaussian(0.1),\n n_step_return_horizon=1,\n ),\n )\n .with_actor_factory_default(hidden_sizes)\n .with_critic_factory_default(hidden_sizes)\n )\n\n experiment_builder.build_and_run(num_experiments=num_experiments, launcher=experiment_launcher)\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/mujoco/mujoco_env.py", "content": "import logging\nimport pickle\n\nfrom gymnasium import Env\n\nfrom tianshou.env import BaseVectorEnv, VectorEnvNormObs\nfrom tianshou.highlevel.env import (\n ContinuousEnvironments,\n EnvFactoryRegistered,\n EnvMode,\n EnvPoolFactory,\n VectorEnvType,\n)\nfrom tianshou.highlevel.persistence import Persistence, PersistEvent, RestoreEvent\nfrom tianshou.highlevel.world import World\n\nenvpool_is_available = True\ntry:\n import envpool\nexcept ImportError:\n envpool_is_available = False\n envpool = None\n\nlog = logging.getLogger(__name__)\n\n\ndef make_mujoco_env(\n task: str,\n seed: int,\n num_training_envs: int,\n num_test_envs: int,\n obs_norm: bool,\n) -> tuple[Env, BaseVectorEnv, BaseVectorEnv]:\n \"\"\"Wrapper function for Mujoco env.\n\n If EnvPool is installed, it will automatically switch to EnvPool's Mujoco env.\n\n :return: a tuple of (single env, training envs, test envs).\n \"\"\"\n envs = MujocoEnvFactory(task, obs_norm=obs_norm).create_envs(\n num_training_envs,\n num_test_envs,\n seed=seed,\n )\n return envs.env, envs.training_envs, envs.test_envs\n\n\nclass MujocoEnvObsRmsPersistence(Persistence):\n FILENAME = \"env_obs_rms.pkl\"\n\n def persist(self, event: PersistEvent, world: World) -> None:\n if event != PersistEvent.PERSIST_POLICY:\n return # type: ignore[unreachable] # since PersistEvent has only one member, mypy infers that line is unreachable\n obs_rms = world.envs.training_envs.get_obs_rms()\n path = world.persist_path(self.FILENAME)\n log.info(f\"Saving environment obs_rms value to {path}\")\n with open(path, \"wb\") as f:\n pickle.dump(obs_rms, f)\n\n def restore(self, event: RestoreEvent, world: World) -> None:\n if event != RestoreEvent.RESTORE_POLICY:\n return # type: ignore[unreachable]\n path = world.restore_path(self.FILENAME)\n log.info(f\"Restoring environment obs_rms value from {path}\")\n with open(path, \"rb\") as f:\n obs_rms = pickle.load(f)\n world.envs.training_envs.set_obs_rms(obs_rms)\n world.envs.test_envs.set_obs_rms(obs_rms)\n if world.envs.watch_env is not None:\n world.envs.watch_env.set_obs_rms(obs_rms)\n\n\nclass MujocoEnvFactory(EnvFactoryRegistered):\n def __init__(\n self,\n task: str,\n obs_norm: bool = True,\n venv_type: VectorEnvType = VectorEnvType.SUBPROC_SHARED_MEM_AUTO,\n ) -> None:\n super().__init__(\n task=task,\n venv_type=venv_type,\n envpool_factory=EnvPoolFactory() if envpool_is_available else None,\n )\n self.obs_norm = obs_norm\n\n def create_venv(self, num_envs: int, mode: EnvMode, seed: int | None = None) -> BaseVectorEnv:\n env = super().create_venv(num_envs, mode, seed=seed)\n # obs norm wrapper\n if self.obs_norm:\n env = VectorEnvNormObs(env, update_obs_rms=mode == EnvMode.TRAINING)\n return env\n\n def create_envs(\n self,\n num_training_envs: int,\n num_test_envs: int,\n create_watch_env: bool = False,\n seed: int | None = None,\n ) -> ContinuousEnvironments:\n envs = super().create_envs(num_training_envs, num_test_envs, create_watch_env, seed=seed)\n assert isinstance(envs, ContinuousEnvironments)\n\n if self.obs_norm:\n envs.test_envs.set_obs_rms(envs.training_envs.get_obs_rms())\n if envs.watch_env is not None:\n envs.watch_env.set_obs_rms(envs.training_envs.get_obs_rms())\n envs.set_persistence(MujocoEnvObsRmsPersistence())\n return envs\n" }, { "path": "examples/mujoco/mujoco_npg.py", "content": "#!/usr/bin/env python3\n\nimport datetime\nimport os\nimport pprint\nfrom typing import Literal\n\nimport numpy as np\nimport torch\nfrom mujoco_env import make_mujoco_env\nfrom sensai.util import logging\nfrom torch import nn\nfrom torch.distributions import Distribution, Independent, Normal\n\nfrom tianshou.algorithm import NPG\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.reinforce import ProbabilisticActorPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory, LRSchedulerFactoryLinear\nfrom tianshou.data import Collector, CollectStats, ReplayBuffer, VectorReplayBuffer\nfrom tianshou.highlevel.logger import LoggerFactoryDefault\nfrom tianshou.trainer import OnPolicyTrainerParams\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.continuous import ContinuousActorProbabilistic, ContinuousCritic\n\nlog = logging.getLogger(__name__)\n\n\ndef main(\n task: str = \"Ant-v4\",\n persistence_base_dir: str = \"log\",\n seed: int = 0,\n hidden_sizes: list | None = None,\n lr: float = 1e-3,\n gamma: float = 0.99,\n epoch: int = 100,\n epoch_num_steps: int = 30000,\n collection_step_num_env_steps: int = 1024,\n batch_size: int | None = None,\n buffer_size: int = 4096,\n update_step_num_repetitions: int = 1,\n num_training_envs: int = 16,\n num_test_envs: int = 10,\n return_scaling: bool = True,\n gae_lambda: float = 0.95,\n bound_action_method: Literal[\"clip\", \"tanh\"] | None = \"clip\",\n lr_decay: bool = True,\n render: float = 0.0,\n advantage_normalization: bool = True,\n optim_critic_iters: int = 20,\n device: str | None = None,\n resume_path: str | None = None,\n trust_region_size: float = 0.1,\n resume_id: str | None = None,\n logger_type: str = \"tensorboard\",\n wandb_project: str = \"mujoco.benchmark\",\n watch: bool = False,\n) -> None:\n # Set defaults for mutable arguments\n if hidden_sizes is None:\n hidden_sizes = [64, 64]\n if device is None:\n device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\n # Get all local variables as config\n params_log_info = locals()\n log.info(f\"Starting training with config:\\n{params_log_info}\")\n\n env, training_envs, test_envs = make_mujoco_env(\n task,\n seed,\n num_training_envs,\n num_test_envs,\n obs_norm=True,\n )\n state_shape = env.observation_space.shape or env.observation_space.n\n action_shape = env.action_space.shape or env.action_space.n\n max_action = env.action_space.high[0]\n log.info(f\"Observations shape: {state_shape}\")\n log.info(f\"Actions shape: {action_shape}\")\n log.info(f\"Action range: {np.min(env.action_space.low)}, {np.max(env.action_space.high)}\")\n # seed\n np.random.seed(seed)\n torch.manual_seed(seed)\n # model\n net_a = Net(\n state_shape=state_shape,\n hidden_sizes=hidden_sizes,\n activation=nn.Tanh,\n )\n actor = ContinuousActorProbabilistic(\n preprocess_net=net_a,\n action_shape=action_shape,\n unbounded=True,\n ).to(device)\n net_c = Net(\n state_shape=state_shape,\n hidden_sizes=hidden_sizes,\n activation=nn.Tanh,\n )\n critic = ContinuousCritic(preprocess_net=net_c).to(device)\n torch.nn.init.constant_(actor.sigma_param, -0.5)\n for m in list(actor.modules()) + list(critic.modules()):\n if isinstance(m, torch.nn.Linear):\n # orthogonal initialization\n torch.nn.init.orthogonal_(m.weight, gain=np.sqrt(2))\n torch.nn.init.zeros_(m.bias)\n # do last policy layer scaling, this will make initial actions have (close to)\n # 0 mean and std, and will help boost performances,\n # see https://arxiv.org/abs/2006.05990, Fig.24 for details\n for m in actor.mu.modules():\n if isinstance(m, torch.nn.Linear):\n torch.nn.init.zeros_(m.bias)\n m.weight.data.copy_(0.01 * m.weight.data)\n\n optim = AdamOptimizerFactory(lr=lr)\n if lr_decay:\n optim.with_lr_scheduler_factory(\n LRSchedulerFactoryLinear(\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n collection_step_num_env_steps=collection_step_num_env_steps,\n )\n )\n\n def dist(loc_scale: tuple[torch.Tensor, torch.Tensor]) -> Distribution:\n loc, scale = loc_scale\n return Independent(Normal(loc, scale), 1)\n\n policy = ProbabilisticActorPolicy(\n actor=actor,\n dist_fn=dist,\n action_scaling=True,\n action_bound_method=bound_action_method,\n action_space=env.action_space,\n )\n algorithm: NPG = NPG(\n policy=policy,\n critic=critic,\n optim=optim,\n gamma=gamma,\n gae_lambda=gae_lambda,\n return_scaling=return_scaling,\n advantage_normalization=advantage_normalization,\n optim_critic_iters=optim_critic_iters,\n trust_region_size=trust_region_size,\n )\n\n # load a previous policy\n if resume_path:\n ckpt = torch.load(resume_path, map_location=device)\n algorithm.load_state_dict(ckpt[\"model\"])\n training_envs.set_obs_rms(ckpt[\"obs_rms\"])\n test_envs.set_obs_rms(ckpt[\"obs_rms\"])\n log.info(f\"Loaded agent from: {resume_path}\")\n\n # collector\n buffer: VectorReplayBuffer | ReplayBuffer\n if num_training_envs > 1:\n buffer = VectorReplayBuffer(buffer_size, len(training_envs))\n else:\n buffer = ReplayBuffer(buffer_size)\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buffer, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n algo_name = \"npg\"\n log_name = os.path.join(task, algo_name, str(seed), now)\n log_path = os.path.join(persistence_base_dir, log_name)\n\n # logger\n logger_factory = LoggerFactoryDefault()\n if logger_type == \"wandb\":\n logger_factory.logger_type = \"wandb\"\n logger_factory.wandb_project = wandb_project\n else:\n logger_factory.logger_type = \"tensorboard\"\n\n logger = logger_factory.create_logger(\n log_dir=log_path,\n experiment_name=log_name,\n run_id=resume_id,\n config_dict=params_log_info,\n )\n\n def save_best_fn(policy: Algorithm) -> None:\n state = {\"model\": policy.state_dict(), \"obs_rms\": training_envs.get_obs_rms()}\n torch.save(state, os.path.join(log_path, \"policy.pth\"))\n\n if not watch:\n # train\n result = algorithm.run_training(\n OnPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n update_step_num_repetitions=update_step_num_repetitions,\n test_step_num_episodes=num_test_envs,\n batch_size=batch_size,\n collection_step_num_env_steps=collection_step_num_env_steps,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=False,\n )\n )\n pprint.pprint(result)\n\n # Let's watch its performance!\n test_envs.seed(seed)\n test_collector.reset()\n collector_stats = test_collector.collect(n_episode=num_test_envs, render=render)\n log.info(f\"Collector stats: {collector_stats}\")\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/mujoco/mujoco_npg_hl.py", "content": "#!/usr/bin/env python3\n\nimport os\nfrom typing import Literal\n\nimport torch\nfrom sensai.util import logging\n\nfrom examples.mujoco.mujoco_env import MujocoEnvFactory\nfrom tianshou.highlevel.config import OnPolicyTrainingConfig\nfrom tianshou.highlevel.experiment import (\n ExperimentConfig,\n NPGExperimentBuilder,\n)\nfrom tianshou.highlevel.params.algorithm_params import NPGParams\nfrom tianshou.highlevel.params.lr_scheduler import LRSchedulerFactoryFactoryLinear\n\n\ndef main(\n task: str = \"Ant-v4\",\n persistence_base_dir: str = \"log\",\n num_experiments: int = 1,\n experiment_launcher: Literal[\"sequential\", \"joblib\"] = \"sequential\",\n max_epochs: int = 100,\n epoch_num_steps: int = 30000,\n) -> None:\n \"\"\"\n Train an agent using NPG on a specified MuJoCo task, potentially running multiple experiments with different seeds\n and evaluating the results using rliable.\n\n :param task: the MuJoCo task to train on.\n :param persistence_base_dir: the base directory for logging and saving experiment data,\n the task name will be appended to it.\n :param num_experiments: the number of experiments to run. The experiments differ exclusively in the seeds.\n :param experiment_launcher: the type of experiment launcher to use, only has an effect if `num_experiments>1`.\n You can use \"joblib\" for parallel execution of whole experiments.\n :param max_epochs: the maximum number of training epochs.\n :param epoch_num_steps: the number of environment steps per epoch.\n \"\"\"\n persistence_base_dir = os.path.abspath(os.path.join(persistence_base_dir, task))\n experiment_config = ExperimentConfig(persistence_base_dir=persistence_base_dir, watch=False)\n\n training_config = OnPolicyTrainingConfig(\n max_epochs=max_epochs,\n epoch_num_steps=epoch_num_steps,\n batch_size=None,\n num_training_envs=64,\n num_test_envs=10,\n buffer_size=4096,\n collection_step_num_env_steps=1024,\n update_step_num_repetitions=1,\n )\n\n env_factory = MujocoEnvFactory(task, obs_norm=True)\n\n hidden_sizes = (64, 64)\n experiment_builder = (\n NPGExperimentBuilder(env_factory, experiment_config, training_config)\n .with_npg_params(\n NPGParams(\n gamma=0.99,\n gae_lambda=0.95,\n action_bound_method=\"clip\",\n return_scaling=True,\n advantage_normalization=True,\n optim_critic_iters=20,\n trust_region_size=0.1,\n lr=1e-3,\n lr_scheduler=LRSchedulerFactoryFactoryLinear(training_config),\n ),\n )\n .with_actor_factory_default(hidden_sizes, torch.nn.Tanh, continuous_unbounded=True)\n .with_critic_factory_default(hidden_sizes, torch.nn.Tanh)\n )\n\n experiment_builder.build_and_run(num_experiments=num_experiments, launcher=experiment_launcher)\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/mujoco/mujoco_ppo.py", "content": "#!/usr/bin/env python3\n\nimport datetime\nimport os\nimport pprint\nfrom typing import Literal\n\nimport numpy as np\nimport torch\nfrom mujoco_env import make_mujoco_env\nfrom sensai.util import logging\nfrom torch import nn\nfrom torch.distributions import Distribution, Independent, Normal\n\nfrom tianshou.algorithm import PPO\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.reinforce import ProbabilisticActorPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory, LRSchedulerFactoryLinear\nfrom tianshou.data import Collector, CollectStats, ReplayBuffer, VectorReplayBuffer\nfrom tianshou.highlevel.logger import LoggerFactoryDefault\nfrom tianshou.trainer import OnPolicyTrainerParams\nfrom tianshou.utils.net.common import ActorCritic, Net\nfrom tianshou.utils.net.continuous import ContinuousActorProbabilistic, ContinuousCritic\n\nlog = logging.getLogger(__name__)\n\n\ndef main(\n task: str = \"Ant-v4\",\n persistence_base_dir: str = \"log\",\n seed: int = 0,\n buffer_size: int = 4096,\n hidden_sizes: list | None = None,\n lr: float = 3e-4,\n gamma: float = 0.99,\n epoch: int = 100,\n epoch_num_steps: int = 30000,\n collection_step_num_env_steps: int = 2048,\n update_step_num_repetitions: int = 10,\n batch_size: int = 64,\n num_training_envs: int = 8,\n num_test_envs: int = 10,\n return_scaling: bool = True,\n vf_coef: float = 0.25,\n ent_coef: float = 0.0,\n gae_lambda: float = 0.95,\n bound_action_method: Literal[\"clip\", \"tanh\"] | None = \"clip\",\n lr_decay: bool = True,\n max_grad_norm: float = 0.5,\n eps_clip: float = 0.2,\n dual_clip: float | None = None,\n value_clip: bool = True,\n advantage_normalization: bool = False,\n recompute_adv: bool = True,\n render: float = 0.0,\n device: str | None = None,\n resume_path: str | None = None,\n resume_id: str | None = None,\n logger_type: str = \"tensorboard\",\n wandb_project: str = \"mujoco.benchmark\",\n watch: bool = False,\n) -> None:\n # Set defaults for mutable arguments\n if hidden_sizes is None:\n hidden_sizes = [64, 64]\n if device is None:\n device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\n # Get all local variables as config\n params_log_info = locals()\n log.info(f\"Starting training with config:\\n{params_log_info}\")\n\n env, training_envs, test_envs = make_mujoco_env(\n task,\n seed,\n num_training_envs,\n num_test_envs,\n obs_norm=True,\n )\n state_shape = env.observation_space.shape or env.observation_space.n\n action_shape = env.action_space.shape or env.action_space.n\n max_action = env.action_space.high[0]\n log.info(f\"Observations shape: {state_shape}\")\n log.info(f\"Actions shape: {action_shape}\")\n log.info(f\"Action range: {np.min(env.action_space.low)}, {np.max(env.action_space.high)}\")\n # seed\n np.random.seed(seed)\n torch.manual_seed(seed)\n # model\n net_a = Net(\n state_shape=state_shape,\n hidden_sizes=hidden_sizes,\n activation=nn.Tanh,\n )\n actor = ContinuousActorProbabilistic(\n preprocess_net=net_a,\n action_shape=action_shape,\n unbounded=True,\n ).to(device)\n net_c = Net(\n state_shape=state_shape,\n hidden_sizes=hidden_sizes,\n activation=nn.Tanh,\n )\n critic = ContinuousCritic(preprocess_net=net_c).to(device)\n actor_critic = ActorCritic(actor, critic)\n\n torch.nn.init.constant_(actor.sigma_param, -0.5)\n for m in actor_critic.modules():\n if isinstance(m, torch.nn.Linear):\n # orthogonal initialization\n torch.nn.init.orthogonal_(m.weight, gain=np.sqrt(2))\n torch.nn.init.zeros_(m.bias)\n # do last policy layer scaling, this will make initial actions have (close to)\n # 0 mean and std, and will help boost performances,\n # see https://arxiv.org/abs/2006.05990, Fig.24 for details\n for m in actor.mu.modules():\n if isinstance(m, torch.nn.Linear):\n torch.nn.init.zeros_(m.bias)\n m.weight.data.copy_(0.01 * m.weight.data)\n\n optim = AdamOptimizerFactory(lr=lr)\n\n if lr_decay:\n optim.with_lr_scheduler_factory(\n LRSchedulerFactoryLinear(\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n collection_step_num_env_steps=collection_step_num_env_steps,\n )\n )\n\n def dist(loc_scale: tuple[torch.Tensor, torch.Tensor]) -> Distribution:\n loc, scale = loc_scale\n return Independent(Normal(loc, scale), 1)\n\n policy = ProbabilisticActorPolicy(\n actor=actor,\n dist_fn=dist,\n action_scaling=True,\n action_bound_method=bound_action_method,\n action_space=env.action_space,\n )\n algorithm: PPO = PPO(\n policy=policy,\n critic=critic,\n optim=optim,\n gamma=gamma,\n gae_lambda=gae_lambda,\n max_grad_norm=max_grad_norm,\n vf_coef=vf_coef,\n ent_coef=ent_coef,\n return_scaling=return_scaling,\n eps_clip=eps_clip,\n value_clip=value_clip,\n dual_clip=dual_clip,\n advantage_normalization=advantage_normalization,\n recompute_advantage=recompute_adv,\n )\n\n # load a previous policy\n if resume_path:\n ckpt = torch.load(resume_path, map_location=device)\n algorithm.load_state_dict(ckpt[\"model\"])\n training_envs.set_obs_rms(ckpt[\"obs_rms\"])\n test_envs.set_obs_rms(ckpt[\"obs_rms\"])\n log.info(f\"Loaded agent from: {resume_path}\")\n\n # collector\n buffer: VectorReplayBuffer | ReplayBuffer\n if num_training_envs > 1:\n buffer = VectorReplayBuffer(buffer_size, len(training_envs))\n else:\n buffer = ReplayBuffer(buffer_size)\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buffer, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n algo_name = \"ppo\"\n log_name = os.path.join(task, algo_name, str(seed), now)\n log_path = os.path.join(persistence_base_dir, log_name)\n\n # logger\n logger_factory = LoggerFactoryDefault()\n if logger_type == \"wandb\":\n logger_factory.logger_type = \"wandb\"\n logger_factory.wandb_project = wandb_project\n else:\n logger_factory.logger_type = \"tensorboard\"\n\n logger = logger_factory.create_logger(\n log_dir=log_path,\n experiment_name=log_name,\n run_id=resume_id,\n config_dict=params_log_info,\n )\n\n def save_best_fn(policy: Algorithm) -> None:\n state = {\"model\": policy.state_dict(), \"obs_rms\": training_envs.get_obs_rms()}\n torch.save(state, os.path.join(log_path, \"policy.pth\"))\n\n if not watch:\n # train\n result = algorithm.run_training(\n OnPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n update_step_num_repetitions=update_step_num_repetitions,\n test_step_num_episodes=num_test_envs,\n batch_size=batch_size,\n collection_step_num_env_steps=collection_step_num_env_steps,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=False,\n )\n )\n pprint.pprint(result)\n\n # Let's watch its performance!\n test_envs.seed(seed)\n test_collector.reset()\n collector_stats = test_collector.collect(n_episode=num_test_envs, render=render)\n log.info(f\"Collector stats: {collector_stats}\")\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/mujoco/mujoco_ppo_hl.py", "content": "#!/usr/bin/env python3\n\nimport os\nfrom typing import Literal\n\nimport torch\nfrom sensai.util import logging\n\nfrom examples.mujoco.mujoco_env import MujocoEnvFactory\nfrom tianshou.highlevel.config import OnPolicyTrainingConfig\nfrom tianshou.highlevel.experiment import (\n ExperimentConfig,\n PPOExperimentBuilder,\n)\nfrom tianshou.highlevel.params.algorithm_params import PPOParams\nfrom tianshou.highlevel.params.lr_scheduler import LRSchedulerFactoryFactoryLinear\n\n\ndef main(\n task: str = \"Ant-v4\",\n persistence_base_dir: str = \"log\",\n num_experiments: int = 1,\n experiment_launcher: Literal[\"sequential\", \"joblib\"] = \"sequential\",\n max_epochs: int = 100,\n epoch_num_steps: int = 30000,\n) -> None:\n \"\"\"\n Train an agent using PPO on a specified MuJoCo task, potentially running multiple experiments with different seeds\n and evaluating the results using rliable.\n\n :param task: the MuJoCo task to train on.\n :param persistence_base_dir: the base directory for logging and saving experiment data,\n the task name will be appended to it.\n :param num_experiments: the number of experiments to run. The experiments differ exclusively in the seeds.\n :param experiment_launcher: the type of experiment launcher to use, only has an effect if `num_experiments>1`.\n You can use \"joblib\" for parallel execution of whole experiments.\n :param max_epochs: the maximum number of training epochs.\n :param epoch_num_steps: the number of environment steps per epoch.\n \"\"\"\n persistence_base_dir = os.path.abspath(os.path.join(persistence_base_dir, task))\n experiment_config = ExperimentConfig(persistence_base_dir=persistence_base_dir, watch=False)\n\n training_config = OnPolicyTrainingConfig(\n max_epochs=max_epochs,\n epoch_num_steps=epoch_num_steps,\n batch_size=64,\n num_training_envs=64,\n num_test_envs=10,\n buffer_size=4096,\n collection_step_num_env_steps=2048,\n update_step_num_repetitions=1,\n )\n\n env_factory = MujocoEnvFactory(task, obs_norm=True)\n\n hidden_sizes = (64, 64)\n experiment_builder = (\n PPOExperimentBuilder(env_factory, experiment_config, training_config)\n .with_ppo_params(\n PPOParams(\n gamma=0.99,\n gae_lambda=0.95,\n action_bound_method=\"clip\",\n return_scaling=True,\n ent_coef=0.0,\n vf_coef=0.25,\n max_grad_norm=0.5,\n value_clip=False,\n advantage_normalization=False,\n eps_clip=0.2,\n dual_clip=None,\n recompute_advantage=True,\n lr=3e-4,\n lr_scheduler=LRSchedulerFactoryFactoryLinear(training_config),\n ),\n )\n .with_actor_factory_default(hidden_sizes, torch.nn.Tanh, continuous_unbounded=True)\n .with_critic_factory_default(hidden_sizes, torch.nn.Tanh)\n )\n\n experiment_builder.build_and_run(num_experiments=num_experiments, launcher=experiment_launcher)\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/mujoco/mujoco_redq.py", "content": "#!/usr/bin/env python3\n\nimport datetime\nimport os\nimport pprint\nfrom typing import Literal\n\nimport numpy as np\nimport torch\nfrom mujoco_env import make_mujoco_env\nfrom sensai.util import logging\n\nfrom tianshou.algorithm import REDQ\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.redq import REDQPolicy\nfrom tianshou.algorithm.modelfree.sac import AutoAlpha\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, ReplayBuffer, VectorReplayBuffer\nfrom tianshou.highlevel.logger import LoggerFactoryDefault\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils.net.common import EnsembleLinear, Net\nfrom tianshou.utils.net.continuous import ContinuousActorProbabilistic, ContinuousCritic\n\nlog = logging.getLogger(__name__)\n\n\ndef main(\n task: str = \"Ant-v4\",\n persistence_base_dir: str = \"log\",\n seed: int = 0,\n buffer_size: int = 1000000,\n hidden_sizes: list | None = None,\n ensemble_size: int = 10,\n subset_size: int = 2,\n actor_lr: float = 1e-3,\n critic_lr: float = 1e-3,\n gamma: float = 0.99,\n tau: float = 0.005,\n alpha: float = 0.2,\n auto_alpha: bool = False,\n alpha_lr: float = 3e-4,\n start_timesteps: int = 10000,\n epoch: int = 50,\n epoch_num_steps: int = 5000,\n collection_step_num_env_steps: int = 1,\n update_per_step: int = 20,\n n_step: int = 1,\n batch_size: int = 256,\n target_mode: Literal[\"min\", \"mean\"] = \"min\",\n num_training_envs: int = 1,\n num_test_envs: int = 10,\n render: float = 0.0,\n device: str | None = None,\n resume_path: str | None = None,\n resume_id: str | None = None,\n logger_type: str = \"tensorboard\",\n wandb_project: str = \"mujoco.benchmark\",\n watch: bool = False,\n) -> None:\n # Set defaults for mutable arguments\n if hidden_sizes is None:\n hidden_sizes = [256, 256]\n if device is None:\n device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\n # Get all local variables as config\n params_log_info = locals()\n log.info(f\"Starting training with config:\\n{params_log_info}\")\n\n env, training_envs, test_envs = make_mujoco_env(\n task,\n seed,\n num_training_envs,\n num_test_envs,\n obs_norm=False,\n )\n state_shape = env.observation_space.shape or env.observation_space.n\n action_shape = env.action_space.shape or env.action_space.n\n max_action = env.action_space.high[0]\n log.info(f\"Observations shape: {state_shape}\")\n log.info(f\"Actions shape: {action_shape}\")\n log.info(f\"Action range: {np.min(env.action_space.low)}, {np.max(env.action_space.high)}\")\n # seed\n np.random.seed(seed)\n torch.manual_seed(seed)\n # model\n net_a = Net(state_shape=state_shape, hidden_sizes=hidden_sizes)\n actor = ContinuousActorProbabilistic(\n preprocess_net=net_a,\n action_shape=action_shape,\n unbounded=True,\n conditioned_sigma=True,\n ).to(device)\n actor_optim = AdamOptimizerFactory(lr=actor_lr)\n\n def linear(x: int, y: int) -> EnsembleLinear:\n return EnsembleLinear(ensemble_size, x, y)\n\n net_c = Net(\n state_shape=state_shape,\n action_shape=action_shape,\n hidden_sizes=hidden_sizes,\n concat=True,\n linear_layer=linear,\n )\n critics = ContinuousCritic(\n preprocess_net=net_c,\n linear_layer=linear,\n flatten_input=False,\n ).to(device)\n critics_optim = AdamOptimizerFactory(lr=critic_lr)\n\n if auto_alpha:\n target_entropy = -np.prod(env.action_space.shape)\n log_alpha = 0.0\n alpha_optim = AdamOptimizerFactory(lr=alpha_lr)\n alpha = AutoAlpha(target_entropy, log_alpha, alpha_optim).to(device) # type: ignore\n\n policy = REDQPolicy(\n actor=actor,\n action_space=env.action_space,\n )\n algorithm: REDQ = REDQ(\n policy=policy,\n policy_optim=actor_optim,\n critic=critics,\n critic_optim=critics_optim,\n ensemble_size=ensemble_size,\n subset_size=subset_size,\n tau=tau,\n gamma=gamma,\n alpha=alpha,\n n_step_return_horizon=n_step,\n actor_delay=update_per_step,\n target_mode=target_mode,\n )\n\n # load a previous policy\n if resume_path:\n algorithm.load_state_dict(torch.load(resume_path, map_location=device))\n log.info(f\"Loaded agent from: {resume_path}\")\n\n # collector\n buffer: VectorReplayBuffer | ReplayBuffer\n if num_training_envs > 1:\n buffer = VectorReplayBuffer(buffer_size, len(training_envs))\n else:\n buffer = ReplayBuffer(buffer_size)\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buffer, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs)\n training_collector.reset()\n training_collector.collect(n_step=start_timesteps, random=True)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n algo_name = \"redq\"\n log_name = os.path.join(task, algo_name, str(seed), now)\n log_path = os.path.join(persistence_base_dir, log_name)\n\n # logger\n logger_factory = LoggerFactoryDefault()\n if logger_type == \"wandb\":\n logger_factory.logger_type = \"wandb\"\n logger_factory.wandb_project = wandb_project\n else:\n logger_factory.logger_type = \"tensorboard\"\n\n logger = logger_factory.create_logger(\n log_dir=log_path,\n experiment_name=log_name,\n run_id=resume_id,\n config_dict=params_log_info,\n )\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n if not watch:\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n collection_step_num_env_steps=collection_step_num_env_steps,\n test_step_num_episodes=num_test_envs,\n batch_size=batch_size,\n save_best_fn=save_best_fn,\n logger=logger,\n update_step_num_gradient_steps_per_sample=update_per_step,\n test_in_training=False,\n )\n )\n pprint.pprint(result)\n\n # Let's watch its performance!\n test_envs.seed(seed)\n test_collector.reset()\n collector_stats = test_collector.collect(n_episode=num_test_envs, render=render)\n log.info(f\"Collector stats: {collector_stats}\")\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/mujoco/mujoco_redq_hl.py", "content": "#!/usr/bin/env python3\n\nimport os\nfrom typing import Literal\n\nfrom sensai.util import logging\n\nfrom examples.mujoco.mujoco_env import MujocoEnvFactory\nfrom tianshou.highlevel.config import OffPolicyTrainingConfig\nfrom tianshou.highlevel.experiment import (\n ExperimentConfig,\n REDQExperimentBuilder,\n)\nfrom tianshou.highlevel.params.algorithm_params import REDQParams\n\n\ndef main(\n task: str = \"Ant-v4\",\n persistence_base_dir: str = \"log\",\n num_experiments: int = 1,\n experiment_launcher: Literal[\"sequential\", \"joblib\"] = \"joblib\",\n max_epochs: int = 50,\n epoch_num_steps: int = 5000,\n) -> None:\n \"\"\"\n Train an agent using REDQ on a specified MuJoCo task, potentially running multiple experiments with different seeds\n and evaluating the results using rliable.\n\n :param task: the MuJoCo task to train on.\n :param persistence_base_dir: the base directory for logging and saving experiment data,\n the task name will be appended to it.\n :param num_experiments: the number of experiments to run. The experiments differ exclusively in the seeds.\n :param experiment_launcher: the type of experiment launcher to use, only has an effect if `num_experiments>1`.\n You can use \"joblib\" for parallel execution of whole experiments.\n :param max_epochs: the maximum number of training epochs.\n :param epoch_num_steps: the number of environment steps per epoch.\n \"\"\"\n persistence_base_dir = os.path.abspath(os.path.join(persistence_base_dir, task))\n experiment_config = ExperimentConfig(persistence_base_dir=persistence_base_dir, watch=False)\n\n training_config = OffPolicyTrainingConfig(\n max_epochs=max_epochs,\n epoch_num_steps=epoch_num_steps,\n num_training_envs=1,\n num_test_envs=10,\n buffer_size=1000000,\n batch_size=256,\n collection_step_num_env_steps=1,\n update_step_num_gradient_steps_per_sample=20,\n start_timesteps=10000,\n start_timesteps_random=True,\n )\n\n env_factory = MujocoEnvFactory(task, obs_norm=False)\n\n hidden_sizes = (256, 256)\n experiment_builder = (\n REDQExperimentBuilder(env_factory, experiment_config, training_config)\n .with_redq_params(\n REDQParams(\n actor_lr=1e-3,\n critic_lr=1e-3,\n gamma=0.99,\n tau=0.005,\n alpha=0.2,\n n_step_return_horizon=1,\n target_mode=\"min\",\n subset_size=2,\n ensemble_size=10,\n ),\n )\n .with_actor_factory_default(hidden_sizes)\n .with_critic_ensemble_factory_default(hidden_sizes)\n )\n\n experiment_builder.build_and_run(num_experiments=num_experiments, launcher=experiment_launcher)\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/mujoco/mujoco_reinforce.py", "content": "#!/usr/bin/env python3\n\nimport datetime\nimport os\nimport pprint\nfrom typing import Literal\n\nimport numpy as np\nimport torch\nfrom mujoco_env import make_mujoco_env\nfrom sensai.util import logging\nfrom torch import nn\nfrom torch.distributions import Distribution, Independent, Normal\n\nfrom tianshou.algorithm import Reinforce\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.reinforce import ProbabilisticActorPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory, LRSchedulerFactoryLinear\nfrom tianshou.data import Collector, CollectStats, ReplayBuffer, VectorReplayBuffer\nfrom tianshou.highlevel.logger import LoggerFactoryDefault\nfrom tianshou.trainer import OnPolicyTrainerParams\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.continuous import ContinuousActorProbabilistic\n\nlog = logging.getLogger(__name__)\n\n\ndef main(\n task: str = \"Ant-v4\",\n persistence_base_dir: str = \"log\",\n seed: int = 0,\n hidden_sizes: list | None = None,\n lr: float = 1e-3,\n gamma: float = 0.99,\n epoch: int = 100,\n epoch_num_steps: int = 30000,\n collection_step_num_env_steps: int = 2048,\n update_step_num_repetitions: int = 1,\n batch_size: int | None = None,\n buffer_size: int = 4096,\n num_training_envs: int = 10,\n num_test_envs: int = 10,\n return_scaling: bool = True,\n action_bound_method: Literal[\"clip\", \"tanh\"] | None = \"tanh\",\n lr_decay: bool = True,\n device: str | None = None,\n resume_path: str | None = None,\n resume_id: str | None = None,\n logger_type: str = \"tensorboard\",\n wandb_project: str = \"mujoco.benchmark\",\n watch: bool = False,\n render: float = 0.0,\n) -> None:\n # Set defaults for mutable arguments\n if hidden_sizes is None:\n hidden_sizes = [64, 64]\n if device is None:\n device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\n # Get all local variables as config\n params_log_info = locals()\n log.info(f\"Starting training with config:\\n{params_log_info}\")\n\n env, training_envs, test_envs = make_mujoco_env(\n task,\n seed,\n num_training_envs,\n num_test_envs,\n obs_norm=True,\n )\n state_shape = env.observation_space.shape or env.observation_space.n\n action_shape = env.action_space.shape or env.action_space.n\n max_action = env.action_space.high[0]\n log.info(f\"Observations shape: {state_shape}\")\n log.info(f\"Actions shape: {action_shape}\")\n log.info(f\"Action range: {np.min(env.action_space.low)}, {np.max(env.action_space.high)}\")\n # seed\n np.random.seed(seed)\n torch.manual_seed(seed)\n # model\n net_a = Net(\n state_shape=state_shape,\n hidden_sizes=hidden_sizes,\n activation=nn.Tanh,\n )\n actor = ContinuousActorProbabilistic(\n preprocess_net=net_a,\n action_shape=action_shape,\n unbounded=True,\n ).to(device)\n torch.nn.init.constant_(actor.sigma_param, -0.5)\n for m in actor.modules():\n if isinstance(m, torch.nn.Linear):\n # orthogonal initialization\n torch.nn.init.orthogonal_(m.weight, gain=np.sqrt(2))\n torch.nn.init.zeros_(m.bias)\n # do last policy layer scaling, this will make initial actions have (close to)\n # 0 mean and std, and will help boost performances,\n # see https://arxiv.org/abs/2006.05990, Fig.24 for details\n for m in actor.mu.modules():\n if isinstance(m, torch.nn.Linear):\n torch.nn.init.zeros_(m.bias)\n m.weight.data.copy_(0.01 * m.weight.data)\n\n optim = AdamOptimizerFactory(lr=lr)\n if lr_decay:\n optim.with_lr_scheduler_factory(\n LRSchedulerFactoryLinear(\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n collection_step_num_env_steps=collection_step_num_env_steps,\n )\n )\n\n def dist(loc_scale: tuple[torch.Tensor, torch.Tensor]) -> Distribution:\n loc, scale = loc_scale\n return Independent(Normal(loc, scale), 1)\n\n policy = ProbabilisticActorPolicy(\n actor=actor,\n dist_fn=dist,\n action_space=env.action_space,\n action_scaling=True,\n action_bound_method=action_bound_method,\n )\n algorithm: Reinforce = Reinforce(\n policy=policy,\n optim=optim,\n gamma=gamma,\n return_standardization=return_scaling,\n )\n\n # load a previous policy\n if resume_path:\n ckpt = torch.load(resume_path, map_location=device)\n algorithm.load_state_dict(ckpt[\"model\"])\n training_envs.set_obs_rms(ckpt[\"obs_rms\"])\n test_envs.set_obs_rms(ckpt[\"obs_rms\"])\n log.info(f\"Loaded agent from: {resume_path}\")\n\n # collector\n buffer: VectorReplayBuffer | ReplayBuffer\n if num_training_envs > 1:\n buffer = VectorReplayBuffer(buffer_size, len(training_envs))\n else:\n buffer = ReplayBuffer(buffer_size)\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buffer, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n algo_name = \"reinforce\"\n log_name = os.path.join(task, algo_name, str(seed), now)\n log_path = os.path.join(persistence_base_dir, log_name)\n\n # logger\n logger_factory = LoggerFactoryDefault()\n if logger_type == \"wandb\":\n logger_factory.logger_type = \"wandb\"\n logger_factory.wandb_project = wandb_project\n else:\n logger_factory.logger_type = \"tensorboard\"\n\n logger = logger_factory.create_logger(\n log_dir=log_path,\n experiment_name=log_name,\n run_id=resume_id,\n config_dict=params_log_info,\n )\n\n def save_best_fn(policy: Algorithm) -> None:\n state = {\"model\": policy.state_dict(), \"obs_rms\": training_envs.get_obs_rms()}\n torch.save(state, os.path.join(log_path, \"policy.pth\"))\n\n if not watch:\n # train\n result = algorithm.run_training(\n OnPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n update_step_num_repetitions=update_step_num_repetitions,\n test_step_num_episodes=num_test_envs,\n batch_size=batch_size,\n collection_step_num_env_steps=collection_step_num_env_steps,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=False,\n )\n )\n pprint.pprint(result)\n\n # Let's watch its performance!\n test_envs.seed(seed)\n test_collector.reset()\n collector_stats = test_collector.collect(n_episode=num_test_envs, render=render)\n log.info(f\"Collector stats: {collector_stats}\")\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/mujoco/mujoco_reinforce_hl.py", "content": "#!/usr/bin/env python3\n\nimport os\nfrom typing import Literal\n\nimport torch\nfrom sensai.util import logging\n\nfrom examples.mujoco.mujoco_env import MujocoEnvFactory\nfrom tianshou.highlevel.config import OnPolicyTrainingConfig\nfrom tianshou.highlevel.experiment import (\n ExperimentConfig,\n ReinforceExperimentBuilder,\n)\nfrom tianshou.highlevel.params.algorithm_params import ReinforceParams\nfrom tianshou.highlevel.params.lr_scheduler import LRSchedulerFactoryFactoryLinear\n\n\ndef main(\n task: str = \"Ant-v4\",\n persistence_base_dir: str = \"log\",\n num_experiments: int = 1,\n experiment_launcher: Literal[\"sequential\", \"joblib\"] = \"sequential\",\n max_epochs: int = 100,\n epoch_num_steps: int = 30000,\n) -> None:\n \"\"\"\n Train an agent using REINFORCE on a specified MuJoCo task, potentially running multiple experiments with different seeds\n and evaluating the results using rliable.\n\n :param task: the MuJoCo task to train on.\n :param persistence_base_dir: the base directory for logging and saving experiment data,\n the task name will be appended to it.\n :param num_experiments: the number of experiments to run. The experiments differ exclusively in the seeds.\n :param experiment_launcher: the type of experiment launcher to use, only has an effect if `num_experiments>1`.\n You can use \"joblib\" for parallel execution of whole experiments.\n :param max_epochs: the maximum number of training epochs.\n :param epoch_num_steps: the number of environment steps per epoch.\n \"\"\"\n persistence_base_dir = os.path.abspath(os.path.join(persistence_base_dir, task))\n experiment_config = ExperimentConfig(persistence_base_dir=persistence_base_dir, watch=False)\n\n training_config = OnPolicyTrainingConfig(\n max_epochs=max_epochs,\n epoch_num_steps=epoch_num_steps,\n batch_size=None,\n num_training_envs=64,\n num_test_envs=10,\n buffer_size=4096,\n collection_step_num_env_steps=2048,\n update_step_num_repetitions=1,\n )\n\n env_factory = MujocoEnvFactory(task, obs_norm=True)\n\n hidden_sizes = (64, 64)\n experiment_builder = (\n ReinforceExperimentBuilder(env_factory, experiment_config, training_config)\n .with_reinforce_params(\n ReinforceParams(\n gamma=0.99,\n action_bound_method=\"tanh\",\n return_standardization=True,\n lr=1e-3,\n lr_scheduler=LRSchedulerFactoryFactoryLinear(training_config),\n ),\n )\n .with_actor_factory_default(hidden_sizes, torch.nn.Tanh, continuous_unbounded=True)\n )\n\n experiment_builder.build_and_run(num_experiments=num_experiments, launcher=experiment_launcher)\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/mujoco/mujoco_sac.py", "content": "#!/usr/bin/env python3\n\nimport datetime\nimport os\nimport pprint\n\nimport numpy as np\nimport torch\nfrom mujoco_env import make_mujoco_env\nfrom sensai.util import logging\n\nfrom tianshou.algorithm import SAC\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.sac import AutoAlpha, SACPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, ReplayBuffer, VectorReplayBuffer\nfrom tianshou.highlevel.logger import LoggerFactoryDefault\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.continuous import ContinuousActorProbabilistic, ContinuousCritic\n\nlog = logging.getLogger(__name__)\n\n\ndef main(\n task: str = \"Ant-v4\",\n persistence_base_dir: str = \"log\",\n seed: int = 0,\n buffer_size: int = 1000000,\n hidden_sizes: list | None = None,\n actor_lr: float = 1e-3,\n critic_lr: float = 1e-3,\n gamma: float = 0.99,\n tau: float = 0.005,\n alpha: float = 0.2,\n auto_alpha: bool = False,\n alpha_lr: float = 3e-4,\n start_timesteps: int = 10000,\n epoch: int = 50,\n epoch_num_steps: int = 5000,\n collection_step_num_env_steps: int = 1,\n update_per_step: int = 1,\n n_step: int = 1,\n batch_size: int = 256,\n num_training_envs: int = 1,\n num_test_envs: int = 10,\n render: float = 0.0,\n device: str | None = None,\n resume_path: str | None = None,\n resume_id: str | None = None,\n logger_type: str = \"tensorboard\",\n wandb_project: str = \"mujoco.benchmark\",\n watch: bool = False,\n) -> None:\n # Set defaults for mutable arguments\n if hidden_sizes is None:\n hidden_sizes = [256, 256]\n if device is None:\n device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\n # Get all local variables as config\n params_log_info = locals()\n log.info(f\"Starting training with config:\\n{params_log_info}\")\n\n env, training_envs, test_envs = make_mujoco_env(\n task,\n seed,\n num_training_envs,\n num_test_envs,\n obs_norm=False,\n )\n state_shape = env.observation_space.shape or env.observation_space.n\n action_shape = env.action_space.shape or env.action_space.n\n max_action = env.action_space.high[0]\n log.info(f\"Observations shape: {state_shape}\")\n log.info(f\"Actions shape: {action_shape}\")\n log.info(f\"Action range: {np.min(env.action_space.low)}, {np.max(env.action_space.high)}\")\n # seed\n np.random.seed(seed)\n torch.manual_seed(seed)\n # model\n net_a = Net(state_shape=state_shape, hidden_sizes=hidden_sizes)\n actor = ContinuousActorProbabilistic(\n preprocess_net=net_a,\n action_shape=action_shape,\n unbounded=True,\n conditioned_sigma=True,\n ).to(device)\n actor_optim = AdamOptimizerFactory(lr=actor_lr)\n net_c1 = Net(\n state_shape=state_shape,\n action_shape=action_shape,\n hidden_sizes=hidden_sizes,\n concat=True,\n )\n net_c2 = Net(\n state_shape=state_shape,\n action_shape=action_shape,\n hidden_sizes=hidden_sizes,\n concat=True,\n )\n critic1 = ContinuousCritic(preprocess_net=net_c1).to(device)\n critic1_optim = AdamOptimizerFactory(lr=critic_lr)\n critic2 = ContinuousCritic(preprocess_net=net_c2).to(device)\n critic2_optim = AdamOptimizerFactory(lr=critic_lr)\n\n if auto_alpha:\n target_entropy = -np.prod(env.action_space.shape)\n log_alpha = 0.0\n alpha_optim = AdamOptimizerFactory(lr=alpha_lr)\n alpha = AutoAlpha(target_entropy, log_alpha, alpha_optim).to(device) # type: ignore\n\n policy = SACPolicy(\n actor=actor,\n action_space=env.action_space,\n )\n algorithm: SAC = SAC(\n policy=policy,\n policy_optim=actor_optim,\n critic=critic1,\n critic_optim=critic1_optim,\n critic2=critic2,\n critic2_optim=critic2_optim,\n tau=tau,\n gamma=gamma,\n alpha=alpha,\n n_step_return_horizon=n_step,\n )\n\n # load a previous policy\n if resume_path:\n algorithm.load_state_dict(torch.load(resume_path, map_location=device))\n log.info(f\"Loaded agent from: {resume_path}\")\n\n # collector\n buffer: VectorReplayBuffer | ReplayBuffer\n if num_training_envs > 1:\n buffer = VectorReplayBuffer(buffer_size, len(training_envs))\n else:\n buffer = ReplayBuffer(buffer_size)\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buffer, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs)\n training_collector.reset()\n training_collector.collect(n_step=start_timesteps, random=True)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n algo_name = \"sac\"\n log_name = os.path.join(task, algo_name, str(seed), now)\n log_path = os.path.join(persistence_base_dir, log_name)\n\n # logger\n logger_factory = LoggerFactoryDefault()\n if logger_type == \"wandb\":\n logger_factory.logger_type = \"wandb\"\n logger_factory.wandb_project = wandb_project\n else:\n logger_factory.logger_type = \"tensorboard\"\n\n logger = logger_factory.create_logger(\n log_dir=log_path,\n experiment_name=log_name,\n run_id=resume_id,\n config_dict=params_log_info,\n )\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n if not watch:\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n collection_step_num_env_steps=collection_step_num_env_steps,\n test_step_num_episodes=num_test_envs,\n batch_size=batch_size,\n save_best_fn=save_best_fn,\n logger=logger,\n update_step_num_gradient_steps_per_sample=update_per_step,\n test_in_training=False,\n )\n )\n pprint.pprint(result)\n\n # Let's watch its performance!\n test_envs.seed(seed)\n test_collector.reset()\n collector_stats = test_collector.collect(n_episode=num_test_envs, render=render)\n log.info(collector_stats)\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/mujoco/mujoco_sac_hl.py", "content": "#!/usr/bin/env python3\n\nimport os\nfrom typing import Literal\n\nfrom sensai.util import logging\n\nfrom examples.mujoco.mujoco_env import MujocoEnvFactory\nfrom tianshou.highlevel.config import OffPolicyTrainingConfig\nfrom tianshou.highlevel.experiment import (\n ExperimentConfig,\n SACExperimentBuilder,\n)\nfrom tianshou.highlevel.params.algorithm_params import SACParams\n\n\ndef main(\n task: str = \"Ant-v4\",\n persistence_base_dir: str = \"log\",\n num_experiments: int = 1,\n experiment_launcher: Literal[\"sequential\", \"joblib\"] = \"joblib\",\n max_epochs: int = 50,\n epoch_num_steps: int = 5000,\n) -> None:\n \"\"\"\n Train an agent using SAC on a specified MuJoCo task, potentially running multiple experiments with different seeds\n and evaluating the results using rliable.\n\n :param task: the MuJoCo task to train on.\n :param persistence_base_dir: the base directory for logging and saving experiment data,\n the task name will be appended to it.\n :param num_experiments: the number of experiments to run. The experiments differ exclusively in the seeds.\n :param experiment_launcher: the type of experiment launcher to use, only has an effect if `num_experiments>1`.\n You can use \"joblib\" for parallel execution of whole experiments.\n :param max_epochs: the maximum number of training epochs.\n :param epoch_num_steps: the number of environment steps per epoch.\n \"\"\"\n persistence_base_dir = os.path.abspath(os.path.join(persistence_base_dir, task))\n experiment_config = ExperimentConfig(persistence_base_dir=persistence_base_dir, watch=False)\n\n training_config = OffPolicyTrainingConfig(\n max_epochs=max_epochs,\n epoch_num_steps=epoch_num_steps,\n num_training_envs=1,\n num_test_envs=10,\n buffer_size=1000000,\n batch_size=256,\n collection_step_num_env_steps=1,\n update_step_num_gradient_steps_per_sample=1,\n start_timesteps=10000,\n start_timesteps_random=True,\n )\n\n env_factory = MujocoEnvFactory(task, obs_norm=False)\n\n hidden_sizes = (256, 256)\n experiment_builder = (\n SACExperimentBuilder(env_factory, experiment_config, training_config)\n .with_sac_params(\n SACParams(\n tau=0.005,\n gamma=0.99,\n alpha=0.2,\n n_step_return_horizon=1,\n actor_lr=1e-3,\n critic1_lr=1e-3,\n critic2_lr=1e-3,\n ),\n )\n .with_actor_factory_default(\n hidden_sizes,\n continuous_unbounded=True,\n continuous_conditioned_sigma=True,\n )\n .with_common_critic_factory_default(hidden_sizes)\n )\n\n experiment_builder.build_and_run(num_experiments=num_experiments, launcher=experiment_launcher)\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/mujoco/mujoco_td3.py", "content": "#!/usr/bin/env python3\n\nimport datetime\nimport os\nimport pprint\n\nimport numpy as np\nimport torch\nfrom mujoco_env import make_mujoco_env\nfrom sensai.util import logging\n\nfrom tianshou.algorithm import TD3\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.ddpg import ContinuousDeterministicPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, ReplayBuffer, VectorReplayBuffer\nfrom tianshou.exploration import GaussianNoise\nfrom tianshou.highlevel.logger import LoggerFactoryDefault\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.continuous import ContinuousActorDeterministic, ContinuousCritic\n\nlog = logging.getLogger(__name__)\n\n\ndef main(\n task: str = \"Ant-v4\",\n persistence_base_dir: str = \"log\",\n seed: int = 0,\n hidden_sizes: list | None = None,\n actor_lr: float = 3e-4,\n critic_lr: float = 3e-4,\n gamma: float = 0.99,\n tau: float = 0.005,\n exploration_noise: float = 0.1,\n policy_noise: float = 0.2,\n noise_clip: float = 0.5,\n update_actor_freq: int = 2,\n start_timesteps: int = 25000,\n epoch: int = 50,\n epoch_num_steps: int = 5000,\n collection_step_num_env_steps: int = 1,\n update_per_step: int = 1,\n n_step: int = 1,\n batch_size: int = 256,\n buffer_size: int = 1000000,\n num_training_envs: int = 1,\n num_test_envs: int = 10,\n device: str | None = None,\n resume_path: str | None = None,\n resume_id: str | None = None,\n logger_type: str = \"tensorboard\",\n wandb_project: str = \"mujoco.benchmark\",\n watch: bool = False,\n render: float = 0.0,\n) -> None:\n # Set defaults for mutable arguments\n if hidden_sizes is None:\n hidden_sizes = [256, 256]\n if device is None:\n device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n # Get all local variables as config\n params_log_info = locals()\n log.info(f\"Starting training with config:\\n{params_log_info}\")\n\n env, training_envs, test_envs = make_mujoco_env(\n task,\n seed,\n num_training_envs,\n num_test_envs,\n obs_norm=False,\n )\n state_shape = env.observation_space.shape or env.observation_space.n\n action_shape = env.action_space.shape or env.action_space.n\n max_action = env.action_space.high[0]\n exploration_noise = exploration_noise * max_action\n policy_noise = policy_noise * max_action\n log.info(f\"Observations shape: {state_shape}\")\n log.info(f\"Actions shape: {action_shape}\")\n log.info(f\"Action range: {np.min(env.action_space.low)}, {np.max(env.action_space.high)}\")\n # seed\n np.random.seed(seed)\n torch.manual_seed(seed)\n # model\n net_a = Net(state_shape=state_shape, hidden_sizes=hidden_sizes)\n actor = ContinuousActorDeterministic(\n preprocess_net=net_a, action_shape=action_shape, max_action=max_action\n ).to(device)\n actor_optim = AdamOptimizerFactory(lr=actor_lr)\n net_c1 = Net(\n state_shape=state_shape,\n action_shape=action_shape,\n hidden_sizes=hidden_sizes,\n concat=True,\n )\n net_c2 = Net(\n state_shape=state_shape,\n action_shape=action_shape,\n hidden_sizes=hidden_sizes,\n concat=True,\n )\n critic1 = ContinuousCritic(preprocess_net=net_c1).to(device)\n critic1_optim = AdamOptimizerFactory(lr=critic_lr)\n critic2 = ContinuousCritic(preprocess_net=net_c2).to(device)\n critic2_optim = AdamOptimizerFactory(lr=critic_lr)\n\n policy = ContinuousDeterministicPolicy(\n actor=actor,\n exploration_noise=GaussianNoise(sigma=exploration_noise),\n action_space=env.action_space,\n )\n algorithm: TD3 = TD3(\n policy=policy,\n policy_optim=actor_optim,\n critic=critic1,\n critic_optim=critic1_optim,\n critic2=critic2,\n critic2_optim=critic2_optim,\n tau=tau,\n gamma=gamma,\n policy_noise=policy_noise,\n update_actor_freq=update_actor_freq,\n noise_clip=noise_clip,\n n_step_return_horizon=n_step,\n )\n\n # load a previous policy\n if resume_path:\n log.info(f\"Loaded agent from: {resume_path}\")\n\n # collector\n buffer: VectorReplayBuffer | ReplayBuffer\n if num_training_envs > 1:\n buffer = VectorReplayBuffer(buffer_size, len(training_envs))\n else:\n buffer = ReplayBuffer(buffer_size)\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buffer, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs)\n training_collector.reset()\n training_collector.collect(n_step=start_timesteps, random=True)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n algo_name = \"td3\"\n log_name = os.path.join(task, algo_name, str(seed), now)\n log_path = os.path.join(persistence_base_dir, log_name)\n\n # logger\n logger_factory = LoggerFactoryDefault()\n if logger_type == \"wandb\":\n logger_factory.logger_type = \"wandb\"\n logger_factory.wandb_project = wandb_project\n else:\n logger_factory.logger_type = \"tensorboard\"\n\n logger = logger_factory.create_logger(\n log_dir=log_path,\n experiment_name=log_name,\n run_id=resume_id,\n config_dict=params_log_info,\n )\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n if not watch:\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n collection_step_num_env_steps=collection_step_num_env_steps,\n test_step_num_episodes=num_test_envs,\n batch_size=batch_size,\n save_best_fn=save_best_fn,\n logger=logger,\n update_step_num_gradient_steps_per_sample=update_per_step,\n test_in_training=False,\n )\n )\n pprint.pprint(result)\n\n # Let's watch its performance!\n test_envs.seed(seed)\n test_collector.reset()\n collector_stats = test_collector.collect(n_episode=num_test_envs, render=render)\n log.info(collector_stats)\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/mujoco/mujoco_td3_hl.py", "content": "#!/usr/bin/env python3\n\nimport os\nfrom typing import Literal\n\nimport torch\nfrom sensai.util import logging\n\nfrom examples.mujoco.mujoco_env import MujocoEnvFactory\nfrom tianshou.highlevel.config import OffPolicyTrainingConfig\nfrom tianshou.highlevel.experiment import (\n ExperimentConfig,\n TD3ExperimentBuilder,\n)\nfrom tianshou.highlevel.params.algorithm_params import TD3Params\nfrom tianshou.highlevel.params.env_param import MaxActionScaled\nfrom tianshou.highlevel.params.noise import (\n MaxActionScaledGaussian,\n)\n\n\ndef main(\n task: str = \"Ant-v4\",\n persistence_base_dir: str = \"log\",\n num_experiments: int = 1,\n experiment_launcher: Literal[\"sequential\", \"joblib\"] = \"joblib\",\n max_epochs: int = 50,\n epoch_num_steps: int = 5000,\n) -> None:\n \"\"\"\n Train an agent using TD3 on a specified MuJoCo task, potentially running multiple experiments with different seeds\n and evaluating the results using rliable.\n\n :param task: the MuJoCo task to train on.\n :param persistence_base_dir: the base directory for logging and saving experiment data,\n the task name will be appended to it.\n :param num_experiments: the number of experiments to run. The experiments differ exclusively in the seeds.\n :param experiment_launcher: the type of experiment launcher to use, only has an effect if `num_experiments>1`.\n You can use \"joblib\" for parallel execution of whole experiments.\n :param max_epochs: the maximum number of training epochs.\n :param epoch_num_steps: the number of environment steps per epoch.\n \"\"\"\n persistence_base_dir = os.path.abspath(os.path.join(persistence_base_dir, task))\n experiment_config = ExperimentConfig(persistence_base_dir=persistence_base_dir, watch=False)\n\n training_config = OffPolicyTrainingConfig(\n max_epochs=max_epochs,\n epoch_num_steps=epoch_num_steps,\n num_training_envs=1,\n num_test_envs=10,\n buffer_size=1000000,\n batch_size=256,\n collection_step_num_env_steps=1,\n update_step_num_gradient_steps_per_sample=1,\n start_timesteps=25000,\n start_timesteps_random=True,\n )\n\n env_factory = MujocoEnvFactory(task, obs_norm=False)\n\n hidden_sizes = (256, 256)\n experiment_builder = (\n TD3ExperimentBuilder(env_factory, experiment_config, training_config)\n .with_td3_params(\n TD3Params(\n tau=0.005,\n gamma=0.99,\n n_step_return_horizon=1,\n update_actor_freq=2,\n noise_clip=MaxActionScaled(0.5),\n policy_noise=MaxActionScaled(0.2),\n exploration_noise=MaxActionScaledGaussian(0.1),\n actor_lr=3e-4,\n critic1_lr=3e-4,\n critic2_lr=3e-4,\n ),\n )\n .with_actor_factory_default(hidden_sizes, torch.nn.Tanh)\n .with_common_critic_factory_default(hidden_sizes, torch.nn.Tanh)\n )\n\n experiment_builder.build_and_run(num_experiments=num_experiments, launcher=experiment_launcher)\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/mujoco/mujoco_trpo.py", "content": "#!/usr/bin/env python3\n\nimport datetime\nimport os\nimport pprint\nfrom typing import Literal\n\nimport numpy as np\nimport torch\nfrom mujoco_env import make_mujoco_env\nfrom sensai.util import logging\nfrom torch import nn\nfrom torch.distributions import Distribution, Independent, Normal\n\nfrom tianshou.algorithm import TRPO\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.reinforce import ProbabilisticActorPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory, LRSchedulerFactoryLinear\nfrom tianshou.data import Collector, CollectStats, ReplayBuffer, VectorReplayBuffer\nfrom tianshou.highlevel.logger import LoggerFactoryDefault\nfrom tianshou.trainer import OnPolicyTrainerParams\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.continuous import ContinuousActorProbabilistic, ContinuousCritic\n\nlog = logging.getLogger(__name__)\n\n\ndef main(\n task: str = \"Ant-v4\",\n persistence_base_dir: str = \"log\",\n seed: int = 0,\n hidden_sizes: list | None = None,\n lr: float = 1e-3,\n max_backtracks: int = 10,\n buffer_size: int = 4096,\n update_step_num_repetitions: int = 1,\n gamma: float = 0.99,\n epoch: int = 100,\n epoch_num_steps: int = 30000,\n collection_step_num_env_steps: int = 1024,\n batch_size: int | None = None,\n num_training_envs: int = 16,\n num_test_envs: int = 10,\n return_scaling: bool = True,\n gae_lambda: float = 0.95,\n action_bound_method: Literal[\"clip\", \"tanh\"] | None = \"clip\",\n lr_decay: bool = True,\n render: float = 0.0,\n advantage_normalization: bool = True,\n optim_critic_iters: int = 20,\n max_kl: float = 0.01,\n backtrack_coeff: float = 0.8,\n device: str | None = None,\n resume_path: str | None = None,\n resume_id: str | None = None,\n logger_type: str = \"tensorboard\",\n wandb_project: str = \"mujoco.benchmark\",\n watch: bool = False,\n) -> None:\n # Set defaults for mutable arguments\n if hidden_sizes is None:\n hidden_sizes = [64, 64]\n if device is None:\n device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\n # Get all local variables as config\n params_log_info = locals()\n log.info(f\"Starting training with config:\\n{params_log_info}\")\n\n env, training_envs, test_envs = make_mujoco_env(\n task,\n seed,\n num_training_envs,\n num_test_envs,\n obs_norm=True,\n )\n state_shape = env.observation_space.shape or env.observation_space.n\n action_shape = env.action_space.shape or env.action_space.n\n log.info(f\"Observations shape: {state_shape}\")\n log.info(f\"Actions shape: {action_shape}\")\n log.info(f\"Action range: {np.min(env.action_space.low)}, {np.max(env.action_space.high)}\")\n # seed\n np.random.seed(seed)\n torch.manual_seed(seed)\n # model\n net_a = Net(\n state_shape=state_shape,\n hidden_sizes=hidden_sizes,\n activation=nn.Tanh,\n )\n actor = ContinuousActorProbabilistic(\n preprocess_net=net_a,\n action_shape=action_shape,\n unbounded=True,\n ).to(device)\n net_c = Net(\n state_shape=state_shape,\n hidden_sizes=hidden_sizes,\n activation=nn.Tanh,\n )\n critic = ContinuousCritic(preprocess_net=net_c).to(device)\n torch.nn.init.constant_(actor.sigma_param, -0.5)\n for m in list(actor.modules()) + list(critic.modules()):\n if isinstance(m, torch.nn.Linear):\n # orthogonal initialization\n torch.nn.init.orthogonal_(m.weight, gain=np.sqrt(2))\n torch.nn.init.zeros_(m.bias)\n # do last policy layer scaling, this will make initial actions have (close to)\n # 0 mean and std, and will help boost performances,\n # see https://arxiv.org/abs/2006.05990, Fig.24 for details\n for m in actor.mu.modules():\n if isinstance(m, torch.nn.Linear):\n torch.nn.init.zeros_(m.bias)\n m.weight.data.copy_(0.01 * m.weight.data)\n\n optim = AdamOptimizerFactory(lr=lr)\n if lr_decay:\n optim.with_lr_scheduler_factory(\n LRSchedulerFactoryLinear(\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n collection_step_num_env_steps=collection_step_num_env_steps,\n )\n )\n\n def dist(loc_scale: tuple[torch.Tensor, torch.Tensor]) -> Distribution:\n loc, scale = loc_scale\n return Independent(Normal(loc, scale), 1)\n\n policy = ProbabilisticActorPolicy(\n actor=actor,\n dist_fn=dist,\n action_scaling=True,\n action_bound_method=action_bound_method,\n action_space=env.action_space,\n )\n algorithm: TRPO = TRPO(\n policy=policy,\n critic=critic,\n optim=optim,\n gamma=gamma,\n gae_lambda=gae_lambda,\n return_scaling=return_scaling,\n advantage_normalization=advantage_normalization,\n optim_critic_iters=optim_critic_iters,\n max_kl=max_kl,\n backtrack_coeff=backtrack_coeff,\n max_backtracks=max_backtracks,\n )\n\n # load a previous policy\n if resume_path:\n ckpt = torch.load(resume_path, map_location=device)\n algorithm.load_state_dict(ckpt[\"model\"])\n training_envs.set_obs_rms(ckpt[\"obs_rms\"])\n log.info(f\"Loaded agent from: {resume_path}\")\n\n # collector\n buffer: VectorReplayBuffer | ReplayBuffer\n if num_training_envs > 1:\n buffer = VectorReplayBuffer(buffer_size, len(training_envs))\n else:\n buffer = ReplayBuffer(buffer_size)\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buffer, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n algo_name = \"trpo\"\n log_name = os.path.join(task, algo_name, str(seed), now)\n log_path = os.path.join(persistence_base_dir, log_name)\n\n # logger\n logger_factory = LoggerFactoryDefault()\n if logger_type == \"wandb\":\n logger_factory.logger_type = \"wandb\"\n logger_factory.wandb_project = wandb_project\n else:\n logger_factory.logger_type = \"tensorboard\"\n\n logger = logger_factory.create_logger(\n log_dir=log_path,\n experiment_name=log_name,\n run_id=resume_id,\n config_dict=params_log_info,\n )\n\n def save_best_fn(policy: Algorithm) -> None:\n state = {\"model\": policy.state_dict(), \"obs_rms\": training_envs.get_obs_rms()}\n torch.save(state, os.path.join(log_path, \"policy.pth\"))\n\n if not watch:\n # trainer\n result = algorithm.run_training(\n OnPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=epoch,\n epoch_num_steps=epoch_num_steps,\n update_step_num_repetitions=update_step_num_repetitions,\n test_step_num_episodes=num_test_envs,\n batch_size=batch_size,\n collection_step_num_env_steps=collection_step_num_env_steps,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=False,\n )\n )\n pprint.pprint(result)\n\n # Let's watch its performance!\n test_envs.seed(seed)\n test_collector.reset()\n collector_stats = test_collector.collect(n_episode=num_test_envs, render=render)\n log.info(collector_stats)\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/mujoco/mujoco_trpo_hl.py", "content": "#!/usr/bin/env python3\n\nimport os\nfrom typing import Literal\n\nimport torch\nfrom sensai.util import logging\n\nfrom examples.mujoco.mujoco_env import MujocoEnvFactory\nfrom tianshou.highlevel.config import OnPolicyTrainingConfig\nfrom tianshou.highlevel.experiment import (\n ExperimentConfig,\n TRPOExperimentBuilder,\n)\nfrom tianshou.highlevel.params.algorithm_params import TRPOParams\nfrom tianshou.highlevel.params.lr_scheduler import LRSchedulerFactoryFactoryLinear\n\n\ndef main(\n task: str = \"Ant-v4\",\n persistence_base_dir: str = \"log\",\n num_experiments: int = 1,\n experiment_launcher: Literal[\"sequential\", \"joblib\"] = \"sequential\",\n max_epochs: int = 100,\n epoch_num_steps: int = 30000,\n) -> None:\n \"\"\"\n Train an agent using TRPO on a specified MuJoCo task, potentially running multiple experiments with different seeds\n and evaluating the results using rliable.\n\n :param task: the MuJoCo task to train on.\n :param persistence_base_dir: the base directory for logging and saving experiment data,\n the task name will be appended to it.\n :param num_experiments: the number of experiments to run. The experiments differ exclusively in the seeds.\n :param experiment_launcher: the type of experiment launcher to use, only has an effect if `num_experiments>1`.\n You can use \"joblib\" for parallel execution of whole experiments.\n :param max_epochs: the maximum number of training epochs.\n :param epoch_num_steps: the number of environment steps per epoch.\n \"\"\"\n persistence_base_dir = os.path.abspath(os.path.join(persistence_base_dir, task))\n experiment_config = ExperimentConfig(persistence_base_dir=persistence_base_dir, watch=False)\n\n training_config = OnPolicyTrainingConfig(\n max_epochs=max_epochs,\n epoch_num_steps=epoch_num_steps,\n batch_size=None,\n num_training_envs=16,\n num_test_envs=10,\n buffer_size=4096,\n collection_step_num_env_steps=1024,\n update_step_num_repetitions=1,\n )\n\n env_factory = MujocoEnvFactory(task, obs_norm=True)\n\n hidden_sizes = (64, 64)\n experiment_builder = (\n TRPOExperimentBuilder(env_factory, experiment_config, training_config)\n .with_trpo_params(\n TRPOParams(\n gamma=0.99,\n gae_lambda=0.95,\n action_bound_method=\"clip\",\n return_standardization=True,\n advantage_normalization=True,\n optim_critic_iters=20,\n max_kl=0.01,\n backtrack_coeff=0.8,\n max_backtracks=10,\n lr=1e-3,\n lr_scheduler=LRSchedulerFactoryFactoryLinear(training_config),\n ),\n )\n .with_actor_factory_default(hidden_sizes, torch.nn.Tanh, continuous_unbounded=True)\n .with_critic_factory_default(hidden_sizes, torch.nn.Tanh)\n )\n\n experiment_builder.build_and_run(num_experiments=num_experiments, launcher=experiment_launcher)\n\n\nif __name__ == \"__main__\":\n result = logging.run_cli(main, level=logging.INFO)\n" }, { "path": "examples/mujoco/plotter.py", "content": "#!/usr/bin/env python3\n\nimport argparse\nimport os\nimport re\nfrom typing import Any, Literal\n\nimport matplotlib.pyplot as plt\nimport matplotlib.ticker as mticker\nimport numpy as np\nfrom tools import csv2numpy, find_all_files, group_files\n\n\ndef smooth(\n y: np.ndarray,\n radius: int,\n mode: Literal[\"two_sided\", \"causal\"] = \"two_sided\",\n valid_only: bool = False,\n) -> np.ndarray:\n \"\"\"Smooth signal y, where radius is determines the size of the window.\n\n mode='twosided':\n average over the window [max(index - radius, 0), min(index + radius, len(y)-1)]\n mode='causal':\n average over the window [max(index - radius, 0), index]\n valid_only: put nan in entries where the full-sized window is not available\n \"\"\"\n if len(y) < 2 * radius + 1:\n return np.ones_like(y) * y.mean()\n if mode == \"two_sided\":\n convkernel = np.ones(2 * radius + 1)\n out = np.convolve(y, convkernel, mode=\"same\") / np.convolve(\n np.ones_like(y),\n convkernel,\n mode=\"same\",\n )\n if valid_only:\n out[:radius] = out[-radius:] = np.nan\n elif mode == \"causal\":\n convkernel = np.ones(radius)\n out = np.convolve(y, convkernel, mode=\"full\") / np.convolve(\n np.ones_like(y),\n convkernel,\n mode=\"full\",\n )\n out = out[: -radius + 1]\n if valid_only:\n out[:radius] = np.nan\n return out\n\n\nCOLORS = [\n # deepmind style\n \"#0072B2\",\n \"#009E73\",\n \"#D55E00\",\n \"#CC79A7\",\n # '#F0E442',\n \"#d73027\", # RED\n # built-in color\n \"blue\",\n \"red\",\n \"pink\",\n \"cyan\",\n \"magenta\",\n \"yellow\",\n \"black\",\n \"purple\",\n \"brown\",\n \"orange\",\n \"teal\",\n \"lightblue\",\n \"lime\",\n \"lavender\",\n \"turquoise\",\n \"darkgreen\",\n \"tan\",\n \"salmon\",\n \"gold\",\n \"darkred\",\n \"darkblue\",\n \"green\",\n # personal color\n \"#313695\", # DARK BLUE\n \"#74add1\", # LIGHT BLUE\n \"#f46d43\", # ORANGE\n \"#4daf4a\", # GREEN\n \"#984ea3\", # PURPLE\n \"#f781bf\", # PINK\n \"#ffc832\", # YELLOW\n \"#000000\", # BLACK\n]\n\n\ndef plot_ax(\n ax: plt.Axes,\n file_lists: list[str],\n legend_pattern: str = \".*\",\n xlabel: str | None = None,\n ylabel: str | None = None,\n title: str = \"\",\n xlim: float | None = None,\n xkey: str = \"env_step\",\n ykey: str = \"reward\",\n smooth_radius: int = 0,\n shaded_std: bool = True,\n legend_outside: bool = False,\n) -> None:\n def legend_fn(x: str) -> str:\n # return os.path.split(os.path.join(\n # args.root_dir, x))[0].replace('/', '_') + \" (10)\"\n match = re.search(legend_pattern, x)\n assert match is not None # for mypy\n return match.group(0)\n\n legneds = map(legend_fn, file_lists)\n # sort filelist according to legends\n file_lists = [f for _, f in sorted(zip(legneds, file_lists, strict=True))]\n legneds = list(map(legend_fn, file_lists))\n\n for index, csv_file in enumerate(file_lists):\n csv_dict = csv2numpy(csv_file)\n x, y = csv_dict[xkey], csv_dict[ykey]\n y = smooth(y, radius=smooth_radius)\n color = COLORS[index % len(COLORS)]\n ax.plot(x, y, color=color)\n if shaded_std and ykey + \":shaded\" in csv_dict:\n y_shaded = smooth(csv_dict[ykey + \":shaded\"], radius=smooth_radius)\n ax.fill_between(x, y - y_shaded, y + y_shaded, color=color, alpha=0.2)\n\n ax.legend(\n legneds,\n loc=2 if legend_outside else None,\n bbox_to_anchor=(1, 1) if legend_outside else None,\n )\n ax.xaxis.set_major_formatter(mticker.EngFormatter())\n if xlim is not None:\n ax.set_xlim(xmin=0, xmax=xlim)\n # add title\n ax.set_title(title)\n # add labels\n if xlabel is not None:\n ax.set_xlabel(xlabel)\n if ylabel is not None:\n ax.set_ylabel(ylabel)\n\n\ndef plot_figure(\n file_lists: list[str],\n group_pattern: str | None = None,\n fig_length: int = 6,\n fig_width: int = 6,\n sharex: bool = False,\n sharey: bool = False,\n title: str = \"\",\n **kwargs: Any,\n) -> None:\n if not group_pattern:\n fig, ax = plt.subplots(figsize=(fig_length, fig_width))\n plot_ax(ax, file_lists, title=title, **kwargs)\n else:\n res = group_files(file_lists, group_pattern)\n row_n = int(np.ceil(len(res) / 3))\n col_n = min(len(res), 3)\n fig, axes = plt.subplots(\n row_n,\n col_n,\n sharex=sharex,\n sharey=sharey,\n figsize=(fig_length * col_n, fig_width * row_n),\n squeeze=False,\n )\n axes = axes.flatten()\n for i, (k, v) in enumerate(res.items()):\n plot_ax(axes[i], v, title=k, **kwargs)\n if title: # add title\n fig.suptitle(title, fontsize=20)\n\n\nif __name__ == \"__main__\":\n parser = argparse.ArgumentParser(description=\"plotter\")\n parser.add_argument(\n \"--fig_length\",\n type=int,\n default=6,\n help=\"matplotlib figure length (default: 6)\",\n )\n parser.add_argument(\n \"--fig_width\",\n type=int,\n default=6,\n help=\"matplotlib figure width (default: 6)\",\n )\n parser.add_argument(\n \"--style\",\n default=\"seaborn\",\n help=\"matplotlib figure style (default: seaborn)\",\n )\n parser.add_argument(\"--title\", default=None, help=\"matplotlib figure title (default: None)\")\n parser.add_argument(\n \"--xkey\",\n default=\"env_step\",\n help=\"x-axis key in csv file (default: env_step)\",\n )\n parser.add_argument(\"--ykey\", default=\"rew\", help=\"y-axis key in csv file (default: rew)\")\n parser.add_argument(\n \"--smooth\",\n type=int,\n default=0,\n help=\"smooth radius of y axis (default: 0)\",\n )\n parser.add_argument(\"--xlabel\", default=\"Timesteps\", help=\"matplotlib figure xlabel\")\n parser.add_argument(\"--ylabel\", default=\"Episode Reward\", help=\"matplotlib figure ylabel\")\n parser.add_argument(\n \"--shaded_std\",\n action=\"store_true\",\n help=\"shaded region corresponding to standard deviation of the group\",\n )\n parser.add_argument(\n \"--sharex\",\n action=\"store_true\",\n help=\"whether to share x axis within multiple sub-figures\",\n )\n parser.add_argument(\n \"--sharey\",\n action=\"store_true\",\n help=\"whether to share y axis within multiple sub-figures\",\n )\n parser.add_argument(\n \"--legend_outside\",\n action=\"store_true\",\n help=\"place the legend outside of the figure\",\n )\n parser.add_argument(\"--xlim\", type=int, default=None, help=\"x-axis limitation (default: None)\")\n parser.add_argument(\"--root_dir\", default=\"./\", help=\"root dir (default: ./)\")\n parser.add_argument(\n \"--file_pattern\",\n type=str,\n default=r\".*/test_rew_\\d+seeds.csv$\",\n help=\"regular expression to determine whether or not to include target csv \"\n \"file, default to including all test_rew_{num}seeds.csv file under rootdir\",\n )\n parser.add_argument(\n \"--group_pattern\",\n type=str,\n default=r\"(/|^)\\w*?\\-v(\\d|$)\",\n help=\"regular expression to group files in sub-figure, default to grouping \"\n 'according to env_name dir, \"\" means no grouping',\n )\n parser.add_argument(\n \"--legend_pattern\",\n type=str,\n default=r\".*\",\n help=\"regular expression to extract legend from csv file path, default to \"\n \"using file path as legend name.\",\n )\n parser.add_argument(\"--show\", action=\"store_true\", help=\"show figure\")\n parser.add_argument(\"--output_path\", type=str, help=\"figure save path\", default=\"./figure.png\")\n parser.add_argument(\"--dpi\", type=int, default=200, help=\"figure dpi (default: 200)\")\n args = parser.parse_args()\n file_lists = find_all_files(args.root_dir, re.compile(args.file_pattern))\n file_lists = [os.path.relpath(f, args.root_dir) for f in file_lists]\n if args.style:\n plt.style.use(args.style)\n os.chdir(args.root_dir)\n plot_figure(\n file_lists,\n group_pattern=args.group_pattern,\n legend_pattern=args.legend_pattern,\n fig_length=args.fig_length,\n fig_width=args.fig_width,\n title=args.title,\n xlabel=args.xlabel,\n ylabel=args.ylabel,\n xkey=args.xkey,\n ykey=args.ykey,\n xlim=args.xlim,\n sharex=args.sharex,\n sharey=args.sharey,\n smooth_radius=args.smooth,\n shaded_std=args.shaded_std,\n legend_outside=args.legend_outside,\n )\n if args.output_path:\n plt.savefig(args.output_path, dpi=args.dpi, bbox_inches=\"tight\")\n if args.show:\n plt.show()\n" }, { "path": "examples/mujoco/tools.py", "content": "#!/usr/bin/env python3\n\nimport argparse\nimport csv\nimport os\nimport re\nfrom collections import defaultdict\nfrom os import PathLike\nfrom re import Pattern\nfrom typing import Any\n\nimport numpy as np\nimport tqdm\nfrom tensorboard.backend.event_processing import event_accumulator\n\n\ndef find_all_files(root_dir: str | PathLike[str], pattern: str | Pattern[str]) -> list:\n \"\"\"Find all files under root_dir according to relative pattern.\"\"\"\n file_list = []\n for dirname, _, files in os.walk(root_dir):\n for f in files:\n absolute_path = os.path.join(dirname, f)\n if re.match(pattern, absolute_path):\n file_list.append(absolute_path)\n return file_list\n\n\ndef group_files(file_list: list[str], pattern: str | Pattern[str]) -> dict[str, list]:\n res = defaultdict(list)\n for f in file_list:\n match = re.search(pattern, f)\n key = match.group() if match else \"\"\n res[key].append(f)\n return res\n\n\ndef csv2numpy(csv_file: str) -> dict[Any, np.ndarray]:\n csv_dict = defaultdict(list)\n with open(csv_file) as f:\n for row in csv.DictReader(f):\n for k, v in row.items():\n csv_dict[k].append(eval(v))\n return {k: np.array(v) for k, v in csv_dict.items()}\n\n\ndef convert_tfevents_to_csv(\n root_dir: str | PathLike[str],\n refresh: bool = False,\n) -> dict[str, list]:\n \"\"\"Recursively convert test/reward from all tfevent file under root_dir to csv.\n\n This function assumes that there is at most one tfevents file in each directory\n and will add suffix to that directory.\n\n :param bool refresh: re-create csv file under any condition.\n \"\"\"\n tfevent_files = find_all_files(root_dir, re.compile(r\"^.*tfevents.*$\"))\n print(f\"Converting {len(tfevent_files)} tfevents files under {root_dir} ...\")\n result = {}\n with tqdm.tqdm(tfevent_files) as t:\n for tfevent_file in t:\n t.set_postfix(file=tfevent_file)\n output_file = os.path.join(os.path.split(tfevent_file)[0], \"test_reward.csv\")\n if os.path.exists(output_file) and not refresh:\n with open(output_file) as f:\n content = list(csv.reader(f))\n if content[0] == [\"env_step\", \"reward\", \"time\"]:\n for i in range(1, len(content)):\n content[i] = list(map(eval, content[i]))\n result[output_file] = content\n continue\n ea = event_accumulator.EventAccumulator(tfevent_file)\n ea.Reload()\n initial_time = ea._first_event_timestamp\n content = [[\"env_step\", \"reward\", \"time\"]]\n for test_reward in ea.scalars.Items(\"test/reward\"):\n content.append(\n [\n round(test_reward.step, 4),\n round(test_reward.value, 4),\n round(test_reward.wall_time - initial_time, 4),\n ],\n )\n with open(output_file, \"w\") as f:\n csv.writer(f).writerows(content)\n result[output_file] = content\n return result\n\n\ndef merge_csv(\n csv_files: dict[str, list],\n root_dir: str | PathLike[str],\n remove_zero: bool = False,\n) -> None:\n \"\"\"Merge result in csv_files into a single csv file.\"\"\"\n assert len(csv_files) > 0\n if remove_zero:\n for v in csv_files.values():\n if v[1][0] == 0:\n v.pop(1)\n sorted_keys = sorted(csv_files.keys())\n sorted_values = [csv_files[k][1:] for k in sorted_keys]\n content = [\n [\n \"env_step\",\n \"reward\",\n \"reward:shaded\",\n *[\"reward:\" + os.path.relpath(f, root_dir) for f in sorted_keys],\n ],\n ]\n for rows in zip(*sorted_values, strict=True):\n array = np.array(rows)\n assert len(set(array[:, 0])) == 1, (set(array[:, 0]), array[:, 0])\n line = [rows[0][0], round(array[:, 1].mean(), 4), round(array[:, 1].std(), 4)]\n line += array[:, 1].tolist()\n content.append(line)\n output_path = os.path.join(root_dir, f\"test_reward_{len(csv_files)}seeds.csv\")\n print(f\"Output merged csv file to {output_path} with {len(content[1:])} lines.\")\n with open(output_path, \"w\") as f:\n csv.writer(f).writerows(content)\n\n\nif __name__ == \"__main__\":\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"--refresh\",\n action=\"store_true\",\n help=\"Re-generate all csv files instead of using existing one.\",\n )\n parser.add_argument(\n \"--remove_zero\",\n action=\"store_true\",\n help=\"Remove the data point of env_step == 0.\",\n )\n parser.add_argument(\"--root_dir\", type=str)\n args = parser.parse_args()\n\n csv_files = convert_tfevents_to_csv(args.root_dir, args.refresh)\n merge_csv(csv_files, args.root_dir, args.remove_zero)\n" }, { "path": "examples/offline/README.md", "content": "# Offline\n\nIn offline reinforcement learning setting, the agent learns a policy from a fixed dataset which is collected once with\nany policy. And the agent does not interact with environment anymore.\n\n## Continuous control\n\nOnce the dataset is collected, it will not be changed during training. We\nuse [d4rl](https://github.com/rail-berkeley/d4rl) datasets to train offline agent for continuous control. You can refer\nto [d4rl](https://github.com/rail-berkeley/d4rl) to see how to use d4rl datasets.\n\nWe provide implementation of BCQ and CQL algorithm for continuous control.\n\n### Train\n\nTianshou provides an `offline_trainer` for offline reinforcement learning. You can parse d4rl datasets into a\n`ReplayBuffer` , and set it as the parameter `buffer` of `offline_trainer`. `d4rl_bcq.py` is an example of offline RL\nusing the d4rl dataset.\n\n## Results\n\n### IL (Imitation Learning, aka, Behavior Cloning)\n\n| Environment | Dataset | IL | Parameters |\n|----------------|-----------------------|----------|-------------------------------------------------------------------------------------|\n| HalfCheetah-v2 | halfcheetah-expert-v2 | 11355.31 | `python3 d4rl_il.py --task HalfCheetah-v2 --expert-data-task halfcheetah-expert-v2` |\n| HalfCheetah-v2 | halfcheetah-medium-v2 | 5098.16 | `python3 d4rl_il.py --task HalfCheetah-v2 --expert-data-task halfcheetah-medium-v2` |\n\n### BCQ\n\n| Environment | Dataset | BCQ | Parameters |\n|----------------|-----------------------|----------|--------------------------------------------------------------------------------------|\n| HalfCheetah-v2 | halfcheetah-expert-v2 | 11509.95 | `python3 d4rl_bcq.py --task HalfCheetah-v2 --expert-data-task halfcheetah-expert-v2` |\n| HalfCheetah-v2 | halfcheetah-medium-v2 | 5147.43 | `python3 d4rl_bcq.py --task HalfCheetah-v2 --expert-data-task halfcheetah-medium-v2` |\n\n### CQL\n\n| Environment | Dataset | CQL | Parameters |\n|----------------|-----------------------|---------|--------------------------------------------------------------------------------------|\n| HalfCheetah-v2 | halfcheetah-expert-v2 | 2864.37 | `python3 d4rl_cql.py --task HalfCheetah-v2 --expert-data-task halfcheetah-expert-v2` |\n| HalfCheetah-v2 | halfcheetah-medium-v2 | 6505.41 | `python3 d4rl_cql.py --task HalfCheetah-v2 --expert-data-task halfcheetah-medium-v2` |\n\n### TD3+BC\n\n| Environment | Dataset | CQL | Parameters |\n|----------------|-----------------------|----------|-----------------------------------------------------------------------------------------|\n| HalfCheetah-v2 | halfcheetah-expert-v2 | 11788.25 | `python3 d4rl_td3_bc.py --task HalfCheetah-v2 --expert-data-task halfcheetah-expert-v2` |\n| HalfCheetah-v2 | halfcheetah-medium-v2 | 5741.13 | `python3 d4rl_td3_bc.py --task HalfCheetah-v2 --expert-data-task halfcheetah-medium-v2` |\n\n#### Observation normalization\n\nFollowing the original paper, we use observation normalization by default. You can turn it off by setting\n`--norm-obs 0`. The difference are small but consistent.\n\n| Dataset | w/ norm-obs | w/o norm-obs |\n|:---------------------|:------------|:-------------|\n| halfcheeta-medium-v2 | 5741.13 | 5724.41 |\n| halfcheeta-expert-v2 | 11788.25 | 11665.77 |\n| walker2d-medium-v2 | 4051.76 | 3985.59 |\n| walker2d-expert-v2 | 5068.15 | 5027.75 |\n\n## Discrete control\n\nFor discrete control, we currently use ad hoc Atari data generated from a trained QRDQN agent.\n\n### Gather Data\n\nTo running CQL algorithm on Atari, you need to do the following things:\n\n- Train an expert, by using the command listed in the QRDQN section of Atari examples:\n `python3 atari_qrdqn.py --task {your_task}`\n- Generate buffer with noise:\n `python3 atari_qrdqn.py --task {your_task} --watch --resume-path log/{your_task}/qrdqn/policy.pth --eps-test 0.2 --buffer-size 1000000 --save-buffer-name expert.hdf5` (\n note that 1M Atari buffer cannot be saved as `.pkl` format because it is too large and will cause error);\n- Train offline model: `python3 atari_{bcq,cql,crr}.py --task {your_task} --load-buffer-name expert.hdf5`.\n\n### IL\n\nWe test our IL implementation on two example tasks (different from author's version, we use v4 instead of v0; one epoch\nmeans 10k gradient step):\n\n| Task | Online QRDQN | Behavioral | IL | parameters |\n|------------------------|--------------|------------|-----------------------------------|--------------------------------------------------------------------------------------------------------------------------------|\n| PongNoFrameskip-v4 | 20.5 | 6.8 | 20.0 (epoch 5) | `python3 atari_il.py --task PongNoFrameskip-v4 --load-buffer-name log/PongNoFrameskip-v4/qrdqn/expert.hdf5 --epoch 5` |\n| BreakoutNoFrameskip-v4 | 394.3 | 46.9 | 121.9 (epoch 12, could be higher) | `python3 atari_il.py --task BreakoutNoFrameskip-v4 --load-buffer-name log/BreakoutNoFrameskip-v4/qrdqn/expert.hdf5 --epoch 12` |\n\n### BCQ\n\nWe test our BCQ implementation on two example tasks (different from author's version, we use v4 instead of v0; one epoch\nmeans 10k gradient step):\n\n| Task | Online QRDQN | Behavioral | BCQ | parameters |\n|------------------------|--------------|------------|----------------------------------|---------------------------------------------------------------------------------------------------------------------------------|\n| PongNoFrameskip-v4 | 20.5 | 6.8 | 20.1 (epoch 5) | `python3 atari_bcq.py --task PongNoFrameskip-v4 --load-buffer-name log/PongNoFrameskip-v4/qrdqn/expert.hdf5 --epoch 5` |\n| BreakoutNoFrameskip-v4 | 394.3 | 46.9 | 64.6 (epoch 12, could be higher) | `python3 atari_bcq.py --task BreakoutNoFrameskip-v4 --load-buffer-name log/BreakoutNoFrameskip-v4/qrdqn/expert.hdf5 --epoch 12` |\n\n### CQL\n\nWe test our CQL implementation on two example tasks (different from author's version, we use v4 instead of v0; one epoch\nmeans 10k gradient step):\n\n| Task | Online QRDQN | Behavioral | CQL | parameters |\n|------------------------|--------------|------------|------------------|---------------------------------------------------------------------------------------------------------------------------------------------------|\n| PongNoFrameskip-v4 | 20.5 | 6.8 | 20.4 (epoch 5) | `python3 atari_cql.py --task PongNoFrameskip-v4 --load-buffer-name log/PongNoFrameskip-v4/qrdqn/expert.hdf5 --epoch 5` |\n| BreakoutNoFrameskip-v4 | 394.3 | 46.9 | 129.4 (epoch 12) | `python3 atari_cql.py --task BreakoutNoFrameskip-v4 --load-buffer-name log/BreakoutNoFrameskip-v4/qrdqn/expert.hdf5 --epoch 12 --min-q-weight 50` |\n\nWe reduce the size of the offline data to 10% and 1% of the above and get:\n\nBuffer size 100000:\n\n| Task | Online QRDQN | Behavioral | CQL | parameters |\n|------------------------|--------------|------------|-----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| PongNoFrameskip-v4 | 20.5 | 5.8 | 21 (epoch 5) | `python3 atari_cql.py --task PongNoFrameskip-v4 --load-buffer-name log/PongNoFrameskip-v4/qrdqn/expert.size_1e5.hdf5 --epoch 5` |\n| BreakoutNoFrameskip-v4 | 394.3 | 41.4 | 40.8 (epoch 12) | `python3 atari_cql.py --task BreakoutNoFrameskip-v4 --load-buffer-name log/BreakoutNoFrameskip-v4/qrdqn/expert.size_1e5.hdf5 --epoch 12 --min-q-weight 20` |\n\nBuffer size 10000:\n\n| Task | Online QRDQN | Behavioral | CQL | parameters |\n|------------------------|--------------|------------|-----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| PongNoFrameskip-v4 | 20.5 | nan | 1.8 (epoch 5) | `python3 atari_cql.py --task PongNoFrameskip-v4 --load-buffer-name log/PongNoFrameskip-v4/qrdqn/expert.size_1e4.hdf5 --epoch 5 --min-q-weight 1` |\n| BreakoutNoFrameskip-v4 | 394.3 | 31.7 | 22.5 (epoch 12) | `python3 atari_cql.py --task BreakoutNoFrameskip-v4 --load-buffer-name log/BreakoutNoFrameskip-v4/qrdqn/expert.size_1e4.hdf5 --epoch 12 --min-q-weight 10` |\n\n### CRR\n\nWe test our CRR implementation on two example tasks (different from author's version, we use v4 instead of v0; one epoch\nmeans 10k gradient step):\n\n| Task | Online QRDQN | Behavioral | CRR | CRR w/ CQL | parameters |\n|------------------------|--------------|------------|-----------------|-----------------|---------------------------------------------------------------------------------------------------------------------------------------------------|\n| PongNoFrameskip-v4 | 20.5 | 6.8 | -21 (epoch 5) | 17.7 (epoch 5) | `python3 atari_crr.py --task PongNoFrameskip-v4 --load-buffer-name log/PongNoFrameskip-v4/qrdqn/expert.hdf5 --epoch 5` |\n| BreakoutNoFrameskip-v4 | 394.3 | 46.9 | 23.3 (epoch 12) | 76.9 (epoch 12) | `python3 atari_crr.py --task BreakoutNoFrameskip-v4 --load-buffer-name log/BreakoutNoFrameskip-v4/qrdqn/expert.hdf5 --epoch 12 --min-q-weight 50` |\n\nNote that CRR itself does not work well in Atari tasks but adding CQL loss/regularizer helps.\n\n### RL Unplugged Data\n\nWe provide a script to convert the Atari datasets\nof [RL Unplugged](https://github.com/deepmind/deepmind-research/tree/master/rl_unplugged) to Tianshou ReplayBuffer.\n\nFor example, the following command will download the first shard of the first run of Breakout game to\n`~/.rl_unplugged/datasets/Breakout/run_1-00001-of-00100` then convert it to a `tianshou.data.ReplayBuffer` and save it\nto `~/.rl_unplugged/buffers/Breakout/run_1-00001-of-00100.hdf5` (use `--dataset-dir` and `--buffer-dir` to change the\ndefault directories):\n\n```bash\npython3 convert_rl_unplugged_atari.py --task Breakout --run-id 1 --shard-id 1\n```\n\nThen you can use it to train an agent by:\n\n```bash\npython3 atari_bcq.py --task BreakoutNoFrameskip-v4 --load-buffer-name ~/.rl_unplugged/datasets/Breakout/run_1-00001-of-00100.hdf5 --buffer-from-rl-unplugged --epoch 12\n```\n\nNote:\n\n- Each shard contains about 500k transitions.\n- This conversion script depends on Tensorflow.\n- It takes about 1 hour to process one shard on my machine. YMMV.\n" }, { "path": "examples/offline/atari_bcq.py", "content": "#!/usr/bin/env python3\n\nimport argparse\nimport datetime\nimport os\nimport pickle\nimport pprint\nimport sys\n\nimport numpy as np\nimport torch\nfrom gymnasium.spaces import Discrete\n\nfrom examples.offline.utils import load_buffer\nfrom tianshou.algorithm import DiscreteBCQ\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.imitation.discrete_bcq import DiscreteBCQPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env.atari.atari_network import DQNet\nfrom tianshou.env.atari.atari_wrapper import make_atari_env\nfrom tianshou.highlevel.logger import LoggerFactoryDefault\nfrom tianshou.trainer import OfflineTrainerParams\nfrom tianshou.utils.net.discrete import DiscreteActor\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"PongNoFrameskip-v4\")\n parser.add_argument(\"--seed\", type=int, default=1626)\n parser.add_argument(\"--eps_test\", type=float, default=0.001)\n parser.add_argument(\"--lr\", type=float, default=6.25e-5)\n parser.add_argument(\"--gamma\", type=float, default=0.99)\n parser.add_argument(\"--n_step\", type=int, default=1)\n parser.add_argument(\"--target_update_freq\", type=int, default=8000)\n parser.add_argument(\"--unlikely_action_threshold\", type=float, default=0.3)\n parser.add_argument(\"--imitation_logits_penalty\", type=float, default=0.01)\n parser.add_argument(\"--epoch\", type=int, default=100)\n parser.add_argument(\"--update_per_epoch\", type=int, default=10000)\n parser.add_argument(\"--batch_size\", type=int, default=32)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[512])\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\n parser.add_argument(\"--frames_stack\", type=int, default=4)\n parser.add_argument(\"--scale_obs\", type=int, default=0)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\"--resume_path\", type=str, default=None)\n parser.add_argument(\"--resume_id\", type=str, default=None)\n parser.add_argument(\n \"--logger\",\n type=str,\n default=\"tensorboard\",\n choices=[\"tensorboard\", \"wandb\"],\n )\n parser.add_argument(\"--wandb_project\", type=str, default=\"offline_atari.benchmark\")\n parser.add_argument(\n \"--watch\",\n default=False,\n action=\"store_true\",\n help=\"watch the play of pre-trained policy only\",\n )\n parser.add_argument(\"--log_interval\", type=int, default=100)\n parser.add_argument(\n \"--load_buffer_name\",\n type=str,\n default=\"./expert_DQN_PongNoFrameskip-v4.hdf5\",\n )\n parser.add_argument(\"--buffer_from_rl_unplugged\", action=\"store_true\", default=False)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n return parser.parse_known_args()[0]\n\n\ndef main(args: argparse.Namespace = get_args()) -> None:\n # envs\n env, _, test_envs = make_atari_env(\n args.task,\n args.seed,\n 1,\n args.num_test_envs,\n scale=args.scale_obs,\n frame_stack=args.frames_stack,\n )\n assert isinstance(env.action_space, Discrete)\n args.state_shape = env.observation_space.shape\n args.action_shape = int(env.action_space.n)\n # should be N_FRAMES x H x W\n print(\"Observations shape:\", args.state_shape)\n print(\"Actions shape:\", args.action_shape)\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n # model\n assert args.state_shape is not None\n assert len(args.state_shape) == 3\n c, h, w = args.state_shape\n feature_net = DQNet(\n c=c,\n h=h,\n w=w,\n action_shape=args.action_shape,\n features_only=True,\n ).to(args.device)\n policy_net = DiscreteActor(\n preprocess_net=feature_net,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n softmax_output=False,\n ).to(args.device)\n imitation_net = DiscreteActor(\n preprocess_net=feature_net,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n softmax_output=False,\n ).to(args.device)\n optim = AdamOptimizerFactory(lr=args.lr)\n # define policy and algorithm\n policy = DiscreteBCQPolicy(\n model=policy_net,\n imitator=imitation_net,\n action_space=env.action_space,\n unlikely_action_threshold=args.unlikely_action_threshold,\n eps_inference=args.eps_test,\n )\n algorithm: DiscreteBCQ = DiscreteBCQ(\n policy=policy,\n optim=optim,\n gamma=args.gamma,\n n_step_return_horizon=args.n_step,\n target_update_freq=args.target_update_freq,\n imitation_logits_penalty=args.imitation_logits_penalty,\n )\n # load a previous policy\n if args.resume_path:\n algorithm.load_state_dict(torch.load(args.resume_path, map_location=args.device))\n print(\"Loaded agent from: \", args.resume_path)\n # buffer\n if args.buffer_from_rl_unplugged:\n buffer = load_buffer(args.load_buffer_name)\n else:\n assert os.path.exists(\n args.load_buffer_name,\n ), \"Please run atari_dqn.py first to get expert's data buffer.\"\n if args.load_buffer_name.endswith(\".pkl\"):\n with open(args.load_buffer_name, \"rb\") as f:\n buffer = pickle.load(f)\n elif args.load_buffer_name.endswith(\".hdf5\"):\n buffer = VectorReplayBuffer.load_hdf5(args.load_buffer_name)\n else:\n print(f\"Unknown buffer format: {args.load_buffer_name}\")\n sys.exit(0)\n print(\"Replay buffer size:\", len(buffer), flush=True)\n\n # collector\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n args.algo_name = \"bcq\"\n log_name = os.path.join(args.task, args.algo_name, str(args.seed), now)\n log_path = os.path.join(args.logdir, log_name)\n\n # logger\n logger_factory = LoggerFactoryDefault()\n if args.logger == \"wandb\":\n logger_factory.logger_type = \"wandb\"\n logger_factory.wandb_project = args.wandb_project\n else:\n logger_factory.logger_type = \"tensorboard\"\n\n logger = logger_factory.create_logger(\n log_dir=log_path,\n experiment_name=log_name,\n run_id=args.resume_id,\n config_dict=vars(args),\n )\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return False\n\n # watch agent's performance\n def watch() -> None:\n print(\"Setup test envs ...\")\n policy.set_eps(args.eps_test)\n test_envs.seed(args.seed)\n print(\"Testing agent ...\")\n test_collector.reset()\n result = test_collector.collect(n_episode=args.num_test_envs, render=args.render)\n result.pprint_asdict()\n\n if args.watch:\n watch()\n sys.exit(0)\n\n result = algorithm.run_training(\n OfflineTrainerParams(\n buffer=buffer,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.update_per_epoch,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n )\n )\n\n pprint.pprint(result)\n watch()\n\n\nif __name__ == \"__main__\":\n main(get_args())\n" }, { "path": "examples/offline/atari_cql.py", "content": "#!/usr/bin/env python3\n\nimport argparse\nimport datetime\nimport os\nimport pickle\nimport pprint\nimport sys\nfrom collections.abc import Sequence\n\nimport numpy as np\nimport torch\nfrom gymnasium.spaces import Discrete\n\nfrom examples.offline.utils import load_buffer\nfrom tianshou.algorithm import DiscreteCQL\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.qrdqn import QRDQNPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env.atari.atari_network import QRDQNet\nfrom tianshou.env.atari.atari_wrapper import make_atari_env\nfrom tianshou.highlevel.logger import LoggerFactoryDefault\nfrom tianshou.trainer import OfflineTrainerParams\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"PongNoFrameskip-v4\")\n parser.add_argument(\"--seed\", type=int, default=1626)\n parser.add_argument(\"--lr\", type=float, default=0.0001)\n parser.add_argument(\"--gamma\", type=float, default=0.99)\n parser.add_argument(\"--num_quantiles\", type=int, default=200)\n parser.add_argument(\"--n_step\", type=int, default=1)\n parser.add_argument(\"--target_update_freq\", type=int, default=500)\n parser.add_argument(\"--min_q_weight\", type=float, default=10.0)\n parser.add_argument(\"--epoch\", type=int, default=100)\n parser.add_argument(\"--update_per_epoch\", type=int, default=10000)\n parser.add_argument(\"--batch_size\", type=int, default=32)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[512])\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\n parser.add_argument(\"--frames_stack\", type=int, default=4)\n parser.add_argument(\"--scale_obs\", type=int, default=0)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\"--resume_path\", type=str, default=None)\n parser.add_argument(\"--resume_id\", type=str, default=None)\n parser.add_argument(\n \"--logger\",\n type=str,\n default=\"tensorboard\",\n choices=[\"tensorboard\", \"wandb\"],\n )\n parser.add_argument(\"--wandb_project\", type=str, default=\"offline_atari.benchmark\")\n parser.add_argument(\n \"--watch\",\n default=False,\n action=\"store_true\",\n help=\"watch the play of pre-trained policy only\",\n )\n parser.add_argument(\"--log_interval\", type=int, default=100)\n parser.add_argument(\n \"--load_buffer_name\",\n type=str,\n default=\"./expert_DQN_PongNoFrameskip-v4.hdf5\",\n )\n parser.add_argument(\"--buffer_from_rl_unplugged\", action=\"store_true\", default=False)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n return parser.parse_known_args()[0]\n\n\ndef main(args: argparse.Namespace = get_args()) -> None:\n # envs\n env, _, test_envs = make_atari_env(\n args.task,\n args.seed,\n 1,\n args.num_test_envs,\n scale=args.scale_obs,\n frame_stack=args.frames_stack,\n )\n assert isinstance(env.action_space, Discrete)\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n assert isinstance(args.state_shape, Sequence)\n assert len(args.state_shape) == 3, \"state shape must have only 3 dimensions.\"\n c, h, w = args.state_shape\n # should be N_FRAMES x H x W\n print(\"Observations shape:\", args.state_shape)\n print(\"Actions shape:\", args.action_shape)\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n # model\n net = QRDQNet(\n c=c,\n h=h,\n w=w,\n action_shape=args.action_shape,\n num_quantiles=args.num_quantiles,\n )\n optim = AdamOptimizerFactory(lr=args.lr)\n # define policy\n policy = QRDQNPolicy(\n model=net,\n action_space=env.action_space,\n )\n algorithm: DiscreteCQL = DiscreteCQL(\n policy=policy,\n optim=optim,\n gamma=args.gamma,\n num_quantiles=args.num_quantiles,\n n_step_return_horizon=args.n_step,\n target_update_freq=args.target_update_freq,\n min_q_weight=args.min_q_weight,\n ).to(args.device)\n # load a previous policy\n if args.resume_path:\n algorithm.load_state_dict(torch.load(args.resume_path, map_location=args.device))\n print(\"Loaded agent from: \", args.resume_path)\n # buffer\n if args.buffer_from_rl_unplugged:\n buffer = load_buffer(args.load_buffer_name)\n else:\n assert os.path.exists(\n args.load_buffer_name,\n ), \"Please run atari_dqn.py first to get expert's data buffer.\"\n if args.load_buffer_name.endswith(\".pkl\"):\n with open(args.load_buffer_name, \"rb\") as f:\n buffer = pickle.load(f)\n elif args.load_buffer_name.endswith(\".hdf5\"):\n buffer = VectorReplayBuffer.load_hdf5(args.load_buffer_name)\n else:\n print(f\"Unknown buffer format: {args.load_buffer_name}\")\n sys.exit(0)\n print(\"Replay buffer size:\", len(buffer), flush=True)\n\n # collector\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n args.algo_name = \"cql\"\n log_name = os.path.join(args.task, args.algo_name, str(args.seed), now)\n log_path = os.path.join(args.logdir, log_name)\n\n # logger\n logger_factory = LoggerFactoryDefault()\n if args.logger == \"wandb\":\n logger_factory.logger_type = \"wandb\"\n logger_factory.wandb_project = args.wandb_project\n else:\n logger_factory.logger_type = \"tensorboard\"\n\n logger = logger_factory.create_logger(\n log_dir=log_path,\n experiment_name=log_name,\n run_id=args.resume_id,\n config_dict=vars(args),\n )\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return False\n\n # watch agent's performance\n def watch() -> None:\n print(\"Setup test envs ...\")\n test_envs.seed(args.seed)\n print(\"Testing agent ...\")\n test_collector.reset()\n result = test_collector.collect(n_episode=args.num_test_envs, render=args.render)\n result.pprint_asdict()\n\n if args.watch:\n watch()\n sys.exit(0)\n\n result = algorithm.run_training(\n OfflineTrainerParams(\n buffer=buffer,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.update_per_epoch,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n )\n )\n\n pprint.pprint(result)\n watch()\n\n\nif __name__ == \"__main__\":\n main(get_args())\n" }, { "path": "examples/offline/atari_crr.py", "content": "#!/usr/bin/env python3\n\nimport argparse\nimport datetime\nimport os\nimport pickle\nimport pprint\nimport sys\n\nimport numpy as np\nimport torch\nfrom gymnasium.spaces import Discrete\n\nfrom examples.offline.utils import load_buffer\nfrom tianshou.algorithm import DiscreteCRR\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.reinforce import DiscreteActorPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env.atari.atari_network import DQNet\nfrom tianshou.env.atari.atari_wrapper import make_atari_env\nfrom tianshou.highlevel.logger import LoggerFactoryDefault\nfrom tianshou.trainer import OfflineTrainerParams\nfrom tianshou.utils.net.discrete import DiscreteActor, DiscreteCritic\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"PongNoFrameskip-v4\")\n parser.add_argument(\"--seed\", type=int, default=1626)\n parser.add_argument(\"--lr\", type=float, default=0.0001)\n parser.add_argument(\"--gamma\", type=float, default=0.99)\n parser.add_argument(\"--policy_improvement_mode\", type=str, default=\"exp\")\n parser.add_argument(\"--ratio_upper_bound\", type=float, default=20.0)\n parser.add_argument(\"--beta\", type=float, default=1.0)\n parser.add_argument(\"--min_q_weight\", type=float, default=10.0)\n parser.add_argument(\"--target_update_freq\", type=int, default=500)\n parser.add_argument(\"--epoch\", type=int, default=100)\n parser.add_argument(\"--update_per_epoch\", type=int, default=10000)\n parser.add_argument(\"--batch_size\", type=int, default=32)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[512])\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\n parser.add_argument(\"--frames_stack\", type=int, default=4)\n parser.add_argument(\"--scale_obs\", type=int, default=0)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\"--resume_path\", type=str, default=None)\n parser.add_argument(\"--resume_id\", type=str, default=None)\n parser.add_argument(\n \"--logger\",\n type=str,\n default=\"tensorboard\",\n choices=[\"tensorboard\", \"wandb\"],\n )\n parser.add_argument(\"--wandb_project\", type=str, default=\"offline_atari.benchmark\")\n parser.add_argument(\n \"--watch\",\n default=False,\n action=\"store_true\",\n help=\"watch the play of pre-trained policy only\",\n )\n parser.add_argument(\"--log_interval\", type=int, default=100)\n parser.add_argument(\n \"--load_buffer_name\",\n type=str,\n default=\"./expert_DQN_PongNoFrameskip-v4.hdf5\",\n )\n parser.add_argument(\"--buffer_from_rl_unplugged\", action=\"store_true\", default=False)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n return parser.parse_known_args()[0]\n\n\ndef main(args: argparse.Namespace = get_args()) -> None:\n # envs\n env, _, test_envs = make_atari_env(\n args.task,\n args.seed,\n 1,\n args.num_test_envs,\n scale=args.scale_obs,\n frame_stack=args.frames_stack,\n )\n assert isinstance(env.action_space, Discrete)\n space_info = SpaceInfo.from_env(env)\n args.state_shape = env.observation_space.shape\n args.action_shape = space_info.action_info.action_shape\n # should be N_FRAMES x H x W\n print(\"Observations shape:\", args.state_shape)\n print(\"Actions shape:\", args.action_shape)\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n # model\n assert args.state_shape is not None\n assert len(args.state_shape) == 3\n c, h, w = args.state_shape\n feature_net = DQNet(\n c=c,\n h=h,\n w=w,\n action_shape=args.action_shape,\n features_only=True,\n ).to(args.device)\n actor = DiscreteActor(\n preprocess_net=feature_net,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n softmax_output=False,\n ).to(args.device)\n critic = DiscreteCritic(\n preprocess_net=feature_net,\n hidden_sizes=args.hidden_sizes,\n last_size=int(np.prod(args.action_shape)),\n ).to(args.device)\n optim = AdamOptimizerFactory(lr=args.lr)\n # define policy and algorithm\n policy = DiscreteActorPolicy(\n actor=actor,\n action_space=env.action_space,\n )\n algorithm: DiscreteCRR = DiscreteCRR(\n policy=policy,\n critic=critic,\n optim=optim,\n gamma=args.gamma,\n policy_improvement_mode=args.policy_improvement_mode,\n ratio_upper_bound=args.ratio_upper_bound,\n beta=args.beta,\n min_q_weight=args.min_q_weight,\n target_update_freq=args.target_update_freq,\n ).to(args.device)\n # load a previous policy\n if args.resume_path:\n algorithm.load_state_dict(torch.load(args.resume_path, map_location=args.device))\n print(\"Loaded agent from: \", args.resume_path)\n # buffer\n if args.buffer_from_rl_unplugged:\n buffer = load_buffer(args.load_buffer_name)\n else:\n assert os.path.exists(\n args.load_buffer_name,\n ), \"Please run atari_dqn.py first to get expert's data buffer.\"\n if args.load_buffer_name.endswith(\".pkl\"):\n with open(args.load_buffer_name, \"rb\") as f:\n buffer = pickle.load(f)\n elif args.load_buffer_name.endswith(\".hdf5\"):\n buffer = VectorReplayBuffer.load_hdf5(args.load_buffer_name)\n else:\n print(f\"Unknown buffer format: {args.load_buffer_name}\")\n sys.exit(0)\n print(\"Replay buffer size:\", len(buffer), flush=True)\n\n # collector\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n args.algo_name = \"crr\"\n log_name = os.path.join(args.task, args.algo_name, str(args.seed), now)\n log_path = os.path.join(args.logdir, log_name)\n\n # logger\n logger_factory = LoggerFactoryDefault()\n if args.logger == \"wandb\":\n logger_factory.logger_type = \"wandb\"\n logger_factory.wandb_project = args.wandb_project\n else:\n logger_factory.logger_type = \"tensorboard\"\n\n logger = logger_factory.create_logger(\n log_dir=log_path,\n experiment_name=log_name,\n run_id=args.resume_id,\n config_dict=vars(args),\n )\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return False\n\n # watch agent's performance\n def watch() -> None:\n print(\"Setup test envs ...\")\n test_envs.seed(args.seed)\n print(\"Testing agent ...\")\n test_collector.reset()\n result = test_collector.collect(n_episode=args.num_test_envs, render=args.render)\n result.pprint_asdict()\n\n if args.watch:\n watch()\n sys.exit(0)\n\n result = algorithm.run_training(\n OfflineTrainerParams(\n buffer=buffer,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.update_per_epoch,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n )\n )\n\n pprint.pprint(result)\n watch()\n\n\nif __name__ == \"__main__\":\n main(get_args())\n" }, { "path": "examples/offline/atari_il.py", "content": "#!/usr/bin/env python3\n\nimport argparse\nimport datetime\nimport os\nimport pickle\nimport pprint\nimport sys\n\nimport numpy as np\nimport torch\n\nfrom examples.offline.utils import load_buffer\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.imitation.imitation_base import (\n ImitationPolicy,\n OfflineImitationLearning,\n)\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env.atari.atari_network import DQNet\nfrom tianshou.env.atari.atari_wrapper import make_atari_env\nfrom tianshou.highlevel.logger import LoggerFactoryDefault\nfrom tianshou.trainer import OfflineTrainerParams\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"PongNoFrameskip-v4\")\n parser.add_argument(\"--seed\", type=int, default=1626)\n parser.add_argument(\"--lr\", type=float, default=0.0001)\n parser.add_argument(\"--epoch\", type=int, default=100)\n parser.add_argument(\"--update_per_epoch\", type=int, default=10000)\n parser.add_argument(\"--batch_size\", type=int, default=32)\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\n parser.add_argument(\"--frames_stack\", type=int, default=4)\n parser.add_argument(\"--scale_obs\", type=int, default=0)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\"--resume_path\", type=str, default=None)\n parser.add_argument(\"--resume_id\", type=str, default=None)\n parser.add_argument(\n \"--logger\",\n type=str,\n default=\"tensorboard\",\n choices=[\"tensorboard\", \"wandb\"],\n )\n parser.add_argument(\"--wandb_project\", type=str, default=\"offline_atari.benchmark\")\n parser.add_argument(\n \"--watch\",\n default=False,\n action=\"store_true\",\n help=\"watch the play of pre-trained policy only\",\n )\n parser.add_argument(\"--log_interval\", type=int, default=100)\n parser.add_argument(\n \"--load_buffer_name\",\n type=str,\n default=\"./expert_DQN_PongNoFrameskip-v4.hdf5\",\n )\n parser.add_argument(\"--buffer_from_rl_unplugged\", action=\"store_true\", default=False)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n return parser.parse_known_args()[0]\n\n\ndef test_il(args: argparse.Namespace = get_args()) -> None:\n # envs\n env, _, test_envs = make_atari_env(\n args.task,\n args.seed,\n 1,\n args.num_test_envs,\n scale=args.scale_obs,\n frame_stack=args.frames_stack,\n )\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n assert isinstance(args.state_shape, tuple | list)\n assert len(args.state_shape) == 3\n c, h, w = args.state_shape\n # should be N_FRAMES x H x W\n print(\"Observations shape:\", args.state_shape)\n print(\"Actions shape:\", args.action_shape)\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n # model\n net = DQNet(c=c, h=h, w=w, action_shape=args.action_shape).to(args.device)\n optim = AdamOptimizerFactory(lr=args.lr)\n # define policy\n policy = ImitationPolicy(actor=net, action_space=env.action_space)\n algorithm: OfflineImitationLearning = OfflineImitationLearning(\n policy=policy,\n optim=optim,\n )\n # load a previous policy\n if args.resume_path:\n algorithm.load_state_dict(torch.load(args.resume_path, map_location=args.device))\n print(\"Loaded agent from: \", args.resume_path)\n # buffer\n if args.buffer_from_rl_unplugged:\n buffer = load_buffer(args.load_buffer_name)\n else:\n assert os.path.exists(\n args.load_buffer_name,\n ), \"Please run atari_dqn.py first to get expert's data buffer.\"\n if args.load_buffer_name.endswith(\".pkl\"):\n with open(args.load_buffer_name, \"rb\") as f:\n buffer = pickle.load(f)\n elif args.load_buffer_name.endswith(\".hdf5\"):\n buffer = VectorReplayBuffer.load_hdf5(args.load_buffer_name)\n else:\n print(f\"Unknown buffer format: {args.load_buffer_name}\")\n sys.exit(0)\n print(\"Replay buffer size:\", len(buffer), flush=True)\n\n # collector\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n args.algo_name = \"il\"\n log_name = os.path.join(args.task, args.algo_name, str(args.seed), now)\n log_path = os.path.join(args.logdir, log_name)\n\n # logger\n logger_factory = LoggerFactoryDefault()\n if args.logger == \"wandb\":\n logger_factory.logger_type = \"wandb\"\n logger_factory.wandb_project = args.wandb_project\n else:\n logger_factory.logger_type = \"tensorboard\"\n\n logger = logger_factory.create_logger(\n log_dir=log_path,\n experiment_name=log_name,\n run_id=args.resume_id,\n config_dict=vars(args),\n )\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return False\n\n # watch agent's performance\n def watch() -> None:\n print(\"Setup test envs ...\")\n test_envs.seed(args.seed)\n print(\"Testing agent ...\")\n test_collector.reset()\n result = test_collector.collect(n_episode=args.num_test_envs, render=args.render)\n result.pprint_asdict()\n\n if args.watch:\n watch()\n sys.exit(0)\n\n result = algorithm.run_training(\n OfflineTrainerParams(\n buffer=buffer,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.update_per_epoch,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n )\n )\n\n pprint.pprint(result)\n watch()\n\n\nif __name__ == \"__main__\":\n test_il(get_args())\n" }, { "path": "examples/offline/convert_rl_unplugged_atari.py", "content": "#!/usr/bin/env python3\n#\n# Adapted from\n# https://github.com/deepmind/deepmind-research/blob/master/rl_unplugged/atari.py\n#\n\"\"\"Convert Atari RL Unplugged datasets to HDF5 format.\n\nExamples in the dataset represent SARSA transitions stored during a\nDQN training run as described in https://arxiv.org/pdf/1907.04543.\n\nFor every training run we have recorded all 50 million transitions corresponding\nto 200 million environment steps (4x factor because of frame skipping). There\nare 5 separate datasets for each of the 45 games.\n\nEvery transition in the dataset is a tuple containing the following features:\n\n* o_t: Observation at time t. Observations have been processed using the\n canonical Atari frame processing, including 4x frame stacking. The shape\n of a single observation is [84, 84, 4].\n* a_t: Action taken at time t.\n* r_t: Reward after a_t.\n* d_t: Discount after a_t.\n* o_tp1: Observation at time t+1.\n* a_tp1: Action at time t+1.\n* extras:\n * episode_id: Episode identifier.\n * episode_return: Total episode return computed using per-step [-1, 1]\n clipping.\n\"\"\"\n\nimport os\nfrom argparse import ArgumentParser, Namespace\n\nimport h5py\nimport numpy as np\nimport numpy.typing as npt\nimport requests\nimport tensorflow as tf\nfrom tqdm import tqdm\n\nfrom tianshou.data import Batch\n\ntf.config.set_visible_devices([], \"GPU\")\n\n# 9 tuning games.\nTUNING_SUITE = [\n \"BeamRider\",\n \"DemonAttack\",\n \"DoubleDunk\",\n \"IceHockey\",\n \"MsPacman\",\n \"Pooyan\",\n \"RoadRunner\",\n \"Robotank\",\n \"Zaxxon\",\n]\n\n# 36 testing games.\nTESTING_SUITE = [\n \"Alien\",\n \"Amidar\",\n \"Assault\",\n \"Asterix\",\n \"Atlantis\",\n \"BankHeist\",\n \"BattleZone\",\n \"Boxing\",\n \"Breakout\",\n \"Carnival\",\n \"Centipede\",\n \"ChopperCommand\",\n \"CrazyClimber\",\n \"Enduro\",\n \"FishingDerby\",\n \"Freeway\",\n \"Frostbite\",\n \"Gopher\",\n \"Gravitar\",\n \"Hero\",\n \"Jamesbond\",\n \"Kangaroo\",\n \"Krull\",\n \"KungFuMaster\",\n \"NameThisGame\",\n \"Phoenix\",\n \"Pong\",\n \"Qbert\",\n \"Riverraid\",\n \"Seaquest\",\n \"SpaceInvaders\",\n \"StarGunner\",\n \"TimePilot\",\n \"UpNDown\",\n \"VideoPinball\",\n \"WizardOfWor\",\n \"YarsRevenge\",\n]\n\n# Total of 45 games.\nALL_GAMES = TUNING_SUITE + TESTING_SUITE\nURL_PREFIX = \"http://storage.googleapis.com/rl_unplugged/atari\"\n\n\ndef _filename(run_id: int, shard_id: int, total_num_shards: int = 100) -> str:\n return f\"run_{run_id}-{shard_id:05d}-of-{total_num_shards:05d}\"\n\n\ndef _decode_frames(pngs: tf.Tensor) -> tf.Tensor:\n \"\"\"Decode PNGs.\n\n :param pngs: String Tensor of size (4,) containing PNG encoded images.\n\n :returns: Tensor of size (4, 84, 84) containing decoded images.\n \"\"\"\n # Statically unroll png decoding\n frames = [tf.image.decode_png(pngs[i], channels=1) for i in range(4)]\n # NOTE: to match tianshou's convention for framestacking\n frames = tf.squeeze(tf.stack(frames, axis=0))\n frames.set_shape((4, 84, 84))\n return frames\n\n\ndef _make_tianshou_batch(\n o_t: tf.Tensor,\n a_t: tf.Tensor,\n r_t: tf.Tensor,\n d_t: tf.Tensor,\n o_tp1: tf.Tensor,\n a_tp1: tf.Tensor,\n) -> Batch:\n \"\"\"Create Tianshou batch with offline data.\n\n :param o_t: Observation at time t.\n :param a_t: Action at time t.\n :param r_t: Reward at time t.\n :param d_t: Discount at time t.\n :param o_tp1: Observation at time t+1.\n :param a_tp1: Action at time t+1.\n\n :returns: A tianshou.data.Batch object.\n \"\"\"\n return Batch(\n obs=o_t.numpy(),\n act=a_t.numpy(),\n rew=r_t.numpy(),\n done=1 - d_t.numpy(),\n obs_next=o_tp1.numpy(),\n )\n\n\ndef _tf_example_to_tianshou_batch(tf_example: tf.train.Example) -> Batch:\n \"\"\"Create a tianshou Batch replay sample from a TF example.\"\"\"\n # Parse tf.Example.\n feature_description = {\n \"o_t\": tf.io.FixedLenFeature([4], tf.string),\n \"o_tp1\": tf.io.FixedLenFeature([4], tf.string),\n \"a_t\": tf.io.FixedLenFeature([], tf.int64),\n \"a_tp1\": tf.io.FixedLenFeature([], tf.int64),\n \"r_t\": tf.io.FixedLenFeature([], tf.float32),\n \"d_t\": tf.io.FixedLenFeature([], tf.float32),\n \"episode_id\": tf.io.FixedLenFeature([], tf.int64),\n \"episode_return\": tf.io.FixedLenFeature([], tf.float32),\n }\n data = tf.io.parse_single_example(tf_example, feature_description)\n\n # Process data.\n o_t = _decode_frames(data[\"o_t\"])\n o_tp1 = _decode_frames(data[\"o_tp1\"])\n a_t = tf.cast(data[\"a_t\"], tf.int32)\n a_tp1 = tf.cast(data[\"a_tp1\"], tf.int32)\n\n # Build tianshou Batch replay sample.\n return _make_tianshou_batch(o_t, a_t, data[\"r_t\"], data[\"d_t\"], o_tp1, a_tp1)\n\n\n# Adapted From https://gist.github.com/yanqd0/c13ed29e29432e3cf3e7c38467f42f51\ndef download(url: str, fname: str, chunk_size: int | None = 1024) -> None:\n resp = requests.get(url, stream=True)\n total = int(resp.headers.get(\"content-length\", 0))\n if os.path.exists(fname):\n print(f\"Found cached file at {fname}.\")\n return\n with (\n open(fname, \"wb\") as ofile,\n tqdm(\n desc=fname,\n total=total,\n unit=\"iB\",\n unit_scale=True,\n unit_divisor=1024,\n ) as bar,\n ):\n for data in resp.iter_content(chunk_size=chunk_size):\n size = ofile.write(data)\n bar.update(size)\n\n\ndef process_shard(url: str, fname: str, ofname: str, maxsize: int = 500000) -> None:\n download(url, fname)\n obs: npt.NDArray[np.uint8] = np.ndarray((maxsize, 4, 84, 84), dtype=\"uint8\")\n act: npt.NDArray[np.int64] = np.ndarray((maxsize,), dtype=\"int64\")\n rew: npt.NDArray[np.float32] = np.ndarray((maxsize,), dtype=\"float32\")\n done: npt.NDArray[np.bool_] = np.ndarray((maxsize,), dtype=\"bool\")\n obs_next: npt.NDArray[np.uint8] = np.ndarray((maxsize, 4, 84, 84), dtype=\"uint8\")\n i = 0\n file_ds = tf.data.TFRecordDataset(fname, compression_type=\"GZIP\")\n for example in file_ds:\n if i >= maxsize:\n break\n batch = _tf_example_to_tianshou_batch(example)\n obs[i], act[i], rew[i], done[i], obs_next[i] = (\n batch.obs,\n batch.act,\n batch.rew,\n batch.done,\n batch.obs_next,\n )\n i += 1\n if i % 1000 == 0:\n print(f\"...{i}\", end=\"\", flush=True)\n print(\"\\nDataset size:\", i)\n # Following D4RL dataset naming conventions\n with h5py.File(ofname, \"w\") as f:\n f.create_dataset(\"observations\", data=obs, compression=\"gzip\")\n f.create_dataset(\"actions\", data=act, compression=\"gzip\")\n f.create_dataset(\"rewards\", data=rew, compression=\"gzip\")\n f.create_dataset(\"terminals\", data=done, compression=\"gzip\")\n f.create_dataset(\"next_observations\", data=obs_next, compression=\"gzip\")\n\n\ndef process_dataset(\n task: str,\n download_path: str,\n dst_path: str,\n run_id: int = 1,\n shard_id: int = 0,\n total_num_shards: int = 100,\n) -> None:\n fn = f\"{task}/{_filename(run_id, shard_id, total_num_shards=total_num_shards)}\"\n url = f\"{URL_PREFIX}/{fn}\"\n filepath = f\"{download_path}/{fn}\"\n ofname = f\"{dst_path}/{fn}.hdf5\"\n process_shard(url, filepath, ofname)\n\n\ndef main(args: Namespace) -> None:\n if args.task not in ALL_GAMES:\n raise KeyError(f\"`{args.task}` is not in the list of games.\")\n fn = _filename(args.run_id, args.shard_id, total_num_shards=args.total_num_shards)\n dataset_path = os.path.join(args.dataset_dir, args.task, f\"{fn}.hdf5\")\n if os.path.exists(dataset_path):\n raise OSError(f\"Found existing dataset at {dataset_path}. Will not overwrite.\")\n args.cache_dir = os.environ.get(\"RLU_CACHE_DIR\", args.cache_dir)\n args.dataset_dir = os.environ.get(\"RLU_DATASET_DIR\", args.dataset_dir)\n cache_path = os.path.join(args.cache_dir, args.task)\n os.makedirs(cache_path, exist_ok=True)\n dst_path = os.path.join(args.dataset_dir, args.task)\n os.makedirs(dst_path, exist_ok=True)\n process_dataset(\n args.task,\n args.cache_dir,\n args.dataset_dir,\n run_id=args.run_id,\n shard_id=args.shard_id,\n total_num_shards=args.total_num_shards,\n )\n\n\nif __name__ == \"__main__\":\n parser = ArgumentParser(usage=__doc__)\n parser.add_argument(\"--task\", required=True, help=\"Name of the Atari game.\")\n parser.add_argument(\n \"--run_id\",\n type=int,\n default=1,\n help=\"Run id to download and convert. Value in [1..5].\",\n )\n parser.add_argument(\n \"--shard_id\",\n type=int,\n default=0,\n help=\"Shard id to download and convert. Value in [0..99].\",\n )\n parser.add_argument(\"--total_num_shards\", type=int, default=100, help=\"Total number of shards.\")\n parser.add_argument(\n \"--dataset_dir\",\n default=os.path.expanduser(\"~/.rl_unplugged/datasets\"),\n help=\"Directory for converted hdf5 files.\",\n )\n parser.add_argument(\n \"--cache_dir\",\n default=os.path.expanduser(\"~/.rl_unplugged/cache\"),\n help=\"Directory for downloaded original datasets.\",\n )\n args = parser.parse_args()\n main(args)\n" }, { "path": "examples/offline/d4rl_bcq.py", "content": "#!/usr/bin/env python3\n\nimport argparse\nimport datetime\nimport os\nimport pprint\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom examples.offline.utils import load_buffer_d4rl\nfrom tianshou.algorithm import BCQ\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.imitation.bcq import BCQPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats\nfrom tianshou.env import SubprocVectorEnv\nfrom tianshou.trainer import OfflineTrainerParams\nfrom tianshou.utils import TensorboardLogger, WandbLogger\nfrom tianshou.utils.net.common import MLP, Net\nfrom tianshou.utils.net.continuous import VAE, ContinuousCritic, Perturbation\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"HalfCheetah-v2\")\n parser.add_argument(\"--seed\", type=int, default=0)\n parser.add_argument(\"--expert_data_task\", type=str, default=\"halfcheetah-expert-v2\")\n parser.add_argument(\"--buffer_size\", type=int, default=1000000)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[256, 256])\n parser.add_argument(\"--actor_lr\", type=float, default=1e-3)\n parser.add_argument(\"--critic_lr\", type=float, default=1e-3)\n parser.add_argument(\"--start_timesteps\", type=int, default=10000)\n parser.add_argument(\"--epoch\", type=int, default=200)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=5000)\n parser.add_argument(\"--n_step\", type=int, default=3)\n parser.add_argument(\"--batch_size\", type=int, default=256)\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=1 / 35)\n\n parser.add_argument(\"--vae_hidden_sizes\", type=int, nargs=\"*\", default=[512, 512])\n # default to 2 * action_dim\n parser.add_argument(\"--latent_dim\", type=int)\n parser.add_argument(\"--gamma\", default=0.99)\n parser.add_argument(\"--tau\", default=0.005)\n # Weighting for Clipped Double Q-learning in BCQ\n parser.add_argument(\"--lmbda\", default=0.75)\n # Max perturbation hyper-parameter for BCQ\n parser.add_argument(\"--phi\", default=0.05)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n parser.add_argument(\"--resume_path\", type=str, default=None)\n parser.add_argument(\"--resume_id\", type=str, default=None)\n parser.add_argument(\n \"--logger\",\n type=str,\n default=\"tensorboard\",\n choices=[\"tensorboard\", \"wandb\"],\n )\n parser.add_argument(\"--wandb_project\", type=str, default=\"offline_d4rl.benchmark\")\n parser.add_argument(\n \"--watch\",\n default=False,\n action=\"store_true\",\n help=\"watch the play of pre-trained policy only\",\n )\n return parser.parse_args()\n\n\ndef test_bcq() -> None:\n args = get_args()\n env = gym.make(args.task)\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n args.max_action = space_info.action_info.max_action\n print(\"device:\", args.device)\n print(\"Observations shape:\", args.state_shape)\n print(\"Actions shape:\", args.action_shape)\n print(\"Action range:\", args.min_action, args.max_action)\n\n args.state_dim = space_info.observation_info.obs_dim\n args.action_dim = space_info.action_info.action_dim\n print(\"Max_action\", args.max_action)\n\n # test_envs = gym.make(args.task)\n test_envs = SubprocVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n test_envs.seed(args.seed)\n\n # model\n # perturbation network\n net_a = MLP(\n input_dim=args.state_dim + args.action_dim,\n output_dim=args.action_dim,\n hidden_sizes=args.hidden_sizes,\n )\n actor = Perturbation(preprocess_net=net_a, max_action=args.max_action, phi=args.phi).to(\n args.device,\n )\n actor_optim = AdamOptimizerFactory(lr=args.actor_lr)\n\n net_c1 = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n concat=True,\n )\n net_c2 = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n concat=True,\n )\n critic1 = ContinuousCritic(preprocess_net=net_c1).to(args.device)\n critic1_optim = AdamOptimizerFactory(lr=args.critic_lr)\n critic2 = ContinuousCritic(preprocess_net=net_c2).to(args.device)\n critic2_optim = AdamOptimizerFactory(lr=args.critic_lr)\n\n # vae\n # output_dim = 0, so the last Module in the encoder is ReLU\n vae_encoder = MLP(\n input_dim=args.state_dim + args.action_dim,\n hidden_sizes=args.vae_hidden_sizes,\n )\n if not args.latent_dim:\n args.latent_dim = args.action_dim * 2\n vae_decoder = MLP(\n input_dim=args.state_dim + args.latent_dim,\n output_dim=args.action_dim,\n hidden_sizes=args.vae_hidden_sizes,\n )\n vae = VAE(\n encoder=vae_encoder,\n decoder=vae_decoder,\n hidden_dim=args.vae_hidden_sizes[-1],\n latent_dim=args.latent_dim,\n max_action=args.max_action,\n ).to(args.device)\n vae_optim = AdamOptimizerFactory()\n\n policy = BCQPolicy(\n actor_perturbation=actor,\n action_space=env.action_space,\n critic=critic1,\n vae=vae,\n )\n algorithm: BCQ = BCQ(\n policy=policy,\n actor_perturbation_optim=actor_optim,\n critic_optim=critic1_optim,\n critic2=critic2,\n critic2_optim=critic2_optim,\n vae_optim=vae_optim,\n gamma=args.gamma,\n tau=args.tau,\n lmbda=args.lmbda,\n ).to(args.device)\n\n # load a previous policy\n if args.resume_path:\n algorithm.load_state_dict(torch.load(args.resume_path, map_location=args.device))\n print(\"Loaded agent from: \", args.resume_path)\n\n # collector\n test_collector = Collector[CollectStats](algorithm, test_envs)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n args.algo_name = \"bcq\"\n log_name = os.path.join(args.task, args.algo_name, str(args.seed), now)\n log_path = os.path.join(args.logdir, log_name)\n\n # logger\n writer = SummaryWriter(log_path)\n writer.add_text(\"args\", str(args))\n logger: WandbLogger | TensorboardLogger\n if args.logger == \"tensorboard\":\n logger = TensorboardLogger(writer)\n else:\n logger = WandbLogger(\n save_interval=1,\n name=log_name.replace(os.path.sep, \"__\"),\n run_id=args.resume_id,\n config=args,\n project=args.wandb_project,\n )\n logger.load(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def watch() -> None:\n if args.resume_path is None:\n args.resume_path = os.path.join(log_path, \"policy.pth\")\n\n algorithm.load_state_dict(torch.load(args.resume_path, map_location=torch.device(\"cpu\")))\n collector = Collector[CollectStats](algorithm, env)\n collector.collect(n_episode=1, render=1 / 35)\n\n if not args.watch:\n replay_buffer = load_buffer_d4rl(args.expert_data_task)\n # train\n result = algorithm.run_training(\n OfflineTrainerParams(\n buffer=replay_buffer,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n save_best_fn=save_best_fn,\n logger=logger,\n )\n )\n pprint.pprint(result)\n else:\n watch()\n\n # Let's watch its performance!\n test_envs.seed(args.seed)\n test_collector.reset()\n collector_stats = test_collector.collect(n_episode=args.num_test_envs, render=args.render)\n print(collector_stats)\n\n\nif __name__ == \"__main__\":\n test_bcq()\n" }, { "path": "examples/offline/d4rl_cql.py", "content": "#!/usr/bin/env python3\n\nimport argparse\nimport datetime\nimport os\nimport pprint\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom examples.offline.utils import load_buffer_d4rl\nfrom tianshou.algorithm import CQL\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.sac import AutoAlpha, SACPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats\nfrom tianshou.env import SubprocVectorEnv\nfrom tianshou.trainer import OfflineTrainerParams\nfrom tianshou.utils import TensorboardLogger, WandbLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.continuous import ContinuousActorProbabilistic, ContinuousCritic\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"--task\",\n type=str,\n default=\"Hopper-v2\",\n help=\"The name of the OpenAI Gym environment to train on.\",\n )\n parser.add_argument(\n \"--seed\",\n type=int,\n default=0,\n help=\"The random seed to use.\",\n )\n parser.add_argument(\n \"--expert_data_task\",\n type=str,\n default=\"hopper-expert-v2\",\n help=\"The name of the OpenAI Gym environment to use for expert data collection.\",\n )\n parser.add_argument(\n \"--buffer_size\",\n type=int,\n default=1000000,\n help=\"The size of the replay buffer.\",\n )\n parser.add_argument(\n \"--hidden_sizes\",\n type=int,\n nargs=\"*\",\n default=[256, 256],\n help=\"The list of hidden sizes for the neural networks.\",\n )\n parser.add_argument(\n \"--actor_lr\",\n type=float,\n default=1e-4,\n help=\"The learning rate for the actor network.\",\n )\n parser.add_argument(\n \"--critic_lr\",\n type=float,\n default=3e-4,\n help=\"The learning rate for the critic network.\",\n )\n parser.add_argument(\n \"--alpha\",\n type=float,\n default=0.2,\n help=\"The weight of the entropy term in the loss function.\",\n )\n parser.add_argument(\n \"--auto_alpha\",\n default=True,\n action=\"store_true\",\n help=\"Whether to use automatic entropy tuning.\",\n )\n parser.add_argument(\n \"--alpha_lr\",\n type=float,\n default=1e-4,\n help=\"The learning rate for the entropy tuning.\",\n )\n parser.add_argument(\n \"--cql_alpha_lr\",\n type=float,\n default=3e-4,\n help=\"The learning rate for the CQL entropy tuning.\",\n )\n parser.add_argument(\n \"--start_timesteps\",\n type=int,\n default=10000,\n help=\"The number of timesteps before starting to train.\",\n )\n parser.add_argument(\n \"--epoch\",\n type=int,\n default=200,\n help=\"The number of epochs to train for.\",\n )\n parser.add_argument(\n \"--epoch_num_steps\",\n type=int,\n default=5000,\n help=\"The number of steps per epoch.\",\n )\n parser.add_argument(\n \"--n_step\",\n type=int,\n default=3,\n help=\"The number of steps to use for N-step TD learning.\",\n )\n parser.add_argument(\n \"--batch_size\",\n type=int,\n default=256,\n help=\"The batch size for training.\",\n )\n parser.add_argument(\n \"--tau\",\n type=float,\n default=0.005,\n help=\"The soft target update coefficient.\",\n )\n parser.add_argument(\n \"--temperature\",\n type=float,\n default=1.0,\n help=\"The temperature for the Boltzmann policy.\",\n )\n parser.add_argument(\n \"--cql_weight\",\n type=float,\n default=1.0,\n help=\"The weight of the CQL loss term.\",\n )\n parser.add_argument(\n \"--with_lagrange\",\n type=bool,\n default=True,\n help=\"Whether to use the Lagrange multiplier for CQL.\",\n )\n parser.add_argument(\n \"--calibrated\",\n type=bool,\n default=True,\n help=\"Whether to use calibration for CQL.\",\n )\n parser.add_argument(\n \"--lagrange_threshold\",\n type=float,\n default=10.0,\n help=\"The Lagrange multiplier threshold for CQL.\",\n )\n parser.add_argument(\"--gamma\", type=float, default=0.99, help=\"The discount factor\")\n parser.add_argument(\n \"--eval_freq\",\n type=int,\n default=1,\n help=\"The frequency of evaluation.\",\n )\n parser.add_argument(\n \"--num_test_envs\",\n type=int,\n default=10,\n help=\"The number of episodes to evaluate for.\",\n )\n parser.add_argument(\n \"--logdir\",\n type=str,\n default=\"log\",\n help=\"The directory to save logs to.\",\n )\n parser.add_argument(\n \"--render\",\n type=float,\n default=1 / 35,\n help=\"The frequency of rendering the environment.\",\n )\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n help=\"The device to train on (cpu or cuda).\",\n )\n parser.add_argument(\n \"--resume_path\",\n type=str,\n default=None,\n help=\"The path to the checkpoint to resume from.\",\n )\n parser.add_argument(\n \"--resume_id\",\n type=str,\n default=None,\n help=\"The ID of the checkpoint to resume from.\",\n )\n parser.add_argument(\n \"--logger\",\n type=str,\n default=\"tensorboard\",\n choices=[\"tensorboard\", \"wandb\"],\n )\n parser.add_argument(\"--wandb_project\", type=str, default=\"offline_d4rl.benchmark\")\n parser.add_argument(\n \"--watch\",\n default=False,\n action=\"store_true\",\n help=\"watch the play of pre-trained policy only\",\n )\n return parser.parse_args()\n\n\ndef test_cql() -> None:\n args = get_args()\n env = gym.make(args.task)\n assert isinstance(env.action_space, gym.spaces.Box)\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n args.max_action = space_info.action_info.max_action\n args.min_action = space_info.action_info.min_action\n print(\"device:\", args.device)\n print(\"Observations shape:\", args.state_shape)\n print(\"Actions shape:\", args.action_shape)\n print(\"Action range:\", args.min_action, args.max_action)\n\n args.state_dim = space_info.observation_info.obs_dim\n args.action_dim = space_info.action_info.action_dim\n print(\"Max_action\", args.max_action)\n\n # test_envs = gym.make(args.task)\n test_envs = SubprocVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n test_envs.seed(args.seed)\n\n # model\n # actor network\n net_a = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n )\n actor = ContinuousActorProbabilistic(\n preprocess_net=net_a,\n action_shape=args.action_shape,\n unbounded=True,\n conditioned_sigma=True,\n ).to(args.device)\n actor_optim = AdamOptimizerFactory(lr=args.actor_lr)\n\n # critic network\n net_c1 = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n concat=True,\n )\n net_c2 = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n concat=True,\n )\n critic = ContinuousCritic(preprocess_net=net_c1).to(args.device)\n critic_optim = AdamOptimizerFactory(lr=args.critic_lr)\n critic2 = ContinuousCritic(preprocess_net=net_c2).to(args.device)\n critic2_optim = AdamOptimizerFactory(lr=args.critic_lr)\n\n if args.auto_alpha:\n target_entropy = -args.action_dim\n log_alpha = 0.0\n alpha_optim = AdamOptimizerFactory(lr=args.alpha_lr)\n args.alpha = AutoAlpha(target_entropy, log_alpha, alpha_optim).to(args.device)\n\n policy = SACPolicy(\n actor=actor,\n action_space=env.action_space,\n )\n algorithm: CQL = CQL(\n policy=policy,\n policy_optim=actor_optim,\n critic=critic,\n critic_optim=critic_optim,\n critic2=critic2,\n critic2_optim=critic2_optim,\n calibrated=args.calibrated,\n cql_alpha_lr=args.cql_alpha_lr,\n cql_weight=args.cql_weight,\n tau=args.tau,\n gamma=args.gamma,\n alpha=args.alpha,\n temperature=args.temperature,\n with_lagrange=args.with_lagrange,\n lagrange_threshold=args.lagrange_threshold,\n min_action=args.min_action,\n max_action=args.max_action,\n ).to(args.device)\n\n # load a previous policy\n if args.resume_path:\n algorithm.load_state_dict(torch.load(args.resume_path, map_location=args.device))\n print(\"Loaded agent from: \", args.resume_path)\n\n # collector\n test_collector = Collector[CollectStats](algorithm, test_envs)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n args.algo_name = \"cql\"\n log_name = os.path.join(args.task, args.algo_name, str(args.seed), now)\n log_path = os.path.join(args.logdir, log_name)\n\n # logger\n writer = SummaryWriter(log_path)\n writer.add_text(\"args\", str(args))\n logger: WandbLogger | TensorboardLogger\n if args.logger == \"tensorboard\":\n logger = TensorboardLogger(writer)\n else:\n logger = WandbLogger(\n save_interval=1,\n name=log_name.replace(os.path.sep, \"__\"),\n run_id=args.resume_id,\n config=args,\n project=args.wandb_project,\n )\n logger.load(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def watch() -> None:\n if args.resume_path is None:\n args.resume_path = os.path.join(log_path, \"policy.pth\")\n\n algorithm.load_state_dict(torch.load(args.resume_path, map_location=torch.device(\"cpu\")))\n collector = Collector[CollectStats](algorithm, env)\n collector.collect(n_episode=1, render=1 / 35)\n\n if not args.watch:\n replay_buffer = load_buffer_d4rl(args.expert_data_task)\n # train\n result = algorithm.run_training(\n OfflineTrainerParams(\n buffer=replay_buffer,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n save_best_fn=save_best_fn,\n logger=logger,\n )\n )\n pprint.pprint(result)\n else:\n watch()\n\n # Let's watch its performance!\n test_envs.seed(args.seed)\n test_collector.reset()\n collector_stats = test_collector.collect(n_episode=args.num_test_envs, render=args.render)\n print(collector_stats)\n\n\nif __name__ == \"__main__\":\n test_cql()\n" }, { "path": "examples/offline/d4rl_il.py", "content": "#!/usr/bin/env python3\n\nimport argparse\nimport datetime\nimport os\nimport pprint\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom examples.offline.utils import load_buffer_d4rl\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.imitation.imitation_base import (\n ImitationPolicy,\n OfflineImitationLearning,\n)\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats\nfrom tianshou.env import SubprocVectorEnv\nfrom tianshou.trainer import OfflineTrainerParams\nfrom tianshou.utils import TensorboardLogger, WandbLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.continuous import ContinuousActorDeterministic\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"HalfCheetah-v2\")\n parser.add_argument(\"--seed\", type=int, default=0)\n parser.add_argument(\"--expert_data_task\", type=str, default=\"halfcheetah-expert-v2\")\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[256, 256])\n parser.add_argument(\"--lr\", type=float, default=1e-4)\n parser.add_argument(\"--epoch\", type=int, default=200)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=5000)\n parser.add_argument(\"--batch_size\", type=int, default=256)\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=1 / 35)\n parser.add_argument(\"--gamma\", default=0.99)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n parser.add_argument(\"--resume_path\", type=str, default=None)\n parser.add_argument(\"--resume_id\", type=str, default=None)\n parser.add_argument(\n \"--logger\",\n type=str,\n default=\"tensorboard\",\n choices=[\"tensorboard\", \"wandb\"],\n )\n parser.add_argument(\"--wandb_project\", type=str, default=\"offline_d4rl.benchmark\")\n parser.add_argument(\n \"--watch\",\n default=False,\n action=\"store_true\",\n help=\"watch the play of pre-trained policy only\",\n )\n return parser.parse_args()\n\n\ndef test_il() -> None:\n args = get_args()\n env = gym.make(args.task)\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n args.max_action = space_info.action_info.max_action\n print(\"device:\", args.device)\n print(\"Observations shape:\", args.state_shape)\n print(\"Actions shape:\", args.action_shape)\n print(\"Action range:\", args.min_action, args.max_action)\n\n args.state_dim = space_info.observation_info.obs_dim\n args.action_dim = space_info.action_info.action_dim\n print(\"Max_action\", args.max_action)\n\n test_envs = SubprocVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n test_envs.seed(args.seed)\n\n # model\n net = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n )\n actor = ContinuousActorDeterministic(\n preprocess_net=net,\n action_shape=args.action_shape,\n max_action=args.max_action,\n ).to(args.device)\n optim = AdamOptimizerFactory(lr=args.lr)\n\n policy = ImitationPolicy(\n actor=actor,\n action_space=env.action_space,\n action_scaling=True,\n action_bound_method=\"clip\",\n )\n algorithm: OfflineImitationLearning = OfflineImitationLearning(\n policy=policy,\n optim=optim,\n )\n\n # load a previous policy\n if args.resume_path:\n algorithm.load_state_dict(torch.load(args.resume_path, map_location=args.device))\n print(\"Loaded agent from: \", args.resume_path)\n\n # collector\n test_collector = Collector[CollectStats](algorithm, test_envs)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n args.algo_name = \"cql\"\n log_name = os.path.join(args.task, args.algo_name, str(args.seed), now)\n log_path = os.path.join(args.logdir, log_name)\n\n # logger\n writer = SummaryWriter(log_path)\n writer.add_text(\"args\", str(args))\n logger: WandbLogger | TensorboardLogger\n if args.logger == \"tensorboard\":\n logger = TensorboardLogger(writer)\n else:\n logger = WandbLogger(\n save_interval=1,\n name=log_name.replace(os.path.sep, \"__\"),\n run_id=args.resume_id,\n config=args,\n project=args.wandb_project,\n )\n logger.load(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def watch() -> None:\n if args.resume_path is None:\n args.resume_path = os.path.join(log_path, \"policy.pth\")\n\n algorithm.load_state_dict(torch.load(args.resume_path, map_location=torch.device(\"cpu\")))\n collector = Collector[CollectStats](algorithm, env)\n collector.collect(n_episode=1, render=1 / 35)\n\n if not args.watch:\n replay_buffer = load_buffer_d4rl(args.expert_data_task)\n # train\n result = algorithm.run_training(\n OfflineTrainerParams(\n buffer=replay_buffer,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n save_best_fn=save_best_fn,\n logger=logger,\n )\n )\n pprint.pprint(result)\n else:\n watch()\n\n # Let's watch its performance!\n test_envs.seed(args.seed)\n test_collector.reset()\n collector_stats = test_collector.collect(n_episode=args.num_test_envs, render=args.render)\n print(collector_stats)\n\n\nif __name__ == \"__main__\":\n test_il()\n" }, { "path": "examples/offline/d4rl_td3_bc.py", "content": "#!/usr/bin/env python3\n\nimport argparse\nimport datetime\nimport os\nimport pprint\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom examples.offline.utils import load_buffer_d4rl, normalize_all_obs_in_replay_buffer\nfrom tianshou.algorithm import TD3BC\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.ddpg import ContinuousDeterministicPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats\nfrom tianshou.env import BaseVectorEnv, SubprocVectorEnv, VectorEnvNormObs\nfrom tianshou.exploration import GaussianNoise\nfrom tianshou.trainer import OfflineTrainerParams\nfrom tianshou.utils import TensorboardLogger, WandbLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.continuous import ContinuousActorDeterministic, ContinuousCritic\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"HalfCheetah-v2\")\n parser.add_argument(\"--seed\", type=int, default=0)\n parser.add_argument(\"--expert_data_task\", type=str, default=\"halfcheetah-expert-v2\")\n parser.add_argument(\"--buffer_size\", type=int, default=1000000)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[256, 256])\n parser.add_argument(\"--actor_lr\", type=float, default=3e-4)\n parser.add_argument(\"--critic_lr\", type=float, default=3e-4)\n parser.add_argument(\"--epoch\", type=int, default=200)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=5000)\n parser.add_argument(\"--n_step\", type=int, default=3)\n parser.add_argument(\"--batch_size\", type=int, default=256)\n\n parser.add_argument(\"--alpha\", type=float, default=2.5)\n parser.add_argument(\"--exploration_noise\", type=float, default=0.1)\n parser.add_argument(\"--policy_noise\", type=float, default=0.2)\n parser.add_argument(\"--noise_clip\", type=float, default=0.5)\n parser.add_argument(\"--update_actor_freq\", type=int, default=2)\n parser.add_argument(\"--tau\", type=float, default=0.005)\n parser.add_argument(\"--gamma\", type=float, default=0.99)\n parser.add_argument(\"--norm_obs\", type=int, default=1)\n\n parser.add_argument(\"--eval_freq\", type=int, default=1)\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=1 / 35)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n parser.add_argument(\"--resume_path\", type=str, default=None)\n parser.add_argument(\"--resume_id\", type=str, default=None)\n parser.add_argument(\n \"--logger\",\n type=str,\n default=\"tensorboard\",\n choices=[\"tensorboard\", \"wandb\"],\n )\n parser.add_argument(\"--wandb_project\", type=str, default=\"offline_d4rl.benchmark\")\n parser.add_argument(\n \"--watch\",\n default=False,\n action=\"store_true\",\n help=\"watch the play of pre-trained policy only\",\n )\n return parser.parse_args()\n\n\ndef test_td3_bc() -> None:\n args = get_args()\n env = gym.make(args.task)\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n args.max_action = space_info.action_info.max_action\n args.min_action = space_info.action_info.min_action\n print(\"device:\", args.device)\n print(\"Observations shape:\", args.state_shape)\n print(\"Actions shape:\", args.action_shape)\n print(\"Action range:\", args.min_action, args.max_action)\n\n args.state_dim = space_info.observation_info.obs_dim\n args.action_dim = space_info.action_info.action_dim\n print(\"Max_action\", args.max_action)\n\n test_envs: BaseVectorEnv\n test_envs = SubprocVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n if args.norm_obs:\n test_envs = VectorEnvNormObs(test_envs, update_obs_rms=False)\n\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n test_envs.seed(args.seed)\n\n # model\n # actor network\n net_a = Net(\n state_shape=args.state_shape,\n hidden_sizes=args.hidden_sizes,\n )\n actor = ContinuousActorDeterministic(\n preprocess_net=net_a,\n action_shape=args.action_shape,\n max_action=args.max_action,\n ).to(args.device)\n actor_optim = AdamOptimizerFactory(lr=args.actor_lr)\n\n # critic network\n net_c1 = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n concat=True,\n )\n net_c2 = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n concat=True,\n )\n critic1 = ContinuousCritic(preprocess_net=net_c1).to(args.device)\n critic1_optim = AdamOptimizerFactory(lr=args.critic_lr)\n critic2 = ContinuousCritic(preprocess_net=net_c2).to(args.device)\n critic2_optim = AdamOptimizerFactory(lr=args.critic_lr)\n\n policy = ContinuousDeterministicPolicy(\n actor=actor,\n exploration_noise=GaussianNoise(sigma=args.exploration_noise),\n action_space=env.action_space,\n )\n algorithm: TD3BC = TD3BC(\n policy=policy,\n policy_optim=actor_optim,\n critic=critic1,\n critic_optim=critic1_optim,\n critic2=critic2,\n critic2_optim=critic2_optim,\n tau=args.tau,\n gamma=args.gamma,\n policy_noise=args.policy_noise,\n update_actor_freq=args.update_actor_freq,\n noise_clip=args.noise_clip,\n alpha=args.alpha,\n n_step_return_horizon=args.n_step,\n )\n\n # load a previous policy\n if args.resume_path:\n algorithm.load_state_dict(torch.load(args.resume_path, map_location=args.device))\n print(\"Loaded agent from: \", args.resume_path)\n\n # collector\n test_collector = Collector[CollectStats](algorithm, test_envs)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n args.algo_name = \"td3_bc\"\n log_name = os.path.join(args.task, args.algo_name, str(args.seed), now)\n log_path = os.path.join(args.logdir, log_name)\n\n # logger\n writer = SummaryWriter(log_path)\n writer.add_text(\"args\", str(args))\n logger: WandbLogger | TensorboardLogger\n if args.logger == \"tensorboard\":\n logger = TensorboardLogger(writer)\n else:\n logger = WandbLogger(\n save_interval=1,\n name=log_name.replace(os.path.sep, \"__\"),\n run_id=args.resume_id,\n config=args,\n project=args.wandb_project,\n )\n logger.load(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def watch() -> None:\n if args.resume_path is None:\n args.resume_path = os.path.join(log_path, \"policy.pth\")\n\n algorithm.load_state_dict(torch.load(args.resume_path, map_location=torch.device(\"cpu\")))\n collector = Collector[CollectStats](algorithm, env)\n collector.collect(n_episode=1, render=1 / 35)\n\n if not args.watch:\n replay_buffer = load_buffer_d4rl(args.expert_data_task)\n if args.norm_obs:\n replay_buffer, obs_rms = normalize_all_obs_in_replay_buffer(replay_buffer)\n test_envs.set_obs_rms(obs_rms)\n # train\n result = algorithm.run_training(\n OfflineTrainerParams(\n buffer=replay_buffer,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n save_best_fn=save_best_fn,\n logger=logger,\n )\n )\n pprint.pprint(result)\n else:\n watch()\n\n # Let's watch its performance!\n test_envs.seed(args.seed)\n test_collector.reset()\n collector_stats = test_collector.collect(n_episode=args.num_test_envs, render=args.render)\n print(collector_stats)\n\n\nif __name__ == \"__main__\":\n test_td3_bc()\n" }, { "path": "examples/offline/utils.py", "content": "import d4rl\nimport gymnasium as gym\nimport h5py\nimport numpy as np\n\nfrom tianshou.data import ReplayBuffer\nfrom tianshou.utils import RunningMeanStd\n\n\ndef load_buffer_d4rl(expert_data_task: str) -> ReplayBuffer:\n dataset = d4rl.qlearning_dataset(gym.make(expert_data_task))\n return ReplayBuffer.from_data(\n obs=dataset[\"observations\"],\n act=dataset[\"actions\"],\n rew=dataset[\"rewards\"],\n done=dataset[\"terminals\"],\n obs_next=dataset[\"next_observations\"],\n terminated=dataset[\"terminals\"],\n truncated=np.zeros(len(dataset[\"terminals\"])),\n )\n\n\ndef load_buffer(buffer_path: str) -> ReplayBuffer:\n with h5py.File(buffer_path, \"r\") as dataset:\n return ReplayBuffer.from_data(\n obs=dataset[\"observations\"],\n act=dataset[\"actions\"],\n rew=dataset[\"rewards\"],\n done=dataset[\"terminals\"],\n obs_next=dataset[\"next_observations\"],\n terminated=dataset[\"terminals\"],\n truncated=np.zeros(len(dataset[\"terminals\"])),\n )\n\n\ndef normalize_all_obs_in_replay_buffer(\n replay_buffer: ReplayBuffer,\n) -> tuple[ReplayBuffer, RunningMeanStd]:\n # compute obs mean and var\n obs_rms = RunningMeanStd()\n obs_rms.update(replay_buffer.obs)\n _eps = np.finfo(np.float32).eps.item()\n # normalize obs\n replay_buffer._meta[\"obs\"] = (replay_buffer.obs - obs_rms.mean) / np.sqrt(obs_rms.var + _eps)\n replay_buffer._meta[\"obs_next\"] = (replay_buffer.obs_next - obs_rms.mean) / np.sqrt(\n obs_rms.var + _eps,\n )\n return replay_buffer, obs_rms\n" }, { "path": "examples/vizdoom/.gitignore", "content": "_vizdoom.ini\n" }, { "path": "examples/vizdoom/README.md", "content": "# ViZDoom\n\n[ViZDoom](https://github.com/mwydmuch/ViZDoom) is a popular RL env for a famous first-person shooting game Doom. Here we\nprovide some results and intuitions for this scenario.\n\n## EnvPool\n\nWe highly recommend using envpool to run the following experiments. To install, in a linux machine, type:\n\n```bash\npip install envpool\n```\n\nAfter that, `make_vizdoom_env` will automatically switch to envpool's ViZDoom env. EnvPool's implementation is much\nfaster (about 2\\~3x faster for pure execution speed, 1.5x for overall RL training pipeline) than python vectorized env\nimplementation.\n\nFor more information, please refer to EnvPool's [GitHub](https://github.com/sail-sg/envpool/)\nand [Docs](https://envpool.readthedocs.io/en/latest/api/vizdoom.html).\n\n## Train\n\nTo train an agent:\n\n```bash\npython3 vizdoom_c51.py --task {D1_basic|D2_navigation|D3_battle|D4_battle2}\n```\n\nD1 (health gathering) should finish training (no death) in less than 500k env step (5 epochs);\n\nD3 can reach 1600+ reward (75+ killcount in 5 minutes);\n\nD4 can reach 700+ reward. Here is the result:\n\n(episode length, the maximum length is 2625 because we use frameskip=4, that is 10500/4=2625)\n\n![](results/c51/length.png)\n\n(episode reward)\n\n![](results/c51/reward.png)\n\nTo evaluate an agent's performance:\n\n```bash\npython3 vizdoom_c51.py --num_test_envs 100 --resume-path policy.pth --watch --task {D1_basic|D3_battle|D4_battle2}\n```\n\nTo save `.lmp` files for recording:\n\n```bash\npython3 vizdoom_c51.py --save-lmp --num_test_envs 100 --resume-path policy.pth --watch --task {D1_basic|D3_battle|D4_battle2}\n```\n\nit will store `lmp` file in `lmps/` directory. To watch these `lmp` files (for example, d3 lmp):\n\n```bash\npython3 replay.py maps/D3_battle.cfg episode_8_25.lmp\n```\n\nWe provide two lmp files (d3 best and d4 best) under `results/c51`, you can use the following command to enjoy:\n\n```bash\npython3 replay.py maps/D3_battle.cfg results/c51/d3.lmp\npython3 replay.py maps/D4_battle2.cfg results/c51/d4.lmp\n```\n\n## Maps\n\nSee [maps/README.md](maps/README.md)\n\n## Reward\n\n1. living reward is bad\n2. combo-action is really important\n3. negative reward for health and ammo2 is really helpful for d3/d4\n4. only with positive reward for health is really helpful for d1\n5. remove MOVE_BACKWARD may converge faster but the final performance may be lower\n\n## Algorithms\n\nThe setting is exactly the same as Atari. You can definitely try more algorithms listed in Atari example.\n\n### C51 (single run)\n\n| task | best reward | reward curve | parameters |\n|---------------|-------------|----------------------------------------|-------------------------------------------------|\n| D2_navigation | 747.52 | ![](results/c51/D2_navigation_rew.png) | `python3 vizdoom_c51.py --task \"D2_navigation\"` |\n| D3_battle | 1855.29 | ![](results/c51/D3_battle_rew.png) | `python3 vizdoom_c51.py --task \"D3_battle\"` |\n\n### PPO (single run)\n\n| task | best reward | reward curve | parameters |\n|---------------|-------------|----------------------------------------|-------------------------------------------------|\n| D2_navigation | 770.75 | ![](results/ppo/D2_navigation_rew.png) | `python3 vizdoom_ppo.py --task \"D2_navigation\"` |\n| D3_battle | 320.59 | ![](results/ppo/D3_battle_rew.png) | `python3 vizdoom_ppo.py --task \"D3_battle\"` |\n\n### PPO with ICM (single run)\n\n| task | best reward | reward curve | parameters |\n|---------------|-------------|--------------------------------------------|-------------------------------------------------------------------|\n| D2_navigation | 844.99 | ![](results/ppo_icm/D2_navigation_rew.png) | `python3 vizdoom_ppo.py --task \"D2_navigation\" --icm-lr-scale 10` |\n| D3_battle | 547.08 | ![](results/ppo_icm/D3_battle_rew.png) | `python3 vizdoom_ppo.py --task \"D3_battle\" --icm-lr-scale 10` |\n" }, { "path": "examples/vizdoom/env.py", "content": "import os\nfrom collections.abc import Sequence\nfrom typing import Any\n\nimport cv2\nimport gymnasium as gym\nimport numpy as np\nimport vizdoom as vzd\nfrom numpy.typing import NDArray\n\nfrom tianshou.env import ShmemVectorEnv\n\ntry:\n import envpool\nexcept ImportError:\n envpool = None\n\n\ndef normal_button_comb() -> list:\n actions = []\n m_forward = [[0.0], [1.0]]\n t_left_right = [[0.0, 0.0], [0.0, 1.0], [1.0, 0.0]]\n for i in m_forward:\n for j in t_left_right:\n actions.append(i + j)\n return actions\n\n\ndef battle_button_comb() -> list:\n actions = []\n m_forward_backward = [[0.0, 0.0], [0.0, 1.0], [1.0, 0.0]]\n m_left_right = [[0.0, 0.0], [0.0, 1.0], [1.0, 0.0]]\n t_left_right = [[0.0, 0.0], [0.0, 1.0], [1.0, 0.0]]\n attack = [[0.0], [1.0]]\n speed = [[0.0], [1.0]]\n\n for m in attack:\n for n in speed:\n for j in m_left_right:\n for i in m_forward_backward:\n for k in t_left_right:\n actions.append(i + j + k + m + n)\n return actions\n\n\nclass Env(gym.Env):\n def __init__(\n self,\n cfg_path: str,\n frameskip: int = 4,\n res: Sequence[int] = (4, 40, 60),\n save_lmp: bool = False,\n ) -> None:\n super().__init__()\n self.save_lmp = save_lmp\n self.health_setting = \"battle\" in cfg_path\n if save_lmp:\n os.makedirs(\"lmps\", exist_ok=True)\n self.res = res\n self.skip = frameskip\n self.observation_space = gym.spaces.Box(low=0, high=255, shape=res, dtype=np.float32)\n self.game = vzd.DoomGame()\n self.game.load_config(cfg_path)\n self.game.init()\n if \"battle\" in cfg_path:\n self.available_actions = battle_button_comb()\n else:\n self.available_actions = normal_button_comb()\n self.action_num = len(self.available_actions)\n self.action_space = gym.spaces.Discrete(self.action_num)\n self.spec = gym.envs.registration.EnvSpec(\"vizdoom-v0\")\n self.count = 0\n\n def get_obs(self) -> None:\n state = self.game.get_state()\n if state is None:\n return\n obs = state.screen_buffer\n self.obs_buffer[:-1] = self.obs_buffer[1:]\n self.obs_buffer[-1] = cv2.resize(obs, (self.res[-1], self.res[-2]))\n\n def reset(\n self,\n seed: int | None = None,\n options: dict[str, Any] | None = None,\n ) -> tuple[NDArray[np.uint8], dict[str, Any]]:\n if self.save_lmp:\n self.game.new_episode(f\"lmps/episode_{self.count}.lmp\")\n else:\n self.game.new_episode()\n self.count += 1\n self.obs_buffer = np.zeros(self.res, dtype=np.uint8)\n self.get_obs()\n self.health = self.game.get_game_variable(vzd.GameVariable.HEALTH)\n self.killcount = self.game.get_game_variable(vzd.GameVariable.KILLCOUNT)\n self.ammo2 = self.game.get_game_variable(vzd.GameVariable.AMMO2)\n return self.obs_buffer, {\"TimeLimit.truncated\": False}\n\n def step(self, action: int) -> tuple[NDArray[np.uint8], float, bool, bool, dict[str, Any]]:\n self.game.make_action(self.available_actions[action], self.skip)\n reward = 0.0\n self.get_obs()\n health = self.game.get_game_variable(vzd.GameVariable.HEALTH)\n if self.health_setting or health > self.health: # positive health reward only for d1/d2\n reward += health - self.health\n self.health = health\n killcount = self.game.get_game_variable(vzd.GameVariable.KILLCOUNT)\n reward += 20 * (killcount - self.killcount)\n self.killcount = killcount\n ammo2 = self.game.get_game_variable(vzd.GameVariable.AMMO2)\n # if ammo2 > self.ammo2:\n reward += ammo2 - self.ammo2\n self.ammo2 = ammo2\n done = False\n info = {}\n if self.game.is_player_dead() or self.game.get_state() is None:\n done = True\n elif self.game.is_episode_finished():\n done = True\n info[\"TimeLimit.truncated\"] = True\n return (\n self.obs_buffer,\n reward,\n done,\n info.get(\"TimeLimit.truncated\", False),\n info,\n )\n\n def render(self) -> None:\n pass\n\n def close(self) -> None:\n self.game.close()\n\n\ndef make_vizdoom_env(\n task: str,\n frame_skip: int,\n res: tuple[int],\n save_lmp: bool = False,\n seed: int | None = None,\n num_training_envs: int = 10,\n num_test_envs: int = 10,\n) -> tuple[Env, ShmemVectorEnv, ShmemVectorEnv]:\n cpu_count = os.cpu_count()\n if cpu_count is not None:\n num_test_envs = min(cpu_count - 1, num_test_envs)\n if envpool is not None:\n task_id = \"\".join([i.capitalize() for i in task.split(\"_\")]) + \"-v1\"\n lmp_save_dir = \"lmps/\" if save_lmp else \"\"\n reward_config = {\n \"KILLCOUNT\": [20.0, -20.0],\n \"HEALTH\": [1.0, 0.0],\n \"AMMO2\": [1.0, -1.0],\n }\n if \"battle\" in task:\n reward_config[\"HEALTH\"] = [1.0, -1.0]\n env = training_envs = envpool.make_gymnasium(\n task_id,\n frame_skip=frame_skip,\n stack_num=res[0],\n seed=seed,\n num_envs=num_training_envs,\n reward_config=reward_config,\n use_combined_action=True,\n max_episode_steps=2625,\n use_inter_area_resize=False,\n )\n test_envs = envpool.make_gymnasium(\n task_id,\n frame_skip=frame_skip,\n stack_num=res[0],\n lmp_save_dir=lmp_save_dir,\n seed=seed,\n num_envs=num_test_envs,\n reward_config=reward_config,\n use_combined_action=True,\n max_episode_steps=2625,\n use_inter_area_resize=False,\n )\n else:\n cfg_path = f\"maps/{task}.cfg\"\n env = Env(cfg_path, frame_skip, res)\n training_envs = ShmemVectorEnv(\n [lambda: Env(cfg_path, frame_skip, res) for _ in range(num_training_envs)],\n )\n test_envs = ShmemVectorEnv(\n [lambda: Env(cfg_path, frame_skip, res, save_lmp) for _ in range(num_test_envs)],\n )\n training_envs.seed(seed)\n test_envs.seed(seed)\n return env, training_envs, test_envs\n\n\nif __name__ == \"__main__\":\n # env = Env(\"maps/D1_basic.cfg\", 4, (4, 84, 84))\n env = Env(\"maps/D3_battle.cfg\", 4, (4, 84, 84))\n print(env.available_actions)\n assert isinstance(env.action_space, gym.spaces.Discrete)\n action_num = env.action_space.n\n obs, _ = env.reset()\n if env.spec:\n print(env.spec.reward_threshold)\n print(obs.shape, action_num)\n for _ in range(4000):\n obs, rew, terminated, truncated, info = env.step(0)\n if terminated or truncated:\n env.reset()\n print(obs.shape, rew, terminated, truncated)\n cv2.imwrite(\"test.png\", obs.transpose(1, 2, 0)[..., :3])\n" }, { "path": "examples/vizdoom/maps/D1_basic.cfg", "content": "# Lines starting with # are treated as comments (or with whitespaces+#).\n# It doesn't matter if you use capital letters or not.\n# It doesn't matter if you use underscore or camel notation for keys, e.g. episode_timeout is the same as episodeTimeout.\n\ndoom_scenario_path = D1_basic.wad\ndoom_map = map01\n\n# Rewards\n\n# Each step is good for you!\nliving_reward = 0\n# And death is not!\ndeath_penalty = 0\n\n# Rendering options\nscreen_resolution = RES_160X120\nscreen_format = GRAY8\nrender_hud = false\nrender_crosshair = false\nrender_weapon = false\nrender_decals = false\nrender_particles = false\nwindow_visible = false\n\n# make episodes finish after 10500 actions (tics)\nepisode_timeout = 10500\n\n# Available buttons\navailable_buttons =\n{\n MOVE_FORWARD\n TURN_LEFT\n TURN_RIGHT\n}\n\n# Game variables that will be in the state\navailable_game_variables = { HEALTH }\n\nmode = PLAYER\n" }, { "path": "examples/vizdoom/maps/D2_navigation.cfg", "content": "# Lines starting with # are treated as comments (or with whitespaces+#).\n# It doesn't matter if you use capital letters or not.\n# It doesn't matter if you use underscore or camel notation for keys, e.g. episode_timeout is the same as episodeTimeout.\n\ndoom_scenario_path = D2_navigation.wad\ndoom_map = map01\n\n# Rewards\n\n# Each step is good for you!\nliving_reward = 0\n# And death is not!\ndeath_penalty = 0\n\n# Rendering options\nscreen_resolution = RES_160X120\nscreen_format = GRAY8\nrender_hud = false\nrender_crosshair = false\nrender_weapon = false\nrender_decals = false\nrender_particles = false\nwindow_visible = false\n\n# make episodes finish after 10500 actions (tics)\nepisode_timeout = 10500\n\n# Available buttons\navailable_buttons =\n{\n MOVE_FORWARD\n TURN_LEFT\n TURN_RIGHT\n}\n\n# Game variables that will be in the state\navailable_game_variables = { HEALTH }\n\nmode = PLAYER\n" }, { "path": "examples/vizdoom/maps/D3_battle.cfg", "content": "# Lines starting with # are treated as comments (or with whitespaces+#).\n# It doesn't matter if you use capital letters or not.\n# It doesn't matter if you use underscore or camel notation for keys, e.g. episode_timeout is the same as episodeTimeout.\n\ndoom_scenario_path = D3_battle.wad\ndoom_map = map01\n\n# Rewards\n\nliving_reward = 0\ndeath_penalty = 100\n\n# Rendering options\nscreen_resolution = RES_160X120\nscreen_format = GRAY8\nrender_hud = false\nrender_crosshair = true\nrender_weapon = true\nrender_decals = false\nrender_particles = false\nwindow_visible = false\n\n# make episodes finish after 10500 actions (tics)\nepisode_timeout = 10500\n\n# Available buttons\navailable_buttons =\n{\n MOVE_FORWARD\n MOVE_BACKWARD\n MOVE_LEFT\n MOVE_RIGHT\n TURN_LEFT\n TURN_RIGHT\n ATTACK\n SPEED\n}\n\n# Game variables that will be in the state\navailable_game_variables =\n{\n KILLCOUNT\n AMMO2\n HEALTH\n}\n\nmode = PLAYER\ndoom_skill = 2\n" }, { "path": "examples/vizdoom/maps/D4_battle2.cfg", "content": "# Lines starting with # are treated as comments (or with whitespaces+#).\n# It doesn't matter if you use capital letters or not.\n# It doesn't matter if you use underscore or camel notation for keys, e.g. episode_timeout is the same as episodeTimeout.\n\ndoom_scenario_path = D4_battle2.wad\ndoom_map = map01\n\n# Rewards\n\nliving_reward = 0\ndeath_penalty = 100\n\n# Rendering options\nscreen_resolution = RES_160X120\nscreen_format = GRAY8\nrender_hud = false\nrender_crosshair = true\nrender_weapon = true\nrender_decals = false\nrender_particles = false\nwindow_visible = false\n\n# make episodes finish after 10500 actions (tics)\nepisode_timeout = 10500\n\n# Available buttons\navailable_buttons =\n{ \n MOVE_FORWARD\n MOVE_BACKWARD\n MOVE_LEFT\n MOVE_RIGHT\n TURN_LEFT\n TURN_RIGHT\n ATTACK\n SPEED\n}\n\n# Game variables that will be in the state\navailable_game_variables =\n{\n KILLCOUNT\n AMMO2\n HEALTH\n}\n\nmode = PLAYER\ndoom_skill = 2\n" }, { "path": "examples/vizdoom/maps/README.md", "content": "D1-D4 maps are from https://github.com/intel-isl/DirectFuturePrediction/\n\nMore maps and cfgs: https://github.com/mwydmuch/ViZDoom/tree/master/scenarios\n" }, { "path": "examples/vizdoom/maps/spectator.py", "content": "#!/usr/bin/env python3\n\n#####################################################################\n# This script presents SPECTATOR mode. In SPECTATOR mode you play and\n# your agent can learn from it.\n# Configuration is loaded from \"../../scenarios/.cfg\" file.\n#\n# To see the scenario description go to \"../../scenarios/README.md\"\n#####################################################################\n\n\nfrom argparse import ArgumentParser\nfrom time import sleep\n\nimport vizdoom as vzd\n\n# import cv2\n\nif __name__ == \"__main__\":\n parser = ArgumentParser(\"ViZDoom example showing how to use SPECTATOR mode.\")\n parser.add_argument(\"-c\", type=str, dest=\"config\", default=\"D3_battle.cfg\")\n parser.add_argument(\"-w\", type=str, dest=\"wad_file\", default=\"D3_battle.wad\")\n args = parser.parse_args()\n game = vzd.DoomGame()\n\n # Choose scenario config file you wish to watch.\n # Don't load two configs cause the second will overrite the first one.\n # Multiple config files are ok but combining these ones doesn't make much sense.\n\n game.load_config(args.config)\n game.set_doom_scenario_path(args.wad_file)\n # Enables freelook in engine\n game.add_game_args(\"+freelook 1\")\n\n game.set_screen_resolution(vzd.ScreenResolution.RES_640X480)\n\n # Enables spectator mode, so you can play.\n # Sounds strange but it is the agent who is supposed to watch not you.\n game.set_window_visible(True)\n game.set_mode(vzd.Mode.SPECTATOR)\n\n game.init()\n\n episodes = 1\n\n for i in range(episodes):\n print(\"Episode #\" + str(i + 1))\n\n game.new_episode()\n while not game.is_episode_finished():\n state = game.get_state()\n print(state.screen_buffer.dtype, state.screen_buffer.shape)\n # cv2.imwrite(f'imgs/{state.number}.png', state.screen_buffer)\n\n # game.make_action([0, 0, 0])\n game.advance_action()\n last_action = game.get_last_action()\n reward = game.get_last_reward()\n\n print(\"State #\" + str(state.number))\n print(\"Game variables: \", state.game_variables)\n print(\"Action:\", last_action)\n print(\"Reward:\", reward)\n print(\"=====================\")\n\n print(\"Episode finished!\")\n print(\"Total reward:\", game.get_total_reward())\n print(\"************************\")\n sleep(2.0)\n\n game.close()\n" }, { "path": "examples/vizdoom/replay.py", "content": "# import cv2\nimport os\nimport sys\nimport time\n\nimport tqdm\nimport vizdoom as vzd\n\n\ndef main(\n cfg_path: str = os.path.join(\"maps\", \"D3_battle.cfg\"),\n lmp_path: str = os.path.join(\"test.lmp\"),\n) -> None:\n game = vzd.DoomGame()\n game.load_config(cfg_path)\n game.set_screen_format(vzd.ScreenFormat.CRCGCB)\n game.set_screen_resolution(vzd.ScreenResolution.RES_1024X576)\n game.set_window_visible(True)\n game.set_render_hud(True)\n game.init()\n game.replay_episode(lmp_path)\n\n killcount = 0\n with tqdm.trange(10500) as tq:\n while not game.is_episode_finished():\n game.advance_action()\n state = game.get_state()\n if state is None:\n break\n killcount = game.get_game_variable(vzd.GameVariable.KILLCOUNT)\n time.sleep(1 / 35)\n # cv2.imwrite(f\"imgs/{tq.n}.png\",\n # state.screen_buffer.transpose(1, 2, 0)[..., ::-1])\n tq.update(1)\n game.close()\n print(\"killcount:\", killcount)\n\n\nif __name__ == \"__main__\":\n main(*sys.argv[-2:])\n" }, { "path": "examples/vizdoom/vizdoom_c51.py", "content": "import argparse\nimport datetime\nimport os\nimport pprint\nimport sys\n\nimport numpy as np\nimport torch\nfrom env import make_vizdoom_env\n\nfrom tianshou.algorithm import C51\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.c51 import C51Policy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env.atari.atari_network import C51Net\nfrom tianshou.highlevel.logger import LoggerFactoryDefault\nfrom tianshou.trainer import OffPolicyTrainerParams\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"D1_basic\")\n parser.add_argument(\"--seed\", type=int, default=0)\n parser.add_argument(\"--eps_test\", type=float, default=0.005)\n parser.add_argument(\"--eps_train\", type=float, default=1.0)\n parser.add_argument(\"--eps_train_final\", type=float, default=0.05)\n parser.add_argument(\"--buffer_size\", type=int, default=2000000)\n parser.add_argument(\"--lr\", type=float, default=0.0001)\n parser.add_argument(\"--gamma\", type=float, default=0.99)\n parser.add_argument(\"--num_atoms\", type=int, default=51)\n parser.add_argument(\"--v_min\", type=float, default=-10.0)\n parser.add_argument(\"--v_max\", type=float, default=10.0)\n parser.add_argument(\"--n_step\", type=int, default=3)\n parser.add_argument(\"--target_update_freq\", type=int, default=500)\n parser.add_argument(\"--epoch\", type=int, default=300)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=100000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=10)\n parser.add_argument(\"--update_per_step\", type=float, default=0.1)\n parser.add_argument(\"--batch_size\", type=int, default=64)\n parser.add_argument(\"--num_training_envs\", type=int, default=10)\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n parser.add_argument(\"--frames_stack\", type=int, default=4)\n parser.add_argument(\"--skip_num\", type=int, default=4)\n parser.add_argument(\"--resume_path\", type=str, default=None)\n parser.add_argument(\"--resume_id\", type=str, default=None)\n parser.add_argument(\n \"--logger\",\n type=str,\n default=\"tensorboard\",\n choices=[\"tensorboard\", \"wandb\"],\n )\n parser.add_argument(\"--wandb_project\", type=str, default=\"vizdoom.benchmark\")\n parser.add_argument(\n \"--watch\",\n default=False,\n action=\"store_true\",\n help=\"watch the play of pre-trained policy only\",\n )\n parser.add_argument(\n \"--save_lmp\",\n default=False,\n action=\"store_true\",\n help=\"save lmp file for replay whole episode\",\n )\n parser.add_argument(\"--save_buffer_name\", type=str, default=None)\n return parser.parse_args()\n\n\ndef test_c51(args: argparse.Namespace = get_args()) -> None:\n # make environments\n env, training_envs, test_envs = make_vizdoom_env(\n args.task,\n args.skip_num,\n (args.frames_stack, 84, 84),\n args.save_lmp,\n args.seed,\n args.num_training_envs,\n args.num_test_envs,\n )\n args.state_shape = env.observation_space.shape\n args.action_shape = env.action_space.shape or env.action_space.n\n # should be N_FRAMES x H x W\n print(\"Observations shape:\", args.state_shape)\n print(\"Actions shape:\", args.action_shape)\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n # define model\n c, h, w = args.state_shape\n net = C51Net(c=c, h=h, w=w, action_shape=args.action_shape, num_atoms=args.num_atoms)\n optim = AdamOptimizerFactory(lr=args.lr)\n # define policy and algorithm\n policy = C51Policy(\n model=net,\n action_space=env.action_space,\n num_atoms=args.num_atoms,\n v_min=args.v_min,\n v_max=args.v_max,\n eps_training=args.eps_train,\n eps_inference=args.eps_test,\n )\n algorithm: C51 = C51(\n policy=policy,\n optim=optim,\n gamma=args.gamma,\n n_step_return_horizon=args.n_step,\n target_update_freq=args.target_update_freq,\n ).to(args.device)\n # load a previous policy\n if args.resume_path:\n algorithm.load_state_dict(torch.load(args.resume_path, map_location=args.device))\n print(\"Loaded agent from: \", args.resume_path)\n # replay buffer: `save_last_obs` and `stack_num` can be removed together\n # when you have enough RAM\n buffer = VectorReplayBuffer(\n args.buffer_size,\n buffer_num=len(training_envs),\n ignore_obs_next=True,\n save_only_last_obs=True,\n stack_num=args.frames_stack,\n )\n # collector\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buffer, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n args.algo_name = \"c51\"\n log_name = os.path.join(args.task, args.algo_name, str(args.seed), now)\n log_path = os.path.join(args.logdir, log_name)\n\n # logger\n logger_factory = LoggerFactoryDefault()\n if args.logger == \"wandb\":\n logger_factory.logger_type = \"wandb\"\n logger_factory.wandb_project = args.wandb_project\n else:\n logger_factory.logger_type = \"tensorboard\"\n\n logger = logger_factory.create_logger(\n log_dir=log_path,\n experiment_name=log_name,\n run_id=args.resume_id,\n config_dict=vars(args),\n )\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n if env.spec.reward_threshold:\n return mean_rewards >= env.spec.reward_threshold\n return False\n\n def train_fn(epoch: int, env_step: int) -> None:\n # nature DQN setting, linear decay in the first 1M steps\n if env_step <= 1e6:\n eps = args.eps_train - env_step / 1e6 * (args.eps_train - args.eps_train_final)\n else:\n eps = args.eps_train_final\n policy.set_eps_training(eps)\n if env_step % 1000 == 0:\n logger.write(\"train/env_step\", env_step, {\"train/eps\": eps})\n\n # watch agent's performance\n def watch() -> None:\n print(\"Setup test envs ...\")\n test_envs.seed(args.seed)\n if args.save_buffer_name:\n print(f\"Generate buffer with size {args.buffer_size}\")\n buffer = VectorReplayBuffer(\n args.buffer_size,\n buffer_num=len(test_envs),\n ignore_obs_next=True,\n save_only_last_obs=True,\n stack_num=args.frames_stack,\n )\n collector = Collector[CollectStats](\n algorithm, test_envs, buffer, exploration_noise=True\n )\n result = collector.collect(n_step=args.buffer_size, reset_before_collect=True)\n print(f\"Save buffer into {args.save_buffer_name}\")\n # Unfortunately, pickle will cause oom with 1M buffer size\n buffer.save_hdf5(args.save_buffer_name)\n else:\n print(\"Testing agent ...\")\n test_collector.reset()\n result = test_collector.collect(n_episode=args.num_test_envs, render=args.render)\n result.pprint_asdict()\n\n if args.watch:\n watch()\n sys.exit(0)\n\n # test training_collector and start filling replay buffer\n training_collector.reset()\n training_collector.collect(n_step=args.batch_size * args.num_training_envs)\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n training_fn=train_fn,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n update_step_num_gradient_steps_per_sample=args.update_per_step,\n test_in_training=False,\n )\n )\n\n pprint.pprint(result)\n watch()\n\n\nif __name__ == \"__main__\":\n test_c51(get_args())\n" }, { "path": "examples/vizdoom/vizdoom_ppo.py", "content": "import argparse\nimport datetime\nimport os\nimport pprint\nimport sys\n\nimport numpy as np\nimport torch\nfrom env import make_vizdoom_env\nfrom torch.distributions import Categorical\n\nfrom tianshou.algorithm import PPO\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelbased.icm import ICMOnPolicyWrapper\nfrom tianshou.algorithm.modelfree.reinforce import ProbabilisticActorPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory, LRSchedulerFactoryLinear\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env.atari.atari_network import DQNet\nfrom tianshou.highlevel.logger import LoggerFactoryDefault\nfrom tianshou.trainer import OnPolicyTrainerParams\nfrom tianshou.utils.net.discrete import (\n DiscreteActor,\n DiscreteCritic,\n IntrinsicCuriosityModule,\n)\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"D1_basic\")\n parser.add_argument(\"--seed\", type=int, default=0)\n parser.add_argument(\"--buffer_size\", type=int, default=100000)\n parser.add_argument(\"--lr\", type=float, default=0.00002)\n parser.add_argument(\"--gamma\", type=float, default=0.99)\n parser.add_argument(\"--epoch\", type=int, default=300)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=100000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=1000)\n parser.add_argument(\"--update_step_num_repetitions\", type=int, default=4)\n parser.add_argument(\"--batch_size\", type=int, default=256)\n parser.add_argument(\"--hidden_size\", type=int, default=512)\n parser.add_argument(\"--num_training_envs\", type=int, default=10)\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\n parser.add_argument(\"--return_scaling\", type=int, default=False)\n parser.add_argument(\"--vf_coef\", type=float, default=0.5)\n parser.add_argument(\"--ent_coef\", type=float, default=0.01)\n parser.add_argument(\"--gae_lambda\", type=float, default=0.95)\n parser.add_argument(\"--lr_decay\", type=int, default=True)\n parser.add_argument(\"--max_grad_norm\", type=float, default=0.5)\n parser.add_argument(\"--eps_clip\", type=float, default=0.2)\n parser.add_argument(\"--dual_clip\", type=float, default=None)\n parser.add_argument(\"--value_clip\", type=int, default=0)\n parser.add_argument(\"--advantage_normalization\", type=int, default=1)\n parser.add_argument(\"--recompute_adv\", type=int, default=0)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n parser.add_argument(\"--frames_stack\", type=int, default=4)\n parser.add_argument(\"--skip_num\", type=int, default=4)\n parser.add_argument(\"--resume_path\", type=str, default=None)\n parser.add_argument(\"--resume_id\", type=str, default=None)\n parser.add_argument(\n \"--logger\",\n type=str,\n default=\"tensorboard\",\n choices=[\"tensorboard\", \"wandb\"],\n )\n parser.add_argument(\"--wandb_project\", type=str, default=\"vizdoom.benchmark\")\n parser.add_argument(\n \"--watch\",\n default=False,\n action=\"store_true\",\n help=\"watch the play of pre-trained policy only\",\n )\n parser.add_argument(\n \"--save_lmp\",\n default=False,\n action=\"store_true\",\n help=\"save lmp file for replay whole episode\",\n )\n parser.add_argument(\"--save_buffer_name\", type=str, default=None)\n parser.add_argument(\n \"--icm_lr_scale\",\n type=float,\n default=0.0,\n help=\"use intrinsic curiosity module with this lr scale\",\n )\n parser.add_argument(\n \"--icm_reward_scale\",\n type=float,\n default=0.01,\n help=\"scaling factor for intrinsic curiosity reward\",\n )\n parser.add_argument(\n \"--icm_forward_loss_weight\",\n type=float,\n default=0.2,\n help=\"weight for the forward model loss in ICM\",\n )\n return parser.parse_args()\n\n\ndef test_ppo(args: argparse.Namespace = get_args()) -> None:\n # make environments\n env, training_envs, test_envs = make_vizdoom_env(\n args.task,\n args.skip_num,\n (args.frames_stack, 84, 84),\n args.save_lmp,\n args.seed,\n args.num_training_envs,\n args.num_test_envs,\n )\n args.state_shape = env.observation_space.shape\n args.action_shape = env.action_space.n\n # should be N_FRAMES x H x W\n print(\"Observations shape:\", args.state_shape)\n print(\"Actions shape:\", args.action_shape)\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n # define model\n c, h, w = args.state_shape\n net = DQNet(\n c=c,\n h=h,\n w=w,\n action_shape=args.action_shape,\n features_only=True,\n output_dim_added_layer=args.hidden_size,\n )\n actor = DiscreteActor(preprocess_net=net, action_shape=args.action_shape, softmax_output=False)\n critic = DiscreteCritic(preprocess_net=net)\n optim = AdamOptimizerFactory(lr=args.lr)\n\n if args.lr_decay:\n optim.with_lr_scheduler_factory(\n LRSchedulerFactoryLinear(\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n )\n )\n\n def dist(logits: torch.Tensor) -> Categorical:\n return Categorical(logits=logits)\n\n # define policy and algorithm\n policy = ProbabilisticActorPolicy(\n actor=actor,\n dist_fn=dist,\n action_scaling=False,\n action_space=env.action_space,\n )\n algorithm: PPO | ICMOnPolicyWrapper\n algorithm = PPO(\n policy=policy,\n critic=critic,\n optim=optim,\n gamma=args.gamma,\n gae_lambda=args.gae_lambda,\n max_grad_norm=args.max_grad_norm,\n vf_coef=args.vf_coef,\n ent_coef=args.ent_coef,\n return_scaling=args.return_scaling,\n eps_clip=args.eps_clip,\n value_clip=args.value_clip,\n dual_clip=args.dual_clip,\n advantage_normalization=args.advantage_normalization,\n recompute_advantage=args.recompute_adv,\n ).to(args.device)\n if args.icm_lr_scale > 0:\n c, h, w = args.state_shape\n feature_net = DQNet(\n c=c,\n h=h,\n w=w,\n action_shape=args.action_shape,\n features_only=True,\n output_dim_added_layer=args.hidden_size,\n )\n action_dim = np.prod(args.action_shape)\n feature_dim = feature_net.output_dim\n icm_net = IntrinsicCuriosityModule(\n feature_net=feature_net.net,\n feature_dim=feature_dim,\n action_dim=action_dim,\n )\n icm_optim = AdamOptimizerFactory(lr=args.lr)\n algorithm = ICMOnPolicyWrapper(\n wrapped_algorithm=algorithm,\n model=icm_net,\n optim=icm_optim,\n lr_scale=args.icm_lr_scale,\n reward_scale=args.icm_reward_scale,\n forward_loss_weight=args.icm_forward_loss_weight,\n ).to(args.device)\n # load a previous policy\n if args.resume_path:\n algorithm.load_state_dict(torch.load(args.resume_path, map_location=args.device))\n print(\"Loaded agent from: \", args.resume_path)\n # replay buffer: `save_last_obs` and `stack_num` can be removed together\n # when you have enough RAM\n buffer = VectorReplayBuffer(\n args.buffer_size,\n buffer_num=len(training_envs),\n ignore_obs_next=True,\n save_only_last_obs=True,\n stack_num=args.frames_stack,\n )\n # collector\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buffer, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n\n # log\n now = datetime.datetime.now().strftime(\"%y%m%d-%H%M%S\")\n args.algo_name = \"ppo_icm\" if args.icm_lr_scale > 0 else \"ppo\"\n log_name = os.path.join(args.task, args.algo_name, str(args.seed), now)\n log_path = os.path.join(args.logdir, log_name)\n\n # logger\n logger_factory = LoggerFactoryDefault()\n if args.logger == \"wandb\":\n logger_factory.logger_type = \"wandb\"\n logger_factory.wandb_project = args.wandb_project\n else:\n logger_factory.logger_type = \"tensorboard\"\n\n logger = logger_factory.create_logger(\n log_dir=log_path,\n experiment_name=log_name,\n run_id=args.resume_id,\n config_dict=vars(args),\n )\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n if env.spec.reward_threshold:\n return mean_rewards >= env.spec.reward_threshold\n return False\n\n # watch agent's performance\n def watch() -> None:\n print(\"Setup test envs ...\")\n test_envs.seed(args.seed)\n if args.save_buffer_name:\n print(f\"Generate buffer with size {args.buffer_size}\")\n buffer = VectorReplayBuffer(\n args.buffer_size,\n buffer_num=len(test_envs),\n ignore_obs_next=True,\n save_only_last_obs=True,\n stack_num=args.frames_stack,\n )\n collector = Collector[CollectStats](\n algorithm, test_envs, buffer, exploration_noise=True\n )\n result = collector.collect(n_step=args.buffer_size, reset_before_collect=True)\n print(f\"Save buffer into {args.save_buffer_name}\")\n # Unfortunately, pickle will cause oom with 1M buffer size\n buffer.save_hdf5(args.save_buffer_name)\n else:\n print(\"Testing agent ...\")\n test_collector.reset()\n result = test_collector.collect(n_episode=args.num_test_envs, render=args.render)\n result.pprint_asdict()\n\n if args.watch:\n watch()\n sys.exit(0)\n\n # test training_collector and start filling replay buffer\n training_collector.reset()\n training_collector.collect(n_step=args.batch_size * args.num_training_envs)\n\n # train\n result = algorithm.run_training(\n OnPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n update_step_num_repetitions=args.update_step_num_repetitions,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=False,\n )\n )\n\n pprint.pprint(result)\n watch()\n\n\nif __name__ == \"__main__\":\n test_ppo(get_args())\n" }, { "path": "pyproject.toml", "content": "[build-system]\nrequires = [\"poetry-core>=1.0.0\"]\nbuild-backend = \"poetry.core.masonry.api\"\n\n[tool.poetry]\nname = \"tianshou\"\nversion = \"2.0.0\"\ndescription = \"A Library for Deep Reinforcement Learning\"\nauthors = [\"TSAIL \"]\nlicense = \"MIT\"\nreadme = \"README.md\"\nhomepage = \"https://github.com/thu-ml/tianshou\"\nclassifiers = [\n # 3 - Alpha\n # 4 - Beta\n # 5 - Production/Stable\n \"Development Status :: 4 - Beta\",\n \"Intended Audience :: Science/Research\",\n \"Topic :: Scientific/Engineering :: Artificial Intelligence\",\n \"Topic :: Software Development :: Libraries :: Python Modules\",\n \"License :: OSI Approved :: MIT License\",\n \"Programming Language :: Python :: 3.11\",\n]\nexclude = [\"test/*\", \"examples/*\", \"docs/*\"]\n\n[tool.poetry.dependencies]\npython = \"^3.11\"\ndeepdiff = \"^7.0.1\"\ngymnasium = \">=0.28.0\"\nh5py = \"^3.9.0\"\nmatplotlib = \">=3.0.0\"\nnumba = \">=0.60.0\"\nnumpy = \">=1.24.4\"\noverrides = \"^7.4.0\"\npackaging = \"*\"\npandas = \">=2.0.0\"\npettingzoo = \"^1.22\"\nsensai-utils = \">=1.6.0\"\ntensorboard = \"^2.5.0\"\n# Torch 2.0.1 causes problems, see https://github.com/pytorch/pytorch/issues/100974\ntorch = \"^2.0.0, !=2.0.1, !=2.1.0\"\ntqdm = \"*\"\nvirtualenv = [\n # special sauce b/c of a flaky bug in poetry on windows\n # see https://github.com/python-poetry/poetry/issues/7611#issuecomment-1466478926\n { version = \"^20.4.3,!=20.4.5,!=20.4.6\" },\n { version = \"<20.16.4\", markers = \"sys_platform == 'win32'\" },\n]\n\n\n# Atari, box2d, classic-control, and mujoco environments are all optional dependencies of gymnasium.\n# Unfortunately, we cannot have extras relying on (multiple) optionals of other packages due to a poetry issue (see e.g.\n# https://github.com/python-poetry/poetry/issues/7911) and therefore have to maintain our own list of dependencies.\n# This requires attention and monitoring of gymnasium's dependencies and their version numbers!\nale-py = { version = \"~=0.8.1\", optional = true }\n# We have to pin arch explicitly, the lowest version pinned by rliable can't be installed on python 3.11\n# and poetry doesn't seem to be able to resolve this properly.\narch = { version = \">=5.4.0\", optional = true }\nautorom = { version = \"~=0.4.2\", extras = [\"accept-rom-license\"], optional = true }\nbox2d_py = { version = \"2.3.5\", optional = true }\ncython = { version = \">=0.27.2\", optional = true }\ndocstring-parser = { version = \"^0.15\", optional = true }\nenvpool = { version = \"^0.8.2\", optional = true, markers = \"sys_platform != 'darwin'\"}\ngymnasium-robotics = { version = \"*\", optional = true }\nimageio = { version = \">=2.14.1\", optional = true }\njoblib = { version = \"*\", optional = true }\njsonargparse = {version = \"^4.24.1\", optional = true}\n# we need <3 b/c of https://github.com/Farama-Foundation/Gymnasium/issues/749\nmujoco = { version = \">=2.1.5, <3\", optional = true }\nopencv_python = { version = \"*\", optional = true }\npybullet = { version = \"*\", optional = true }\npygame = { version = \">=2.1.3\", optional = true }\nrliable = {optional = true, version=\"1.2.0\"}\nscipy = { version = \"*\", optional = true }\nshimmy = { version = \">=0.1.0,<1.0\", optional = true }\nswig = { version = \"4.*\", optional = true }\nvizdoom = { version = \"*\", optional = true }\n[tool.poetry.extras]\nargparse = [\"docstring-parser\", \"jsonargparse\"]\natari = [\"ale-py\", \"autorom\", \"opencv-python\", \"shimmy\"]\nbox2d = [\"box2d-py\", \"pygame\", \"swig\"]\nclassic_control = [\"pygame\"]\nmujoco = [\"mujoco\", \"imageio\"]\npybullet = [\"pybullet\"]\nenvpool = [\"envpool\"]\nrobotics = [\"gymnasium-robotics\"]\nvizdoom = [\"vizdoom\"]\neval = [\"rliable\", \"arch\", \"joblib\", \"scipy\", \"jsonargparse\", \"docstring-parser\"]\n\n\n[tool.poetry.group.dev]\noptional = true\n\n[tool.poetry.group.dev.dependencies]\ndocutils = \"0.20.1\"\njinja2 = \"*\"\njupyter = \"^1.0.0\"\njupyter-book = \"^1.0.0\"\nmypy = \"^1.4.1\"\nnbqa = \"^1.7.1\"\nnbstripout = \"^0.6.1\"\n# networkx is used in a test\nnetworkx = \"*\"\npoethepoet = \"^0.20.0\"\npre-commit = \"^3.3.3\"\npygame = \"^2.1.0\"\npymunk = \"^6.2.1\"\npytest = \"*\"\npytest-cov = \"*\"\n# Ray currently causes issues when installed on windows server 2022 in CI\n# If users want to use ray, they should install it manually.\nray = { version = \">=2.10, <3\", markers = \"sys_platform != 'win32'\" }\nruff = \"0.14.1\"\nscipy = \"*\"\nsphinx = \"^7\"\nsphinx-book-theme = \"^1.0.1\"\nsphinx-comments = \"^0.0.3\"\nsphinx-copybutton = \"^0.5.2\"\nsphinx-jupyterbook-latex = \"^1.0.0\"\nsphinx-togglebutton = \"^0.3.2\"\nsphinx-toolbox = \"^3.5.0\"\nsphinxcontrib-bibtex = \"*\"\nsphinxcontrib-spelling = \"^8.0.0\"\nsphinxcontrib-mermaid = \"^1.0.0\"\ntypes-requests = \"^2.31.0.20240311\"\ntypes-tabulate = \"^0.9.0.20240106\"\n# this is needed for wandb only (undisclosed dependency)\ntyping-extensions = \">=4.10\"\nwandb = \">=0.16.0\"\n\n[tool.mypy]\nallow_redefinition = true\ncheck_untyped_defs = true\ndisallow_incomplete_defs = true\ndisallow_untyped_defs = true\nignore_missing_imports = true\nno_implicit_optional = true\npretty = true\nshow_error_codes = true\nshow_error_context = true\nshow_traceback = true\nstrict_equality = true\nstrict_optional = true\nwarn_no_return = true\nwarn_redundant_casts = true\nwarn_unreachable = true\nwarn_unused_configs = true\nwarn_unused_ignores = true\nexclude = \"^build/|^docs/\"\n\n[tool.doc8]\nmax-line-length = 1000\n\n\n[tool.nbqa.exclude]\nruff = \"\\\\.jupyter_cache|jupyter_execute\"\nmypy = \"\\\\.jupyter_cache|jupyter_execute\"\n\n[tool.ruff]\ntarget-version = \"py311\"\nline-length = 100\n\n[tool.ruff.lint]\nselect = [\n \"ASYNC\", \"B\", \"C4\", \"C90\", \"COM\", \"D\", \"DTZ\", \"E\", \"F\", \"FLY\", \"G\", \"I\", \"ISC\", \"PIE\", \"PLC\", \"PLE\", \"PLW\", \"RET\", \"RUF\", \"RSE\", \"SIM\", \"TID\", \"UP\", \"W\", \"YTT\",\n]\nignore = [\n \"RUF003\", # custom (greek) letters\n \"SIM118\", # Needed b/c iter(batch) != iter(batch.keys()). See https://github.com/thu-ml/tianshou/issues/922\n \"E501\", # line too long. ruff does a good enough job\n \"E741\", # variable names like \"l\". this isn't a huge problem\n \"B008\", # do not perform function calls in argument defaults. we do this sometimes\n \"B011\", # assert false. we don't use python -O\n \"B028\", # we don't need explicit stacklevel for warnings\n \"D100\", \"D101\", \"D102\", \"D104\", \"D105\", \"D107\", \"D203\", \"D213\", \"D401\", \"D402\", # docstring stuff\n \"DTZ005\", # we don't need that\n # remaining rules from https://github.com/psf/black/blob/main/.flake8 (except W503)\n # this is a simplified version of config, making vscode plugin happy\n \"E402\", \"E501\", \"E701\", \"E731\", \"C408\", \"E203\",\n # Logging statement uses f-string warning\n \"G004\",\n # Unnecessary `elif` after `return` statement\n \"RET505\",\n \"D106\", # undocumented public nested class\n \"D205\", # blank line after summary (prevents summary-only docstrings, which makes no sense)\n \"D212\", # no blank line after \"\"\". This clashes with sphinx for multiline descriptions of :param: that start directly after \"\"\"\n \"PLW2901\", # overwrite vars in loop\n \"B027\", # empty and non-abstract method in abstract class\n \"D404\", # It's fine to start with \"This\" in docstrings\n \"D407\", \"D408\", \"D409\", # Ruff rules for underlines under 'Example:' and so clash with Sphinx\n \"COM812\", # missing trailing comma: With this enabled, re-application of \"poe format\" chain can cause additional commas and subsequent reformatting\n \"B023\", # forbids function using loop variable without explicit binding\n \"RUF059\", # unused name after unpacking\n \"RUF005\", # concatenation\n \"PLC0415\", # local imports\n \"SIM108\", # if else is fine instead of ternary\n \"PLW1641\", # weird thing requiring __hash__ for Protocol\n \"PLC0206\", # extracting value from dictionary without calling `.items()`\n \"SIM103\", # forces returning of conditions instead of booleans\n \"E721\", # forbids use of equality for type checks\n \"RET504\", # sometimes we want to assign before return (e.g. for debugging)\n]\nunfixable = [\n \"F841\", # unused variable. ruff keeps the call, but mostly we want to get rid of it all\n \"F601\", # automatic fix might obscure issue\n \"F602\", # automatic fix might obscure issue\n \"B018\", # automatic fix might obscure issue\n]\nextend-fixable = [\n \"F401\", # unused import\n \"B905\", # bugbear\n]\n\n[tool.ruff.lint.mccabe]\nmax-complexity = 20\n\n[tool.ruff.lint.per-file-ignores]\n\"test/**\" = [\"D103\"]\n\"docs/**\" = [\"D103\"]\n\"examples/**\" = [\"D103\"]\n\"__init__.py\" = [\"F401\"] # do not remove \"unused\" imports (F401) from __init__.py files\n\n[tool.poetry-sort]\nmove-optionals-to-bottom = true\n\n[tool.poe.env]\nPYDEVD_DISABLE_FILE_VALIDATION=\"1\"\n# keep relevant parts in sync with pre-commit\n[tool.poe.tasks] # https://github.com/nat-n/poethepoet\ntest = \"pytest test\"\ntest-nocov = \"pytest -p no:cov test\"\ntest-reduced = \"pytest test/base test/continuous --cov=tianshou --durations=0 -v --color=yes\"\n_ruff_fix = \"ruff check --fix .\"\n_ruff_fix_check = \"ruff check .\"\n_ruff_format = \"ruff format .\"\n_ruff_format_check = \"ruff format --check .\"\nlint = [\"_ruff_format_check\", \"_ruff_fix_check\"]\nclean-nbs = \"python docs/nbstripout.py\"\nformat = [\"_ruff_fix\", \"_ruff_format\"]\n_autogen_rst = \"python docs/autogen_rst.py\"\n_sphinx_build = \"sphinx-build -b html docs docs/_build -W --keep-going\"\n_jb_generate_toc = \"python docs/create_toc.py\"\n_jb_generate_config = \"jupyter-book config sphinx docs/\"\ndoc-clean = \"rm -rf docs/_build docs/03_api\"\ndoc-generate-files = [\"_autogen_rst\", \"_jb_generate_toc\", \"_jb_generate_config\"]\ndoc-build = [\"doc-clean\", \"doc-generate-files\", \"_sphinx_build\"]\n_mypy = \"mypy tianshou test examples\"\n_mypy_nb = \"nbqa mypy docs\"\ntype-check = [\"_mypy\"]\n" }, { "path": "test/__init__.py", "content": "" }, { "path": "test/base/__init__.py", "content": "" }, { "path": "test/base/env.py", "content": "import random\nimport time\nfrom copy import deepcopy\nfrom typing import Any, Literal\n\nimport gymnasium as gym\nimport networkx as nx\nimport numpy as np\nfrom gymnasium.spaces import Box, Dict, Discrete, MultiDiscrete, Space, Tuple\n\n\nclass MoveToRightEnv(gym.Env):\n \"\"\"A task for \"going right\". The task is to go right ``size`` steps.\n\n The observation is the current index, and the action is to go left or right.\n Action 0 is to go left, and action 1 is to go right.\n Taking action 0 at index 0 will keep the index at 0.\n Arriving at index ``size`` means the task is done.\n In the current implementation, stepping after the task is done is possible, which will\n lead the index to be larger than ``size``.\n\n Index 0 is the starting point. If reset is called with default options, the index will\n be reset to 0.\n \"\"\"\n\n def __init__(\n self,\n size: int,\n sleep: float = 0.0,\n dict_state: bool = False,\n recurse_state: bool = False,\n ma_rew: int = 0,\n multidiscrete_action: bool = False,\n random_sleep: bool = False,\n array_state: bool = False,\n ) -> None:\n assert dict_state + recurse_state + array_state <= 1, (\n \"dict_state / recurse_state / array_state can be only one true\"\n )\n self.size = size\n self.sleep = sleep\n self.random_sleep = random_sleep\n self.dict_state = dict_state\n self.recurse_state = recurse_state\n self.array_state = array_state\n self.ma_rew = ma_rew\n self._md_action = multidiscrete_action\n # how many steps this env has stepped\n self.steps = 0\n if dict_state:\n self.observation_space = Dict(\n {\n \"index\": Box(shape=(1,), low=0, high=size - 1),\n \"rand\": Box(shape=(1,), low=0, high=1, dtype=np.float64),\n },\n )\n elif recurse_state:\n self.observation_space = Dict(\n {\n \"index\": Box(shape=(1,), low=0, high=size - 1),\n \"dict\": Dict(\n {\n \"tuple\": Tuple(\n (\n Discrete(2),\n Box(shape=(2,), low=0, high=1, dtype=np.float64),\n ),\n ),\n \"rand\": Box(shape=(1, 2), low=0, high=1, dtype=np.float64),\n },\n ),\n },\n )\n elif array_state:\n self.observation_space = Box(shape=(4, 84, 84), low=0, high=255)\n else:\n self.observation_space = Box(shape=(1,), low=0, high=size - 1)\n if multidiscrete_action:\n self.action_space = MultiDiscrete([2, 2])\n else:\n self.action_space = Discrete(2)\n self.terminated = False\n self.index = 0\n\n def reset(\n self,\n seed: int | None = None,\n # TODO: passing a dict here doesn't make any sense\n options: dict[str, Any] | None = None,\n ) -> tuple[dict[str, Any] | np.ndarray, dict]:\n \"\"\":param seed:\n :param options: the start index is provided in options[\"state\"]\n :return:\n \"\"\"\n if options is None:\n options = {\"state\": 0}\n super().reset(seed=seed)\n self.terminated = False\n self.do_sleep()\n self.index = options[\"state\"]\n return self._get_state(), {\"key\": 1, \"env\": self}\n\n def _get_reward(self) -> list[int] | int:\n \"\"\"Generate a non-scalar reward if ma_rew is True.\"\"\"\n end_flag = int(self.terminated)\n if self.ma_rew > 0:\n return [end_flag] * self.ma_rew\n return end_flag\n\n def _get_state(self) -> dict[str, Any] | np.ndarray:\n \"\"\"Generate state(observation) of MyTestEnv.\"\"\"\n if self.dict_state:\n return {\n \"index\": np.array([self.index], dtype=np.float32),\n \"rand\": self.np_random.random(1),\n }\n if self.recurse_state:\n return {\n \"index\": np.array([self.index], dtype=np.float32),\n \"dict\": {\n \"tuple\": (np.array([1], dtype=int), self.np_random.random(2)),\n \"rand\": self.np_random.random((1, 2)),\n },\n }\n if self.array_state:\n img = np.zeros([4, 84, 84], int)\n img[3, np.arange(84), np.arange(84)] = self.index\n img[2, np.arange(84)] = self.index\n img[1, :, np.arange(84)] = self.index\n img[0] = self.index\n return img\n return np.array([self.index], dtype=np.float32)\n\n def do_sleep(self) -> None:\n if self.sleep > 0:\n sleep_time = random.random() if self.random_sleep else 1\n sleep_time *= self.sleep\n time.sleep(sleep_time)\n\n def step(self, action: np.ndarray | int): # type: ignore[no-untyped-def] # cf. issue #1080\n self.steps += 1\n if self._md_action and isinstance(action, np.ndarray):\n action = action[0]\n if self.terminated:\n raise ValueError(\"step after done !!!\")\n self.do_sleep()\n if self.index == self.size:\n self.terminated = True\n return self._get_state(), self._get_reward(), self.terminated, False, {}\n\n info_dict = {\"key\": 1, \"env\": self}\n if action == 0:\n self.index = max(self.index - 1, 0)\n return (\n self._get_state(),\n self._get_reward(),\n self.terminated,\n False,\n info_dict,\n )\n if action == 1:\n self.index += 1\n self.terminated = self.index == self.size\n return (\n self._get_state(),\n self._get_reward(),\n self.terminated,\n False,\n info_dict,\n )\n raise ValueError(f\"Invalid action {action}\")\n\n\nclass NXEnv(gym.Env):\n def __init__(self, size: int, obs_type: str, feat_dim: int = 32) -> None:\n self.size = size\n self.feat_dim = feat_dim\n self.graph = nx.Graph()\n self.graph.add_nodes_from(list(range(size)))\n assert obs_type in [\"array\", \"object\"]\n self.obs_type = obs_type\n\n def _encode_obs(self) -> np.ndarray | nx.Graph:\n if self.obs_type == \"array\":\n return np.stack([v[\"data\"] for v in self.graph._node.values()])\n return deepcopy(self.graph)\n\n def reset(\n self,\n seed: int | None = None,\n options: dict[str, Any] | None = None,\n ) -> tuple[np.ndarray | nx.Graph, dict]:\n super().reset(seed=seed)\n graph_state = np.random.rand(self.size, self.feat_dim)\n for i in range(self.size):\n self.graph.nodes[i][\"data\"] = graph_state[i]\n return self._encode_obs(), {}\n\n def step(\n self,\n action: Space,\n ) -> tuple[np.ndarray | nx.Graph, float, Literal[False], Literal[False], dict]:\n next_graph_state = np.random.rand(self.size, self.feat_dim)\n for i in range(self.size):\n self.graph.nodes[i][\"data\"] = next_graph_state[i]\n return self._encode_obs(), 1.0, False, False, {}\n\n\nclass MyGoalEnv(MoveToRightEnv):\n def __init__(self, *args: Any, **kwargs: Any) -> None:\n assert kwargs.get(\"dict_state\", 0) + kwargs.get(\"recurse_state\", 0) == 0, (\n \"dict_state / recurse_state not supported\"\n )\n super().__init__(*args, **kwargs)\n super().reset(options={\"state\": 0})\n\n # will result in obs=1, I guess, so the goal is to reach the max size by moving right\n obs, _, _, _, _ = super().step(1)\n\n self._goal = obs * self.size\n super_obsv = self.observation_space\n self.observation_space = gym.spaces.Dict(\n {\n \"observation\": super_obsv,\n \"achieved_goal\": super_obsv,\n \"desired_goal\": super_obsv,\n },\n )\n\n def reset(self, *args: Any, **kwargs: Any) -> tuple[dict[str, Any], dict]:\n obs, info = super().reset(*args, **kwargs)\n new_obs = {\"observation\": obs, \"achieved_goal\": obs, \"desired_goal\": self._goal}\n return new_obs, info\n\n def step(self, *args: Any, **kwargs: Any) -> tuple[dict[str, Any], float, bool, bool, dict]:\n obs_next, rew, terminated, truncated, info = super().step(*args, **kwargs)\n new_obs_next = {\n \"observation\": obs_next,\n \"achieved_goal\": obs_next,\n \"desired_goal\": self._goal,\n }\n return new_obs_next, rew, terminated, truncated, info\n\n def compute_reward_fn(\n self,\n achieved_goal: np.ndarray,\n desired_goal: np.ndarray,\n info: dict,\n ) -> np.ndarray:\n axis: tuple[int, ...] = (-3, -2, -1) if self.array_state else (-1,)\n return (achieved_goal == desired_goal).all(axis=axis)\n" }, { "path": "test/base/test_action_space_sampling.py", "content": "import gymnasium as gym\n\nfrom tianshou.env import DummyVectorEnv, ShmemVectorEnv, SubprocVectorEnv\n\n\ndef test_gym_env_action_space() -> None:\n env = gym.make(\"Pendulum-v1\")\n env.action_space.seed(0)\n action1 = env.action_space.sample()\n\n env.action_space.seed(0)\n action2 = env.action_space.sample()\n\n assert action1 == action2\n\n\ndef test_dummy_vec_env_action_space() -> None:\n num_envs = 4\n envs = DummyVectorEnv([lambda: gym.make(\"Pendulum-v1\") for _ in range(num_envs)])\n envs.seed(0)\n action1 = [ac_space.sample() for ac_space in envs.action_space]\n\n envs.seed(0)\n action2 = [ac_space.sample() for ac_space in envs.action_space]\n\n assert action1 == action2\n\n\ndef test_subproc_vec_env_action_space() -> None:\n num_envs = 4\n envs = SubprocVectorEnv([lambda: gym.make(\"Pendulum-v1\") for _ in range(num_envs)])\n envs.seed(0)\n action1 = [ac_space.sample() for ac_space in envs.action_space]\n\n envs.seed(0)\n action2 = [ac_space.sample() for ac_space in envs.action_space]\n\n assert action1 == action2\n\n\ndef test_shmem_vec_env_action_space() -> None:\n num_envs = 4\n envs = ShmemVectorEnv([lambda: gym.make(\"Pendulum-v1\") for _ in range(num_envs)])\n envs.seed(0)\n action1 = [ac_space.sample() for ac_space in envs.action_space]\n\n envs.seed(0)\n action2 = [ac_space.sample() for ac_space in envs.action_space]\n\n assert action1 == action2\n" }, { "path": "test/base/test_batch.py", "content": "import copy\nimport pickle\nfrom itertools import starmap\nfrom typing import Any, cast\n\nimport networkx as nx\nimport numpy as np\nimport pytest\nimport torch\nfrom deepdiff import DeepDiff\nfrom torch.distributions import Distribution, Independent, Normal\nfrom torch.distributions.categorical import Categorical\n\nfrom tianshou.data import Batch, to_numpy, to_torch\nfrom tianshou.data.batch import IndexType, dist_to_atleast_2d, get_sliced_dist\n\n\ndef test_batch() -> None:\n assert list(Batch()) == []\n assert len(Batch().get_keys()) == 0\n assert len(Batch(b={\"c\": {}}).get_keys()) != 0\n assert len(Batch(b={\"c\": {}})) == 0\n assert len(Batch(a=Batch(), b=Batch(c=Batch())).get_keys()) != 0\n assert len(Batch(a=Batch(), b=Batch(c=Batch()))) == 0\n assert len(Batch(d=1).get_keys()) != 0\n assert len(Batch(a=np.float64(1.0)).get_keys()) != 0\n assert len(Batch(a=[1, 2, 3], b={\"c\": {}})) == 3\n assert len(Batch(a=[1, 2, 3]).get_keys()) != 0\n b = Batch({\"a\": [4, 4], \"b\": [5, 5]}, c=[None, None])\n assert b.c.dtype == object\n b = Batch(d=[None], e=[starmap], f=Batch)\n assert b.d.dtype == b.e.dtype == object\n assert b.f == Batch\n b = Batch()\n b.update()\n assert len(b.get_keys()) == 0\n b.update(c=[3, 5])\n assert np.allclose(b.c, [3, 5])\n # mimic the behavior of dict.update, where kwargs can overwrite keys\n b.update({\"a\": 2}, a=3)\n assert \"a\" in b\n assert b.a == 3\n assert b.pop(\"a\") == 3\n assert \"a\" not in b\n with pytest.raises(AssertionError):\n Batch({1: 2})\n batch = Batch(a=[torch.ones(3), torch.ones(3)])\n assert Batch(a=[np.zeros((2, 3)), np.zeros((3, 3))]).a.dtype == object\n with pytest.raises(TypeError):\n Batch(a=[np.zeros((3, 2)), np.zeros((3, 3))])\n with pytest.raises(TypeError):\n Batch(a=[torch.zeros((2, 3)), torch.zeros((3, 3))])\n assert torch.allclose(batch.a, torch.ones(2, 3))\n batch.cat_(batch)\n assert torch.allclose(batch.a, torch.ones(4, 3))\n Batch(a=[])\n batch = Batch(obs=[0], np=np.zeros([3, 4]))\n assert batch.obs == batch[\"obs\"]\n batch.obs = [1]\n assert batch.obs == [1]\n batch.cat_(batch)\n assert np.allclose(batch.obs, [1, 1])\n assert batch.np.shape == (6, 4)\n assert np.allclose(batch[0].obs, batch[1].obs)\n batch.obs = np.arange(5)\n for i, b in enumerate(batch.split(1, shuffle=False)):\n if i != 5:\n assert b.obs == batch[i].obs\n else:\n with pytest.raises(AttributeError):\n batch[i].obs # noqa: B018\n with pytest.raises(AttributeError):\n b.obs # noqa: B018\n print(batch)\n batch = Batch(a=np.arange(10))\n with pytest.raises(AssertionError):\n list(batch.split(0))\n data = [\n (1, False, [[0], [1], [2], [3], [4], [5], [6], [7], [8], [9]]),\n (1, True, [[0], [1], [2], [3], [4], [5], [6], [7], [8], [9]]),\n (3, False, [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]),\n (3, True, [[0, 1, 2], [3, 4, 5], [6, 7, 8, 9]]),\n (5, False, [[0, 1, 2, 3, 4], [5, 6, 7, 8, 9]]),\n (5, True, [[0, 1, 2, 3, 4], [5, 6, 7, 8, 9]]),\n (7, False, [[0, 1, 2, 3, 4, 5, 6], [7, 8, 9]]),\n (7, True, [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]]),\n (10, False, [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]]),\n (10, True, [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]]),\n (15, False, [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]]),\n (15, True, [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]]),\n (100, False, [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]]),\n (100, True, [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]]),\n ]\n for size, merge_last, result in data:\n bs = list(batch.split(size, shuffle=False, merge_last=merge_last))\n assert [bs[i].a.tolist() for i in range(len(bs))] == result\n batch_dict = {\"b\": np.array([1.0]), \"c\": 2.0, \"d\": torch.Tensor([3.0])}\n batch_item = Batch({\"a\": [batch_dict]})[0]\n assert isinstance(batch_item.a.b, np.ndarray)\n assert batch_item.a.b == batch_dict[\"b\"]\n assert isinstance(batch_item.a.c, float)\n assert batch_item.a.c == batch_dict[\"c\"]\n assert isinstance(batch_item.a.d, torch.Tensor)\n assert batch_item.a.d == batch_dict[\"d\"]\n batch2 = Batch(a=[{\"b\": np.float64(1.0), \"c\": np.zeros(1), \"d\": Batch(e=np.array(3.0))}])\n assert len(batch2) == 1\n assert Batch().shape == []\n assert Batch(a=1).shape == []\n assert Batch(a={1, 2}).shape == []\n assert batch2.shape[0] == 1\n assert \"a\" in batch2\n assert all(i in batch2.a for i in \"bcd\")\n with pytest.raises(IndexError):\n batch2[-2]\n with pytest.raises(IndexError):\n batch2[1]\n assert batch2[0].shape == []\n with pytest.raises(IndexError):\n batch2[0][0]\n with pytest.raises(TypeError):\n len(batch2[0])\n assert isinstance(batch2[0].a.c, np.ndarray)\n assert isinstance(batch2[0].a.b, float)\n assert isinstance(batch2[0].a.d.e, float)\n batch2_from_list = Batch(list(batch2))\n batch2_from_comp = Batch(list(batch2))\n assert batch2_from_list.a.b == batch2.a.b\n assert batch2_from_list.a.c == batch2.a.c\n assert batch2_from_list.a.d.e == batch2.a.d.e\n assert batch2_from_comp.a.b == batch2.a.b\n assert batch2_from_comp.a.c == batch2.a.c\n assert batch2_from_comp.a.d.e == batch2.a.d.e\n for batch_slice in [batch2[slice(0, 1)], batch2[:1], batch2[0:]]:\n assert batch_slice.a.b == batch2.a.b\n assert batch_slice.a.c == batch2.a.c\n assert batch_slice.a.d.e == batch2.a.d.e\n batch2.a.d.f = {}\n batch2_sum = (batch2 + 1.0) * 2 # type: ignore # __add__ supports Number as input type\n assert batch2_sum.a.b == (batch2.a.b + 1.0) * 2\n assert batch2_sum.a.c == (batch2.a.c + 1.0) * 2\n assert batch2_sum.a.d.e == (batch2.a.d.e + 1.0) * 2\n assert len(batch2_sum.a.d.f.get_keys()) == 0\n with pytest.raises(TypeError):\n batch2 += [1] # type: ignore # error is raised explicitly\n batch3 = Batch(a={\"c\": np.zeros(1), \"d\": Batch(e=np.array([0.0]), f=np.array([3.0]))})\n batch3.a.d[0] = {\"e\": 4.0}\n assert batch3.a.d.e[0] == 4.0\n batch3.a.d[0] = Batch(f=5.0)\n assert batch3.a.d.f[0] == 5.0\n with pytest.raises(ValueError):\n batch3.a.d[0] = Batch(f=5.0, g=0.0)\n with pytest.raises(ValueError):\n batch3[0] = Batch(a={\"c\": 2, \"e\": 1})\n # auto convert\n batch4 = Batch(a=np.array([\"a\", \"b\"]))\n assert batch4.a.dtype == object # auto convert to object\n batch4.update(a=np.array([\"c\", \"d\"]))\n assert list(batch4.a) == [\"c\", \"d\"]\n assert batch4.a.dtype == object # auto convert to object\n batch5 = Batch(a=np.array([{\"index\": 0}]))\n assert isinstance(batch5.a, Batch)\n assert np.allclose(batch5.a.index, [0])\n # We use setattr b/c the setattr of Batch will actually change the type of the field that is being set!\n # However, mypy would not understand this, and rightly expect that batch.b = some_array would lead to\n # batch.b being an array (which it is not, it's turned into a Batch instead)\n batch5.b = np.array([{\"index\": 1}])\n batch5.b = cast(Batch, batch5.b)\n assert isinstance(batch5.b, Batch)\n assert np.allclose(batch5.b.index, [1])\n\n # None is a valid object and can be stored in Batch\n a = Batch.stack([Batch(a=None), Batch(b=None)])\n assert a.a[0] is None\n assert a.a[1] is None\n assert a.b[0] is None\n assert a.b[1] is None\n\n # nx.Graph corner case\n assert Batch(a=np.array([nx.Graph(), nx.Graph()], dtype=object)).a.dtype == object\n g1 = nx.Graph()\n g1.add_nodes_from(list(range(10)))\n g2 = nx.Graph()\n g2.add_nodes_from(list(range(20)))\n assert Batch(a=np.array([g1, g2], dtype=object)).a.dtype == object\n\n\ndef test_batch_over_batch() -> None:\n batch = Batch(a=[3, 4, 5], b=[4, 5, 6])\n batch2 = Batch({\"c\": [6, 7, 8], \"b\": batch})\n batch2.b.b[-1] = 0\n print(batch2)\n for k, v in batch2.items():\n assert np.all(batch2[k] == v)\n assert batch2[-1].b.b == 0\n batch2.cat_(Batch(c=[6, 7, 8], b=batch))\n assert np.allclose(batch2.c, [6, 7, 8, 6, 7, 8])\n assert np.allclose(batch2.b.a, [3, 4, 5, 3, 4, 5])\n assert np.allclose(batch2.b.b, [4, 5, 0, 4, 5, 0])\n batch2.update(batch2.b, six=[6, 6, 6])\n assert np.allclose(batch2.c, [6, 7, 8, 6, 7, 8])\n assert np.allclose(batch2.a, [3, 4, 5, 3, 4, 5])\n assert np.allclose(batch2.b, [4, 5, 0, 4, 5, 0])\n assert np.allclose(batch2.six, [6, 6, 6])\n d = {\"a\": [3, 4, 5], \"b\": [4, 5, 6]}\n batch3 = Batch(c=[6, 7, 8], b=d)\n batch3.cat_(Batch(c=[6, 7, 8], b=d))\n assert np.allclose(batch3.c, [6, 7, 8, 6, 7, 8])\n assert np.allclose(batch3.b.a, [3, 4, 5, 3, 4, 5])\n assert np.allclose(batch3.b.b, [4, 5, 6, 4, 5, 6])\n batch4 = Batch(({\"a\": {\"b\": np.array([1.0])}},))\n assert batch4.a.b.ndim == 2\n assert batch4.a.b[0, 0] == 1.0\n # advanced slicing\n batch5 = Batch(a=[[1, 2]], b={\"c\": np.zeros([3, 2, 1])})\n assert batch5.shape == [1, 2]\n with pytest.raises(IndexError):\n batch5[2]\n with pytest.raises(IndexError):\n batch5[:, 3]\n with pytest.raises(IndexError):\n batch5[:, :, -1]\n batch5[:, -1] += np.int_(1)\n assert np.allclose(batch5.a, [1, 3])\n assert np.allclose(batch5.b.c.squeeze(), [[0, 1]] * 3)\n with pytest.raises(ValueError):\n batch5[:, -1] = 1\n batch5[:, 0] = {\"a\": -1}\n assert np.allclose(batch5.a, [-1, 3])\n assert np.allclose(batch5.b.c.squeeze(), [[0, 1]] * 3)\n\n\ndef test_batch_cat_and_stack() -> None:\n # test cat with compatible keys\n b1 = Batch(a=[{\"b\": np.float64(1.0), \"d\": Batch(e=np.array(3.0))}])\n b2 = Batch(a=[{\"b\": np.float64(4.0), \"d\": {\"e\": np.array(6.0)}}])\n b12_cat_out = Batch.cat([b1, b2])\n b12_cat_in = copy.deepcopy(b1)\n b12_cat_in.cat_(b2)\n assert np.all(b12_cat_in.a.d.e == b12_cat_out.a.d.e)\n assert np.all(b12_cat_in.a.d.e == b12_cat_out.a.d.e)\n assert isinstance(b12_cat_in.a.d.e, np.ndarray)\n assert b12_cat_in.a.d.e.ndim == 1\n\n a = Batch(a=Batch(a=np.random.randn(3, 4)))\n a_empty = Batch(a=Batch(a=Batch()))\n assert np.allclose(\n np.concatenate([a.a.a, a.a.a]),\n Batch.cat([a, a_empty, a]).a.a,\n )\n\n # test cat with lens infer\n a = Batch(a=Batch(a=np.random.randn(3, 4), t=Batch()), b=np.random.randn(3, 4))\n b = Batch(a=Batch(a=Batch(), t=Batch()), b=np.random.randn(3, 4))\n ans = Batch.cat([a, b, a])\n assert np.allclose(ans.a.a, np.concatenate([a.a.a, np.zeros((3, 4)), a.a.a]))\n assert np.allclose(ans.b, np.concatenate([a.b, b.b, a.b]))\n assert len(ans.a.t.get_keys()) == 0\n\n b1.stack_([b2])\n assert isinstance(b1.a.d.e, np.ndarray)\n assert b1.a.d.e.ndim == 2\n\n # test cat with all reserved keys (values are Batch())\n b1 = Batch(a=Batch(), b=torch.zeros(3, 3), common=Batch(c=np.random.rand(3, 5)))\n b2 = Batch(a=Batch(), b=torch.rand(4, 3), common=Batch(c=np.random.rand(4, 5)))\n test = Batch.cat([b1, b2])\n ans = Batch(\n a=Batch(),\n b=torch.cat([torch.zeros(3, 3), b2.b]),\n common=Batch(c=np.concatenate([b1.common.c, b2.common.c])),\n )\n assert len(ans.a.get_keys()) == 0\n assert torch.allclose(test.b, ans.b)\n assert np.allclose(test.common.c, ans.common.c)\n\n # test stack with compatible keys\n b3 = Batch(a=np.zeros((3, 4)), b=torch.ones((2, 5)), c=Batch(d=[[1], [2]]))\n b4 = Batch(a=np.ones((3, 4)), b=torch.ones((2, 5)), c=Batch(d=[[0], [3]]))\n b34_stack = Batch.stack((b3, b4), axis=1)\n assert np.all(b34_stack.a == np.stack((b3.a, b4.a), axis=1))\n assert np.all(b34_stack.c.d == list(map(list, zip(b3.c.d, b4.c.d, strict=True))))\n b5_dict = np.array([{\"a\": False, \"b\": {\"c\": 2.0, \"d\": 1.0}}, {\"a\": True, \"b\": {\"c\": 3.0}}])\n b5 = Batch(b5_dict)\n assert b5.a[0] == np.array(False)\n assert b5.a[1] == np.array(True)\n assert np.all(b5.b.c == np.stack([e[\"b\"][\"c\"] for e in b5_dict], axis=0))\n assert b5.b.d[0] == b5_dict[0][\"b\"][\"d\"]\n assert b5.b.d[1] == 0.0\n\n # test stack with incompatible keys\n a = Batch(a=1, b=2, c=3)\n b = Batch(a=4, b=5, d=6)\n c = Batch(c=7, b=6, d=9)\n d = Batch.stack([a, b, c])\n assert np.allclose(d.a, [1, 4, 0])\n assert np.allclose(d.b, [2, 5, 6])\n assert np.allclose(d.c, [3, 0, 7])\n assert np.allclose(d.d, [0, 6, 9])\n\n # test stack with empty Batch()\n assert len(Batch.stack([Batch(), Batch(), Batch()]).get_keys()) == 0\n a = Batch(a=1, b=2, c=3, d=Batch(), e=Batch())\n b = Batch(a=4, b=5, d=6, e=Batch())\n c = Batch(c=7, b=6, d=9, e=Batch())\n d = Batch.stack([a, b, c])\n assert np.allclose(d.a, [1, 4, 0])\n assert np.allclose(d.b, [2, 5, 6])\n assert np.allclose(d.c, [3, 0, 7])\n assert np.allclose(d.d, [0, 6, 9])\n assert len(d.e.get_keys()) == 0\n b1 = Batch(a=Batch(), common=Batch(c=np.random.rand(4, 5)))\n b2 = Batch(b=Batch(), common=Batch(c=np.random.rand(4, 5)))\n test = Batch.stack([b1, b2], axis=-1)\n assert len(test.a.get_keys()) == 0\n assert len(test.b.get_keys()) == 0\n assert np.allclose(test.common.c, np.stack([b1.common.c, b2.common.c], axis=-1))\n\n b1 = Batch(a=np.random.rand(4, 4), common=Batch(c=np.random.rand(4, 5)))\n b2 = Batch(b=torch.rand(4, 6), common=Batch(c=np.random.rand(4, 5)))\n test = Batch.stack([b1, b2])\n ans = Batch(\n a=np.stack([b1.a, np.zeros((4, 4))]),\n b=torch.stack([torch.zeros(4, 6), b2.b]),\n common=Batch(c=np.stack([b1.common.c, b2.common.c])),\n )\n assert np.allclose(test.a, ans.a)\n assert torch.allclose(test.b, ans.b)\n assert np.allclose(test.common.c, ans.common.c)\n\n # test with illegal input format\n with pytest.raises(ValueError):\n Batch.cat([[Batch(a=1)], [Batch(a=1)]]) # type: ignore # cat() tested with invalid inp\n with pytest.raises(ValueError):\n Batch.stack([[Batch(a=1)], [Batch(a=1)]]) # type: ignore # stack() tested with invalid inp\n\n # exceptions\n batch_cat: Batch = Batch.cat([])\n assert len(batch_cat.get_keys()) == 0\n batch_stack: Batch = Batch.stack([])\n assert len(batch_stack.get_keys()) == 0\n b1 = Batch(e=[4, 5], d=6)\n b2 = Batch(e=[4, 6])\n with pytest.raises(ValueError):\n Batch.cat([b1, b2])\n with pytest.raises(ValueError):\n Batch.stack([b1, b2], axis=1)\n\n\ndef test_utils_to_torch_numpy() -> None:\n batch = Batch(\n a=np.float64(1.0),\n b=Batch(c=np.ones((1,), dtype=np.float32), d=torch.ones((1,), dtype=torch.float64)),\n )\n a_torch_float = to_torch(batch.a, dtype=torch.float32)\n assert a_torch_float.dtype == torch.float32\n a_torch_double = to_torch(batch.a, dtype=torch.float64)\n assert a_torch_double.dtype == torch.float64\n batch_torch_float = to_torch(batch, dtype=torch.float32)\n assert batch_torch_float.a.dtype == torch.float64\n assert batch_torch_float.b.c.dtype == torch.float32\n assert batch_torch_float.b.d.dtype == torch.float32\n data_list = [float(\"nan\"), 1]\n data_list_torch = to_torch(data_list)\n assert data_list_torch.dtype == torch.float64\n data_list_2 = [np.random.rand(3, 3), np.random.rand(3, 3)]\n data_list_2_torch = to_torch(data_list_2)\n assert data_list_2_torch.shape == (2, 3, 3)\n assert np.allclose(to_numpy(to_torch(data_list_2)), data_list_2)\n data_list_3 = [np.zeros((3, 2)), np.zeros((3, 3))]\n data_list_3_torch = [torch.zeros((3, 2)), torch.zeros((3, 3))]\n with pytest.raises(TypeError):\n to_torch(data_list_3)\n with pytest.raises(TypeError):\n to_numpy(data_list_3_torch)\n data_list_4 = [np.zeros((2, 3)), np.zeros((3, 3))]\n data_list_4_torch = [torch.zeros((2, 3)), torch.zeros((3, 3))]\n with pytest.raises(TypeError):\n to_torch(data_list_4)\n with pytest.raises(TypeError):\n to_numpy(data_list_4_torch)\n data_list_5 = [np.zeros(2), np.zeros((3, 3))]\n data_list_5_torch = [torch.zeros(2), torch.zeros((3, 3))]\n with pytest.raises(TypeError):\n to_torch(data_list_5)\n with pytest.raises(TypeError):\n to_numpy(data_list_5_torch)\n data_array = np.random.rand(3, 2, 2)\n data_empty_tensor = to_torch(data_array[[]])\n assert isinstance(data_empty_tensor, torch.Tensor)\n assert data_empty_tensor.shape == (0, 2, 2)\n data_empty_array = to_numpy(data_empty_tensor)\n assert isinstance(data_empty_array, np.ndarray)\n assert data_empty_array.shape == (0, 2, 2)\n assert np.allclose(to_numpy(to_torch(data_array)), data_array)\n # additional test for to_numpy, for code-coverage\n assert isinstance(to_numpy(1), np.ndarray)\n assert isinstance(to_numpy(1.0), np.ndarray)\n assert isinstance(to_numpy({\"a\": torch.tensor(1)})[\"a\"], np.ndarray)\n assert isinstance(to_numpy(Batch(a=torch.tensor(1))).a, np.ndarray)\n assert to_numpy(None).item() is None\n assert to_numpy(to_numpy).item() == to_numpy\n # additional test for to_torch, for code-coverage\n assert isinstance(to_torch(1), torch.Tensor)\n assert to_torch(1).dtype in (torch.int64, torch.int32)\n assert to_torch(1.0).dtype == torch.float64\n assert isinstance(to_torch({\"a\": [1]})[\"a\"], torch.Tensor)\n with pytest.raises(TypeError):\n to_torch(None)\n with pytest.raises(TypeError):\n to_torch(np.array([{}, \"2\"]))\n\n\ndef test_batch_pickle() -> None:\n batch = Batch(obs=Batch(a=0.0, c=torch.Tensor([1.0, 2.0])), np=np.zeros([3, 4]))\n batch_pk = pickle.loads(pickle.dumps(batch))\n assert batch.obs.a == batch_pk.obs.a\n assert torch.all(batch.obs.c == batch_pk.obs.c)\n assert np.all(batch.np == batch_pk.np)\n\n\ndef test_batch_copy() -> None:\n batch = Batch(a=np.array([3, 4, 5]), b=np.array([4, 5, 6]))\n batch2 = Batch({\"c\": np.array([6, 7, 8]), \"b\": batch})\n orig_c_addr = batch2.c.__array_interface__[\"data\"][0]\n orig_b_a_addr = batch2.b.a.__array_interface__[\"data\"][0]\n orig_b_b_addr = batch2.b.b.__array_interface__[\"data\"][0]\n # test with copy=False\n batch3 = Batch(copy=False, **batch2)\n curr_c_addr = batch3.c.__array_interface__[\"data\"][0]\n curr_b_a_addr = batch3.b.a.__array_interface__[\"data\"][0]\n curr_b_b_addr = batch3.b.b.__array_interface__[\"data\"][0]\n assert batch2.c is batch3.c\n assert batch2.b is batch3.b\n assert batch2.b.a is batch3.b.a\n assert batch2.b.b is batch3.b.b\n assert orig_c_addr == curr_c_addr\n assert orig_b_a_addr == curr_b_a_addr\n assert orig_b_b_addr == curr_b_b_addr\n # test with copy=True\n batch3 = Batch(copy=True, **batch2)\n curr_c_addr = batch3.c.__array_interface__[\"data\"][0]\n curr_b_a_addr = batch3.b.a.__array_interface__[\"data\"][0]\n curr_b_b_addr = batch3.b.b.__array_interface__[\"data\"][0]\n assert batch2.c is not batch3.c\n assert batch2.b is not batch3.b\n assert batch2.b.a is not batch3.b.a\n assert batch2.b.b is not batch3.b.b\n assert orig_c_addr != curr_c_addr\n assert orig_b_a_addr != curr_b_a_addr\n assert orig_b_b_addr != curr_b_b_addr\n\n\ndef test_batch_empty() -> None:\n b5_dict = np.array([{\"a\": False, \"b\": {\"c\": 2.0, \"d\": 1.0}}, {\"a\": True, \"b\": {\"c\": 3.0}}])\n b5 = Batch(b5_dict)\n b5[1] = Batch.empty(b5[0])\n assert np.allclose(b5.a, [False, False])\n assert np.allclose(b5.b.c, [2, 0])\n assert np.allclose(b5.b.d, [1, 0])\n data = Batch(\n a=[False, True],\n b={\n \"c\": np.array([2.0, \"st\"], dtype=object),\n \"d\": [1, None],\n \"e\": [2.0, float(\"nan\")],\n },\n c=np.array([1, 3, 4], dtype=int),\n t=torch.tensor([4, 5, 6, 7.0]),\n )\n data[-1] = Batch.empty(data[1])\n assert np.allclose(data.c, [1, 3, 0])\n assert np.allclose(data.a, [False, False])\n assert list(data.b.c) == [2.0, None]\n assert list(data.b.d) == [1, None]\n assert np.allclose(data.b.e, [2, 0])\n assert torch.allclose(data.t, torch.tensor([4, 5, 6, 0.0]))\n data[0].empty_() # which will fail in a, b.c, b.d, b.e, c\n assert torch.allclose(data.t, torch.tensor([0.0, 5, 6, 0]))\n data.empty_(index=0)\n assert np.allclose(data.c, [0, 3, 0])\n assert list(data.b.c) == [None, None]\n assert list(data.b.d) == [None, None]\n assert list(data.b.e) == [0, 0]\n b0 = Batch()\n b0.empty_()\n assert b0.shape == []\n\n\ndef test_batch_standard_compatibility() -> None:\n batch = Batch(a=np.array([[1.0, 2.0], [3.0, 4.0]]), b=Batch(), c=np.array([5.0, 6.0]))\n batch_mean = np.mean(batch)\n assert isinstance(batch_mean, Batch) # type: ignore # mypy doesn't know but it works, cf. `batch.rst`\n assert sorted(batch_mean.get_keys()) == [\"a\", \"b\", \"c\"] # type: ignore\n with pytest.raises(TypeError):\n len(batch_mean)\n assert np.all(batch_mean.a == np.mean(batch.a, axis=0))\n assert batch_mean.c == np.mean(batch.c, axis=0)\n with pytest.raises(IndexError):\n Batch()[0]\n\n\nclass TestBatchEquality:\n @staticmethod\n def test_keys_different() -> None:\n batch1 = Batch(a=[1, 2], b=[100, 50])\n batch2 = Batch(b=[1, 2], c=[100, 50])\n assert batch1 != batch2\n\n @staticmethod\n def test_keys_missing() -> None:\n batch1 = Batch(a=[1, 2], b=[2, 3, 4])\n batch2 = Batch(a=[1, 2], b=[2, 3, 4])\n batch2.pop(\"b\")\n assert batch1 != batch2\n\n @staticmethod\n def test_types_keys_different() -> None:\n batch1 = Batch(a=[1, 2, 3], b=[4, 5])\n batch2 = Batch(a=[1, 2, 3], b=Batch(a=[4, 5]))\n assert batch1 != batch2\n\n @staticmethod\n def test_array_types_different() -> None:\n batch1 = Batch(a=[1, 2, 3], b=np.array([4, 5]))\n batch2 = Batch(a=[1, 2, 3], b=torch.Tensor([4, 5]))\n assert batch1 != batch2\n\n @staticmethod\n def test_nested_values_different() -> None:\n batch1 = Batch(a=Batch(a=[1, 2, 3]), b=[4, 5])\n batch2 = Batch(a=Batch(a=[1, 2, 4]), b=[4, 5])\n assert batch1 != batch2\n\n @staticmethod\n def test_nested_shapes_different() -> None:\n batch1 = Batch(a=Batch(a=[1, 2, 3]), b=[4, 5])\n batch2 = Batch(a=Batch(a=[1, 4]), b=[4, 5])\n assert batch1 != batch2\n\n @staticmethod\n def test_array_scalars() -> None:\n batch1 = Batch(a={\"b\": 1})\n batch2 = Batch(a={\"b\": 1})\n assert batch1 == batch2\n\n batch3 = Batch(a={\"c\": 2})\n assert batch1 != batch3\n\n batch4 = Batch(b={\"b\": 1})\n assert batch1 != batch4\n\n batch5 = Batch(a={\"b\": 10})\n assert batch1 != batch5\n\n batch6 = Batch(a={\"b\": [1]})\n assert batch1 == batch6\n\n batch7 = Batch(a=1, b=5)\n batch8 = Batch(a=1, b=5)\n assert batch7 == batch8\n\n @staticmethod\n def test_slice_equal() -> None:\n batch1 = Batch(a=[1, 2, 3])\n assert batch1[:2] == batch1[:2]\n\n @staticmethod\n def test_slice_ellipsis_equal() -> None:\n batch1 = Batch(a=Batch(a=[1, 2, 3]), b=[4, 5], c=[100, 1001, 2000])\n assert batch1[..., 1:] == batch1[..., 1:]\n\n @staticmethod\n def test_empty_batches() -> None:\n assert Batch() == Batch()\n\n @staticmethod\n def test_different_order_keys() -> None:\n assert Batch(a=1, b=2) == Batch(b=2, a=1)\n\n @staticmethod\n def test_tuple_and_list_types() -> None:\n assert Batch(a=(1, 2)) == Batch(a=[1, 2])\n\n @staticmethod\n def test_subbatch_dict_and_batch_types() -> None:\n assert Batch(a={\"x\": 1}) == Batch(a=Batch(x=1))\n\n\nclass TestBatchToDict:\n @staticmethod\n def test_to_dict_empty_batch_no_recurse() -> None:\n batch = Batch()\n expected: dict[Any, Any] = {}\n assert batch.to_dict() == expected\n\n @staticmethod\n def test_to_dict_with_simple_values_recurse() -> None:\n batch = Batch(a=1, b=\"two\", c=np.array([3, 4]))\n expected = {\"a\": np.asanyarray(1), \"b\": \"two\", \"c\": np.array([3, 4])}\n assert not DeepDiff(batch.to_dict(recursive=True), expected)\n\n @staticmethod\n def test_to_dict_simple() -> None:\n batch = Batch(a=1, b=\"two\")\n expected = {\"a\": np.asanyarray(1), \"b\": \"two\"}\n assert batch.to_dict() == expected\n\n @staticmethod\n def test_to_dict_nested_batch_no_recurse() -> None:\n nested_batch = Batch(c=3)\n batch = Batch(a=1, b=nested_batch)\n expected = {\"a\": np.asanyarray(1), \"b\": nested_batch}\n assert not DeepDiff(batch.to_dict(recursive=False), expected)\n\n @staticmethod\n def test_to_dict_nested_batch_recurse() -> None:\n nested_batch = Batch(c=3)\n batch = Batch(a=1, b=nested_batch)\n expected = {\"a\": np.asanyarray(1), \"b\": {\"c\": np.asanyarray(3)}}\n assert not DeepDiff(batch.to_dict(recursive=True), expected)\n\n @staticmethod\n def test_to_dict_multiple_nested_batch_recurse() -> None:\n nested_batch = Batch(c=Batch(e=3), d=[100, 200, 300])\n batch = Batch(a=1, b=nested_batch)\n expected = {\n \"a\": np.asanyarray(1),\n \"b\": {\"c\": {\"e\": np.asanyarray(3)}, \"d\": np.array([100, 200, 300])},\n }\n assert not DeepDiff(batch.to_dict(recursive=True), expected)\n\n @staticmethod\n def test_to_dict_array() -> None:\n batch = Batch(a=np.array([1, 2, 3]))\n expected = {\"a\": np.array([1, 2, 3])}\n assert not DeepDiff(batch.to_dict(), expected)\n\n @staticmethod\n def test_to_dict_nested_batch_with_array() -> None:\n nested_batch = Batch(c=np.array([4, 5]))\n batch = Batch(a=1, b=nested_batch)\n expected = {\"a\": np.asanyarray(1), \"b\": {\"c\": np.array([4, 5])}}\n assert not DeepDiff(batch.to_dict(recursive=True), expected)\n\n @staticmethod\n def test_to_dict_torch_tensor() -> None:\n t1 = torch.tensor([1.0, 2.0]).detach().cpu().numpy()\n batch = Batch(a=t1)\n t2 = torch.tensor([1.0, 2.0]).detach().cpu().numpy()\n expected = {\"a\": t2}\n assert not DeepDiff(batch.to_dict(), expected)\n\n @staticmethod\n def test_to_dict_nested_batch_with_torch_tensor() -> None:\n nested_batch = Batch(c=torch.tensor([4, 5]).detach().cpu().numpy())\n batch = Batch(a=1, b=nested_batch)\n expected = {\n \"a\": np.asanyarray(1),\n \"b\": {\"c\": torch.tensor([4, 5]).detach().cpu().numpy()},\n }\n assert not DeepDiff(batch.to_dict(recursive=True), expected)\n\n\nclass TestBatchConversions:\n @staticmethod\n def test_to_numpy() -> None:\n batch = Batch(a=1, b=torch.arange(5), c={\"d\": torch.tensor([1, 2, 3])})\n new_batch = batch.to_numpy()\n assert id(batch) != id(new_batch)\n assert isinstance(batch.b, torch.Tensor)\n assert isinstance(batch.c.d, torch.Tensor)\n\n assert isinstance(new_batch.b, np.ndarray)\n assert isinstance(new_batch.c.d, np.ndarray)\n\n @staticmethod\n def test_to_numpy_() -> None:\n batch = Batch(a=1, b=torch.arange(5), c={\"d\": torch.tensor([1, 2, 3])})\n id_batch = id(batch)\n batch.to_numpy_()\n assert id_batch == id(batch)\n assert isinstance(batch.b, np.ndarray)\n assert isinstance(batch.c.d, np.ndarray)\n\n @staticmethod\n def test_to_torch() -> None:\n batch = Batch(a=1, b=np.arange(5), c={\"d\": np.array([1, 2, 3])})\n new_batch = batch.to_torch()\n assert id(batch) != id(new_batch)\n assert isinstance(batch.b, np.ndarray)\n assert isinstance(batch.c.d, np.ndarray)\n\n assert isinstance(new_batch.b, torch.Tensor)\n assert isinstance(new_batch.c.d, torch.Tensor)\n\n @staticmethod\n def test_to_torch_() -> None:\n batch = Batch(a=1, b=np.arange(5), c={\"d\": np.array([1, 2, 3])})\n id_batch = id(batch)\n batch.to_torch_()\n assert id_batch == id(batch)\n assert isinstance(batch.b, torch.Tensor)\n assert isinstance(batch.c.d, torch.Tensor)\n\n @staticmethod\n def test_apply_array_func() -> None:\n batch = Batch(a=1, b=np.arange(3), c={\"d\": np.array([1, 2, 3])})\n batch_with_max = batch.apply_values_transform(np.max)\n assert np.array_equal(batch_with_max.a, np.array(1))\n assert np.array_equal(batch_with_max.b, np.array(2))\n assert np.array_equal(batch_with_max.c.d, np.array(3))\n\n batch_array_added = batch.apply_values_transform(lambda x: x + np.array([1, 2, 3]))\n assert np.array_equal(batch_array_added.a, np.array([2, 3, 4]))\n assert np.array_equal(batch_array_added.c.d, np.array([2, 4, 6]))\n\n @staticmethod\n def test_batch_to_numpy_without_copy() -> None:\n batch = Batch(a=np.ones((1,)), b=Batch(c=np.ones((1,))))\n a_mem_addr_orig = batch.a.__array_interface__[\"data\"][0]\n c_mem_addr_orig = batch.b.c.__array_interface__[\"data\"][0]\n batch.to_numpy_()\n a_mem_addr_new = batch.a.__array_interface__[\"data\"][0]\n c_mem_addr_new = batch.b.c.__array_interface__[\"data\"][0]\n assert a_mem_addr_new == a_mem_addr_orig\n assert c_mem_addr_new == c_mem_addr_orig\n\n @staticmethod\n def test_batch_from_to_numpy_without_copy() -> None:\n batch = Batch(a=np.ones((1,)), b=Batch(c=np.ones((1,))))\n a_mem_addr_orig = batch.a.__array_interface__[\"data\"][0]\n c_mem_addr_orig = batch.b.c.__array_interface__[\"data\"][0]\n batch.to_torch_()\n batch.to_numpy_()\n a_mem_addr_new = batch.a.__array_interface__[\"data\"][0]\n c_mem_addr_new = batch.b.c.__array_interface__[\"data\"][0]\n assert a_mem_addr_new == a_mem_addr_orig\n assert c_mem_addr_new == c_mem_addr_orig\n\n @staticmethod\n def test_batch_over_batch_to_torch() -> None:\n batch = Batch(\n a=np.float64(1.0),\n b=Batch(\n c=np.ones((1,), dtype=np.float32),\n d=torch.ones((1,), dtype=torch.float64),\n ),\n )\n batch.b.set_array_at_key(np.array([1]), \"e\")\n batch.to_torch_()\n assert isinstance(batch.a, torch.Tensor)\n assert isinstance(batch.b.c, torch.Tensor)\n assert isinstance(batch.b.d, torch.Tensor)\n assert isinstance(batch.b.e, torch.Tensor)\n assert batch.a.dtype == torch.float64\n assert batch.b.c.dtype == torch.float32\n assert batch.b.d.dtype == torch.float64\n assert batch.b.e.dtype in (torch.int64, torch.int32)\n batch.to_torch_(dtype=torch.float32)\n assert batch.a.dtype == torch.float32\n assert batch.b.c.dtype == torch.float32\n assert batch.b.d.dtype == torch.float32\n assert batch.b.e.dtype == torch.float32\n\n @staticmethod\n @pytest.mark.parametrize(\n \"dist, expected_batch_shape\",\n [\n (Categorical(probs=torch.tensor([0.3, 0.7])), (1,)),\n (Categorical(probs=torch.tensor([[0.3, 0.7], [0.4, 0.6]])), (2,)),\n (Normal(loc=torch.tensor(0.0), scale=torch.tensor(1.0)), (1,)),\n (\n Normal(loc=torch.tensor([0.0, 1.0]), scale=torch.tensor([1.0, 2.0])),\n (2,),\n ),\n (\n Independent(Normal(loc=torch.tensor(0.0), scale=torch.tensor(1.0)), 0),\n (1,),\n ),\n (\n Independent(\n Normal(loc=torch.tensor([0.0, 1.0]), scale=torch.tensor([1.0, 2.0])),\n 0,\n ),\n (2,),\n ),\n ],\n )\n def test_dist_to_atleast_2d(dist: Distribution, expected_batch_shape: tuple[int]) -> None:\n result = dist_to_atleast_2d(dist)\n assert result.batch_shape == expected_batch_shape\n\n # Additionally check that the parameters are correctly transformed\n if isinstance(dist, Categorical):\n assert isinstance(result, Categorical)\n assert result.probs.shape[:-1] == expected_batch_shape\n elif isinstance(dist, Normal):\n assert isinstance(result, Normal)\n assert result.loc.shape == expected_batch_shape\n assert result.scale.shape == expected_batch_shape\n elif isinstance(dist, Independent):\n assert isinstance(result, Independent)\n assert result.base_dist.batch_shape == expected_batch_shape\n\n @staticmethod\n @pytest.mark.parametrize(\n \"dist\",\n [\n Categorical(probs=torch.tensor([0.3, 0.7])),\n Normal(loc=torch.tensor(0.0), scale=torch.tensor(1.0)),\n Independent(Normal(loc=torch.tensor(0.0), scale=torch.tensor(1.0)), 0),\n ],\n )\n def test_dist_to_atleast_2d_idempotent(dist: Distribution) -> None:\n result1 = dist_to_atleast_2d(dist)\n result2 = dist_to_atleast_2d(result1)\n assert result1 == result2\n\n @staticmethod\n def test_batch_to_atleast_2d() -> None:\n scalar_batch = Batch(a=1, b=2, dist=Categorical(probs=torch.ones(3)))\n assert scalar_batch.dist.batch_shape == ()\n assert scalar_batch.a.shape == scalar_batch.b.shape == ()\n scalar_batch_2d = scalar_batch.to_at_least_2d()\n assert scalar_batch_2d.dist.batch_shape == (1,)\n assert scalar_batch_2d.a.shape == scalar_batch_2d.b.shape == (1, 1)\n\n\nclass TestAssignment:\n @staticmethod\n def test_assign_full_length_array() -> None:\n batch = Batch(a=[4, 5, 6], b=[7, 8, 9], c={\"d\": np.array([1, 2, 3])})\n batch.set_array_at_key(np.array([1, 2, 3]), \"a\")\n batch.set_array_at_key(np.array([4, 5, 6]), \"new_key\")\n assert np.array_equal(batch.a, np.array([1, 2, 3]))\n assert np.array_equal(batch.new_key, np.array([4, 5, 6]))\n\n # other keys are not affected\n assert np.array_equal(batch.b, np.array([7, 8, 9]))\n assert np.array_equal(batch.c.d, np.array([1, 2, 3]))\n\n with pytest.raises(ValueError):\n # wrong length\n batch.set_array_at_key(np.array([1, 2]), \"a\")\n\n @staticmethod\n def test_assign_subarray_existing_key() -> None:\n batch = Batch(a=[4, 5, 6], b=[7, 8, 9], c={\"d\": np.array([1, 2, 3])})\n batch.set_array_at_key(np.array([1, 2]), \"a\", index=[0, 1])\n assert np.array_equal(batch.a, np.array([1, 2, 6]))\n batch.set_array_at_key(np.array([10, 12]), \"a\", index=slice(0, 2))\n assert np.array_equal(batch.a, np.array([10, 12, 6]))\n batch.set_array_at_key(np.array([1, 2]), \"a\", index=[0, 2])\n assert np.array_equal(batch.a, np.array([1, 12, 2]))\n batch.set_array_at_key(np.array([1, 2]), \"a\", index=[2, 0])\n assert np.array_equal(batch.a, np.array([2, 12, 1]))\n batch.set_array_at_key(np.array([1, 2, 3]), \"a\", index=[2, 1, 0])\n assert np.array_equal(batch.a, np.array([3, 2, 1]))\n\n with pytest.raises(IndexError):\n # Index out of bounds\n batch.set_array_at_key(np.array([1, 2]), \"a\", index=[10, 11])\n\n # other keys are not affected\n assert np.array_equal(batch.b, np.array([7, 8, 9]))\n assert np.array_equal(batch.c.d, np.array([1, 2, 3]))\n\n @staticmethod\n def test_assign_subarray_new_key() -> None:\n batch = Batch(a=[4, 5, 6], b=[7, 8, 9], c={\"d\": np.array([1, 2, 3])})\n batch.set_array_at_key(np.array([1, 2]), \"new_key\", index=[0, 1], default_value=0)\n assert np.array_equal(batch.new_key, np.array([1, 2, 0]))\n # with float, None can be cast to NaN\n batch.set_array_at_key(np.array([1.0, 2.0]), \"new_key2\", index=[0, 1])\n assert np.array_equal(batch.new_key2, np.array([1.0, 2.0, np.nan]), equal_nan=True)\n\n @staticmethod\n def test_isnull() -> None:\n batch = Batch(a=[4, 5, 6], b=[7, 8, None], c={\"d\": np.array([1, None, 3])})\n batch_isnan = batch.isnull()\n assert not batch_isnan.a.any()\n assert batch_isnan.b[2]\n assert not batch_isnan.b[:2].any()\n assert np.array_equal(batch_isnan.c.d, np.array([False, True, False]))\n\n @staticmethod\n def test_hasnull() -> None:\n batch = Batch(a=[4, 5, 6], b=[7, 8, None], c={\"d\": np.array([1, 2, 3])})\n assert batch.hasnull()\n batch = Batch(a=[4, 5, 6], b=[7, 8, 9], c={\"d\": np.array([1, 2, 3])})\n assert not batch.hasnull()\n batch = Batch(a=[4, 5, 6], c={\"d\": np.array([1, None, 3])})\n assert batch.hasnull()\n\n @staticmethod\n def test_dropnull() -> None:\n batch = Batch(a=[4, 5, 6], b=[7, 8, None], c={\"d\": np.array([None, 2.1, 3.0])})\n assert batch.dropnull() == Batch(\n a=[5],\n b=[8],\n c={\"d\": np.array([2.1])},\n ).apply_values_transform(\n np.atleast_1d,\n )\n batch2 = Batch(a=[4, 5, 6, 7], b=[7, 8, None, 10], c={\"d\": np.array([None, 2, 3, 4])})\n assert batch2.dropnull() == Batch(a=[5, 7], b=[8, 10], c={\"d\": np.array([2, 4])})\n batch_no_nan = Batch(a=[4, 5, 6], b=[7, 8, 9], c={\"d\": np.array([1, 2, 3])})\n assert batch_no_nan.dropnull() == batch_no_nan\n\n\nclass TestSlicing:\n # TODO: parametrize with other dists\n @staticmethod\n def test_slice_distribution() -> None:\n cat_probs = torch.randint(1, 10, (10, 3))\n dist = Categorical(probs=cat_probs)\n batch = Batch(dist=dist)\n selected_idx = [1, 3]\n sliced_batch = batch[selected_idx]\n sliced_probs = cat_probs[selected_idx]\n assert torch.allclose(sliced_batch.dist.probs, Categorical(probs=sliced_probs).probs)\n assert torch.allclose(\n Categorical(probs=sliced_probs).probs,\n get_sliced_dist(dist, selected_idx).probs,\n )\n # retrieving a single index\n assert torch.allclose(batch[0].dist.probs, dist.probs[0])\n\n @staticmethod\n def test_getitem_with_int_gives_scalars() -> None:\n batch = Batch(a=[1, 2], b=Batch(c=[3, 4]))\n batch_sliced = batch[0]\n assert batch_sliced.a == np.array(1)\n assert batch_sliced.b.c == np.array(3)\n\n @staticmethod\n @pytest.mark.parametrize(\"index\", ([0, 1], np.array([0, 1]), torch.tensor([0, 1]), slice(0, 2)))\n def test_getitem_with_slice_gives_subslice(index: IndexType) -> None:\n batch = Batch(a=[1, 2, 3], b=Batch(c=torch.tensor([4, 5, 6])))\n batch_sliced = batch[index]\n assert (batch_sliced.a == batch.a[index]).all()\n assert (batch_sliced.b.c == batch.b.c[index]).all()\n\n @staticmethod\n def test_len_batch_with_dist() -> None:\n batch_with_dist = Batch(a=[1, 2, 3], dist=Categorical(torch.ones((3, 3))), b=None)\n batch_with_dist_sliced = batch_with_dist[:2]\n assert batch_with_dist_sliced.b is None\n assert len(batch_with_dist_sliced) == 2\n assert np.array_equal(batch_with_dist_sliced.a, np.array([1, 2]))\n assert torch.allclose(\n batch_with_dist_sliced.dist.probs,\n Categorical(torch.ones(2, 3)).probs,\n )\n\n with pytest.raises(TypeError):\n # scalar batches have no len\n len(batch_with_dist[0])\n" }, { "path": "test/base/test_buffer.py", "content": "import os\nimport pickle\nimport tempfile\nfrom typing import cast\n\nimport h5py\nimport numpy as np\nimport numpy.typing as npt\nimport pytest\nimport torch\n\nfrom test.base.env import MoveToRightEnv, MyGoalEnv\nfrom tianshou.data import (\n Batch,\n CachedReplayBuffer,\n HERReplayBuffer,\n HERVectorReplayBuffer,\n PrioritizedReplayBuffer,\n PrioritizedVectorReplayBuffer,\n ReplayBuffer,\n SegmentTree,\n VectorReplayBuffer,\n)\nfrom tianshou.data.types import RolloutBatchProtocol\nfrom tianshou.data.utils.converter import to_hdf5\n\n\ndef test_replaybuffer(size: int = 10, bufsize: int = 20) -> None:\n env = MoveToRightEnv(size)\n buf = ReplayBuffer(bufsize)\n buf.update(buf)\n assert str(buf) == buf.__class__.__name__ + \"()\"\n obs, _ = env.reset()\n action_list = [1] * 5 + [0] * 10 + [1] * 10\n for i, act in enumerate(action_list):\n obs_next, rew, terminated, truncated, info = env.step(act)\n buf.add(\n Batch(\n obs=obs,\n act=[act],\n rew=rew,\n terminated=terminated,\n truncated=truncated,\n obs_next=obs_next,\n info=info,\n ),\n )\n obs = obs_next\n assert len(buf) == min(bufsize, i + 1)\n assert buf.act.dtype == int\n assert buf.act.shape == (bufsize, 1)\n data, indices = buf.sample(bufsize * 2)\n assert isinstance(data, Batch)\n assert (indices < len(buf)).all()\n assert (data.obs < size).all()\n assert (data.done >= 0).all()\n assert (data.done <= 1).all()\n assert (data.terminated >= 0).all()\n assert (data.terminated <= 1).all()\n assert (data.truncated >= 0).all()\n assert (data.truncated <= 1).all()\n b = ReplayBuffer(size=10)\n # neg bsz should return empty index\n assert b.sample_indices(-1).tolist() == []\n ptr, ep_rew, ep_len, ep_idx = b.add(\n Batch(\n obs=1,\n act=1,\n rew=1,\n terminated=1,\n truncated=0,\n obs_next=\"str\",\n info={\"a\": 3, \"b\": {\"c\": 5.0}},\n ),\n )\n assert b.obs[0] == 1\n assert b.done[0]\n assert b.terminated[0]\n assert not b.truncated[0]\n assert b.obs_next[0] == \"str\"\n assert np.all(b.obs[1:] == 0)\n assert np.all(b.obs_next[1:] == np.array(None))\n assert b.info.a[0] == 3\n assert b.info.a.dtype == int\n assert np.all(b.info.a[1:] == 0)\n assert b.info.b.c[0] == 5.0\n assert b.info.b.c.dtype == float\n assert np.all(b.info.b.c[1:] == 0.0)\n assert ptr.shape == (1,)\n assert ptr[0] == 0\n assert ep_rew.shape == (1,)\n assert ep_rew[0] == 1\n assert ep_len.shape == (1,)\n assert ep_len[0] == 1\n assert ep_idx.shape == (1,)\n assert ep_idx[0] == 0\n # test extra keys pop up, the buffer should handle it dynamically\n batch = cast(\n RolloutBatchProtocol,\n Batch(\n obs=2,\n act=2,\n rew=2,\n terminated=0,\n truncated=0,\n obs_next=\"str2\",\n info={\"a\": 4, \"d\": {\"e\": -np.inf}},\n ),\n )\n b.add(batch)\n info_keys = [\"a\", \"b\", \"d\"]\n assert set(b.info.keys()) == set(info_keys)\n assert b.info.a[1] == 4\n assert b.info.b.c[1] == 0\n assert b.info.d.e[1] == -np.inf\n # test batch-style adding method, where len(batch) == 1\n batch.done = np.array([True])\n batch.terminated = np.array([False])\n batch.truncated = np.array([True])\n batch.info.e = np.zeros([1, 4]) # type: ignore\n batch: RolloutBatchProtocol = Batch.stack([batch])\n ptr, ep_rew, ep_len, ep_idx = b.add(batch, buffer_ids=[0])\n assert ptr.shape == (1,)\n assert ptr[0] == 2\n assert ep_rew.shape == (1,)\n assert ep_rew[0] == 4\n assert ep_len.shape == (1,)\n assert ep_len[0] == 2\n assert ep_idx.shape == (1,)\n assert ep_idx[0] == 1\n assert set(b.info.keys()) == {*info_keys, \"e\"}\n assert b.info.e.shape == (b.maxsize, 1, 4)\n with pytest.raises(IndexError):\n b[22]\n # test prev / next\n assert np.all(b.prev(np.array([0, 1, 2])) == [0, 1, 1])\n assert np.all(b.next(np.array([0, 1, 2])) == [0, 2, 2])\n batch.done = [0]\n b.add(batch, buffer_ids=[0])\n assert np.all(b.prev(np.array([0, 1, 2, 3])) == [0, 1, 1, 3])\n assert np.all(b.next(np.array([0, 1, 2, 3])) == [0, 2, 2, 3])\n\n\ndef test_ignore_obs_next(size: int = 10) -> None:\n # Issue 82\n buf = ReplayBuffer(size, ignore_obs_next=True)\n for i in range(size):\n buf.add(\n cast(\n RolloutBatchProtocol,\n Batch(\n obs={\n \"mask1\": np.array([i, 1, 1, 0, 0]),\n \"mask2\": np.array([i + 4, 0, 1, 0, 0]),\n \"mask\": i,\n },\n act={\"act_id\": i, \"position_id\": i + 3},\n rew=i,\n terminated=i % 3 == 0,\n truncated=False,\n info={\"if\": i},\n ),\n ),\n )\n indices = np.arange(len(buf))\n orig = np.arange(len(buf))\n data = buf[indices]\n data2 = buf[indices]\n assert isinstance(data, Batch)\n assert isinstance(data2, Batch)\n assert np.allclose(indices, orig)\n assert hasattr(data.obs_next, \"mask\") and hasattr(\n data2.obs_next,\n \"mask\",\n ), \"Both `data.obs_next` and `data2.obs_next` must have attribute `mask`.\"\n assert np.allclose(data.obs_next.mask, data2.obs_next.mask)\n assert np.allclose(data.obs_next.mask, [0, 2, 3, 3, 5, 6, 6, 8, 9, 9])\n buf.stack_num = 4\n data = buf[indices]\n data2 = buf[indices]\n assert hasattr(data.obs_next, \"mask\") and hasattr(\n data2.obs_next,\n \"mask\",\n ), \"Both `data.obs_next` and `data2.obs_next` must have attribute `mask`.\"\n assert np.allclose(data.obs_next.mask, data2.obs_next.mask)\n assert np.allclose(\n data.obs_next.mask,\n np.array(\n [\n [0, 0, 0, 0],\n [1, 1, 1, 2],\n [1, 1, 2, 3],\n [1, 1, 2, 3],\n [4, 4, 4, 5],\n [4, 4, 5, 6],\n [4, 4, 5, 6],\n [7, 7, 7, 8],\n [7, 7, 8, 9],\n [7, 7, 8, 9],\n ],\n ),\n )\n assert np.allclose(data[\"info\"][\"if\"], data2[\"info\"][\"if\"])\n assert np.allclose(\n data[\"info\"][\"if\"],\n np.array(\n [\n [0, 0, 0, 0],\n [1, 1, 1, 1],\n [1, 1, 1, 2],\n [1, 1, 2, 3],\n [4, 4, 4, 4],\n [4, 4, 4, 5],\n [4, 4, 5, 6],\n [7, 7, 7, 7],\n [7, 7, 7, 8],\n [7, 7, 8, 9],\n ],\n ),\n )\n assert data.obs_next\n\n\ndef test_stack(size: int = 5, bufsize: int = 9, stack_num: int = 4, cached_num: int = 3) -> None:\n env = MoveToRightEnv(size)\n buf = ReplayBuffer(bufsize, stack_num=stack_num)\n buf2 = ReplayBuffer(bufsize, stack_num=stack_num, sample_avail=True)\n buf3 = ReplayBuffer(bufsize, stack_num=stack_num, save_only_last_obs=True)\n obs, info = env.reset(options={\"state\": 1})\n for _ in range(16):\n obs_next, rew, terminated, truncated, info = env.step(1)\n done = terminated or truncated\n buf.add(\n cast(\n RolloutBatchProtocol,\n Batch(\n obs=obs,\n act=1,\n rew=rew,\n terminated=terminated,\n truncated=truncated,\n info=info,\n ),\n ),\n )\n buf2.add(\n cast(\n RolloutBatchProtocol,\n Batch(\n obs=obs,\n act=1,\n rew=rew,\n terminated=terminated,\n truncated=truncated,\n info=info,\n ),\n ),\n )\n buf3.add(\n cast(\n RolloutBatchProtocol,\n Batch(\n obs=[obs, obs, obs],\n act=1,\n rew=rew,\n terminated=terminated,\n truncated=truncated,\n obs_next=[obs, obs],\n info=info,\n ),\n ),\n )\n obs = obs_next\n if done:\n obs, info = env.reset(options={\"state\": 1})\n indices = np.arange(len(buf))\n assert np.allclose(\n buf.get(indices, \"obs\")[..., 0],\n [\n [1, 1, 1, 2],\n [1, 1, 2, 3],\n [1, 2, 3, 4],\n [1, 1, 1, 1],\n [1, 1, 1, 2],\n [1, 1, 2, 3],\n [1, 2, 3, 4],\n [4, 4, 4, 4],\n [1, 1, 1, 1],\n ],\n )\n assert np.allclose(buf.get(indices, \"obs\"), buf3.get(indices, \"obs\"))\n assert np.allclose(buf.get(indices, \"obs\"), buf3.get(indices, \"obs_next\"))\n _, indices = buf2.sample(0)\n assert indices.tolist() == [2, 6]\n _, indices = buf2.sample(1)\n assert indices[0] in [2, 6]\n batch, indices = buf2.sample(-1) # neg bsz -> no data\n assert indices.tolist() == []\n assert len(batch) == 0\n with pytest.raises(IndexError):\n buf[bufsize * 2]\n\n\ndef test_prioritized_replaybuffer(size: int = 32, bufsize: int = 15) -> None:\n env = MoveToRightEnv(size)\n buf = PrioritizedReplayBuffer(bufsize, 0.5, 0.5)\n buf2 = PrioritizedVectorReplayBuffer(bufsize, buffer_num=3, alpha=0.5, beta=0.5)\n obs, _ = env.reset()\n action_list = [1] * 5 + [0] * 10 + [1] * 10\n for i, act in enumerate(action_list):\n obs_next, rew, terminated, truncated, info = env.step(act)\n batch = cast(\n RolloutBatchProtocol,\n Batch(\n obs=obs,\n act=act,\n rew=rew,\n terminated=terminated,\n truncated=truncated,\n obs_next=obs_next,\n info=info,\n policy=np.random.randn() - 0.5,\n ),\n )\n batch_stack: RolloutBatchProtocol = Batch.stack([batch, batch, batch])\n buf.add(Batch.stack([batch]), buffer_ids=[0])\n buf2.add(batch_stack, buffer_ids=[0, 1, 2])\n obs = obs_next\n data, indices = buf.sample(len(buf) // 2)\n if len(buf) // 2 == 0:\n assert len(data) == len(buf)\n else:\n assert len(data) == len(buf) // 2\n assert len(buf) == min(bufsize, i + 1)\n assert len(buf2) == min(bufsize, 3 * (i + 1))\n # check single buffer's data\n assert buf.info.key.shape == (buf.maxsize,)\n assert buf.rew.dtype == float\n assert buf.done.dtype == bool\n assert buf.terminated.dtype == bool\n assert buf.truncated.dtype == bool\n data, indices = buf.sample(len(buf) // 2)\n buf.update_weight(indices, -data.weight / 2)\n assert np.allclose(buf.weight[indices], np.abs(-data.weight / 2) ** buf._alpha)\n # check multi buffer's data\n assert np.allclose(buf2[np.arange(buf2.maxsize)].weight, 1)\n batch_sample, indices = buf2.sample(10)\n buf2.update_weight(indices, batch_sample.weight * 0)\n weight = buf2[np.arange(buf2.maxsize)].weight\n assert isinstance(weight, np.ndarray)\n mask = np.isin(np.arange(buf2.maxsize), indices)\n selected_weight = weight[mask]\n unselected_weight = weight[~mask]\n assert np.all(selected_weight == selected_weight[0])\n assert np.all(unselected_weight == unselected_weight[0])\n assert unselected_weight[0] < selected_weight[0]\n assert selected_weight[0] <= 1\n\n\ndef test_herreplaybuffer(size: int = 10, bufsize: int = 100, sample_sz: int = 4) -> None:\n env_size = size\n env = MyGoalEnv(env_size, array_state=True)\n\n def compute_reward_fn(ag: np.ndarray, g: np.ndarray) -> np.ndarray:\n return env.compute_reward_fn(ag, g, {})\n\n buf = HERReplayBuffer(bufsize, compute_reward_fn=compute_reward_fn, horizon=30, future_k=8)\n buf2 = HERVectorReplayBuffer(\n bufsize,\n buffer_num=3,\n compute_reward_fn=compute_reward_fn,\n horizon=30,\n future_k=8,\n )\n # Apply her on every episodes sampled (Hacky but necessary for deterministic test)\n buf.future_p = 1\n for buf2_buf in buf2.buffers:\n buf2_buf.future_p = 1\n\n obs, _ = env.reset()\n action_list = [1] * 5 + [0] * 10 + [1] * 10\n for i, act in enumerate(action_list):\n obs_next, rew, terminated, truncated, info = env.step(act)\n batch = cast(\n RolloutBatchProtocol,\n Batch(\n obs=obs,\n act=[act],\n rew=rew,\n terminated=terminated,\n truncated=truncated,\n obs_next=obs_next,\n info=info,\n ),\n )\n buf.add(batch)\n buf2.add(Batch.stack([batch, batch, batch]), buffer_ids=[0, 1, 2])\n obs = obs_next\n assert len(buf) == min(bufsize, i + 1)\n assert len(buf2) == min(bufsize, 3 * (i + 1))\n\n batch_sample, indices = buf.sample(sample_sz)\n\n # Check that goals are the same for the episode (only 1 ep in buffer)\n tmp_indices = indices.copy()\n for _ in range(2 * env_size):\n obs_in_buf = cast(Batch, buf[tmp_indices].obs)\n obs_next_buf = cast(Batch, buf[tmp_indices].obs_next)\n rew_in_buf = buf[tmp_indices].rew\n g = obs_in_buf.desired_goal.reshape(sample_sz, -1)[:, 0]\n ag_next = obs_next_buf.achieved_goal.reshape(sample_sz, -1)[:, 0]\n g_next = obs_next_buf.desired_goal.reshape(sample_sz, -1)[:, 0]\n assert np.all(g == g[0])\n assert np.all(g_next == g_next[0])\n assert np.all(rew_in_buf == (ag_next == g).astype(np.float32))\n tmp_indices = buf.next(tmp_indices)\n\n # Check that goals are correctly restored\n buf._restore_cache()\n tmp_indices = indices.copy()\n for _ in range(2 * env_size):\n obs_in_buf = cast(Batch, buf[tmp_indices].obs)\n obs_next_buf = cast(Batch, buf[tmp_indices].obs_next)\n g = obs_in_buf.desired_goal.reshape(sample_sz, -1)[:, 0]\n g_next = obs_next_buf.desired_goal.reshape(sample_sz, -1)[:, 0]\n assert np.all(g == env_size)\n assert np.all(g_next == g_next[0])\n assert np.all(g == g[0])\n tmp_indices = buf.next(tmp_indices)\n\n # Test vector buffer\n batch_sample, indices = buf2.sample(sample_sz)\n\n # Check that goals are the same for the episode (only 1 ep in buffer)\n tmp_indices = indices.copy()\n for _ in range(2 * env_size):\n obs_in_buf = cast(Batch, buf2[tmp_indices].obs)\n obs_next_buf = cast(Batch, buf2[tmp_indices].obs_next)\n rew_buf = buf2[tmp_indices].rew\n g = obs_in_buf.desired_goal.reshape(sample_sz, -1)[:, 0]\n ag_next = obs_next_buf.achieved_goal.reshape(sample_sz, -1)[:, 0]\n g_next = obs_next_buf.desired_goal.reshape(sample_sz, -1)[:, 0]\n assert np.all(g == g_next)\n assert np.all(rew_buf == (ag_next == g).astype(np.float32))\n tmp_indices = buf2.next(tmp_indices)\n\n # Check that goals are correctly restored\n buf2._restore_cache()\n tmp_indices = indices.copy()\n for _ in range(2 * env_size):\n obs_in_buf = cast(Batch, buf2[tmp_indices].obs)\n obs_next_buf = cast(Batch, buf2[tmp_indices].obs_next)\n g = obs_in_buf.desired_goal.reshape(sample_sz, -1)[:, 0]\n g_next = obs_next_buf.desired_goal.reshape(sample_sz, -1)[:, 0]\n assert np.all(g == env_size)\n assert np.all(g_next == g_next[0])\n assert np.all(g == g[0])\n tmp_indices = buf2.next(tmp_indices)\n\n # Test handling cycled indices\n env_size = size\n bufsize = 15\n env = MyGoalEnv(size=env_size, array_state=False)\n\n buf = HERReplayBuffer(bufsize, compute_reward_fn=compute_reward_fn, horizon=30, future_k=8)\n buf.future_p = 1\n for ep_len in [5, 10]:\n obs, _ = env.reset()\n for i in range(ep_len):\n act = 1\n obs_next, rew, terminated, truncated, info = env.step(act)\n batch = cast(\n RolloutBatchProtocol,\n Batch(\n obs=obs,\n act=[act],\n rew=rew,\n terminated=(i == ep_len - 1),\n truncated=(i == ep_len - 1),\n obs_next=obs_next,\n info=info,\n ),\n )\n buf.add(batch)\n obs = obs_next\n batch_sample, indices = buf.sample(0)\n assert np.all(buf.obs.desired_goal[:5] == buf.obs.desired_goal[0])\n assert np.all(buf.obs.desired_goal[5:10] == buf.obs.desired_goal[5])\n assert np.all(buf.obs.desired_goal[5:] == buf.obs.desired_goal[14]) # (same ep)\n assert np.all(buf.obs.desired_goal[0] != buf.obs.desired_goal[5]) # (diff ep)\n\n # Another test case for cycled indices\n env_size = 99\n bufsize = 15\n env = MyGoalEnv(env_size, array_state=False)\n buf = HERReplayBuffer(bufsize, compute_reward_fn=compute_reward_fn, horizon=30, future_k=8)\n buf.future_p = 1\n for x, ep_len in enumerate([10, 20]):\n obs, _ = env.reset()\n for i in range(ep_len):\n act = 1\n obs_next, rew, terminated, truncated, info = env.step(act)\n batch = cast(\n RolloutBatchProtocol,\n Batch(\n obs=obs,\n act=[act],\n rew=rew,\n terminated=(i == ep_len - 1),\n truncated=(i == ep_len - 1),\n obs_next=obs_next,\n info=info,\n ),\n )\n if x == 1 and obs[\"observation\"] < 10:\n obs = obs_next\n continue\n buf.add(batch)\n obs = obs_next\n buf._restore_cache()\n sample_indices = np.array([10]) # Suppose the sampled indices is [10]\n buf.rewrite_transitions(sample_indices)\n assert int(buf.obs.desired_goal[10][0]) in [11, 12, 13, 14, 15, 16, 17, 18, 19, 20]\n\n\ndef test_update() -> None:\n buf1 = ReplayBuffer(4, stack_num=2)\n buf2 = ReplayBuffer(4, stack_num=2)\n for i in range(5):\n buf1.add(\n cast(\n RolloutBatchProtocol,\n Batch(\n obs=np.array([i]),\n act=float(i),\n rew=i * i,\n terminated=i % 2 == 0,\n truncated=False,\n info={\"incident\": \"found\"},\n ),\n ),\n )\n assert len(buf1) > len(buf2)\n buf2.update(buf1)\n assert len(buf1) == len(buf2)\n assert (buf2.obs[0] == buf1.obs[1]).all()\n assert (buf2.obs[-1] == buf1.obs[0]).all()\n b = CachedReplayBuffer(ReplayBuffer(10), 4, 5)\n with pytest.raises(NotImplementedError):\n b.update(b)\n\n\ndef test_segtree() -> None:\n realop = np.sum\n # small test\n actual_len = 8\n tree = SegmentTree(actual_len) # 1-15. 8-15 are leaf nodes\n assert len(tree) == actual_len\n assert np.all([tree[i] == 0.0 for i in range(actual_len)])\n with pytest.raises(IndexError):\n tree[actual_len]\n naive = np.zeros(actual_len)\n for _ in range(1000):\n # random choose a place to perform single update\n index: int | np.ndarray = np.random.randint(actual_len)\n value: float | np.ndarray = np.random.rand()\n naive[index] = value\n tree[index] = value\n for i in range(actual_len):\n for j in range(i + 1, actual_len):\n ref = realop(naive[i:j])\n out = tree.reduce(i, j)\n assert np.allclose(ref, out), (ref, out)\n assert np.allclose(tree.reduce(start=1), realop(naive[1:]))\n assert np.allclose(tree.reduce(end=-1), realop(naive[:-1]))\n # batch setitem\n for _ in range(1000):\n index = np.random.choice(actual_len, size=4)\n value = np.random.rand(4)\n naive[index] = value\n tree[index] = value\n assert np.allclose(realop(naive), tree.reduce())\n for _ in range(10):\n left = np.random.randint(actual_len)\n right = np.random.randint(left + 1, actual_len + 1)\n assert np.allclose(realop(naive[left:right]), tree.reduce(left, right))\n # large test\n actual_len = 16384\n tree = SegmentTree(actual_len)\n naive = np.zeros([actual_len])\n for _ in range(1000):\n index = np.random.choice(actual_len, size=64)\n value = np.random.rand(64)\n naive[index] = value\n tree[index] = value\n assert np.allclose(realop(naive), tree.reduce())\n for _ in range(10):\n left = np.random.randint(actual_len)\n right = np.random.randint(left + 1, actual_len + 1)\n assert np.allclose(realop(naive[left:right]), tree.reduce(left, right))\n\n # test prefix-sum-idx\n actual_len = 8\n tree = SegmentTree(actual_len)\n naive = np.random.rand(actual_len)\n tree[np.arange(actual_len)] = naive\n for _ in range(1000):\n scalar = np.random.rand() * naive.sum()\n index = tree.get_prefix_sum_idx(scalar)\n assert naive[:index].sum() <= scalar <= naive[: index + 1].sum()\n # corner case here\n naive = np.ones(actual_len, int)\n tree[np.arange(actual_len)] = naive\n for scalar in range(actual_len):\n index = tree.get_prefix_sum_idx(scalar * 1.0)\n assert naive[:index].sum() <= scalar <= naive[: index + 1].sum()\n tree = SegmentTree(10)\n tree[np.arange(3)] = np.array([0.1, 0, 0.1])\n assert np.allclose(\n tree.get_prefix_sum_idx(np.array([0, 0.1, 0.1 + 1e-6, 0.2 - 1e-6])),\n [0, 0, 2, 2],\n )\n with pytest.raises(AssertionError):\n tree.get_prefix_sum_idx(0.2)\n # test large prefix-sum-idx\n actual_len = 16384\n tree = SegmentTree(actual_len)\n naive = np.random.rand(actual_len)\n tree[np.arange(actual_len)] = naive\n for _ in range(1000):\n scalar = np.random.rand() * naive.sum()\n index = tree.get_prefix_sum_idx(scalar)\n assert naive[:index].sum() <= scalar <= naive[: index + 1].sum()\n\n\ndef test_pickle() -> None:\n size = 100\n vbuf = ReplayBuffer(size, stack_num=2)\n pbuf = PrioritizedReplayBuffer(size, 0.6, 0.4)\n rew = np.array([1, 1])\n for i in range(4):\n vbuf.add(\n cast(\n RolloutBatchProtocol,\n Batch(\n obs=Batch(index=np.array([i])),\n act=0,\n rew=rew,\n terminated=0,\n truncated=0,\n ),\n ),\n )\n for i in range(5):\n pbuf.add(\n cast(\n RolloutBatchProtocol,\n Batch(\n obs=Batch(index=np.array([i])),\n act=2,\n rew=rew,\n terminated=0,\n truncated=0,\n info=np.random.rand(),\n ),\n ),\n )\n # save & load\n _vbuf = pickle.loads(pickle.dumps(vbuf))\n _pbuf = pickle.loads(pickle.dumps(pbuf))\n assert len(_vbuf) == len(vbuf)\n assert np.allclose(_vbuf.act, vbuf.act)\n assert len(_pbuf) == len(pbuf)\n assert np.allclose(_pbuf.act, pbuf.act)\n # make sure the meta var is identical\n assert _vbuf.stack_num == vbuf.stack_num\n assert np.allclose(_pbuf.weight[np.arange(len(_pbuf))], pbuf.weight[np.arange(len(pbuf))])\n\n\ndef test_hdf5() -> None:\n size = 100\n buffers = {\n \"array\": ReplayBuffer(size, stack_num=2),\n \"prioritized\": PrioritizedReplayBuffer(size, 0.6, 0.4),\n }\n buffer_types = {k: b.__class__ for k, b in buffers.items()}\n device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n info_t = torch.tensor([1.0]).to(device)\n for i in range(4):\n kwargs = {\n \"obs\": Batch(index=np.array([i])),\n \"act\": i,\n \"rew\": np.array([1, 2]),\n \"terminated\": i % 3 == 2,\n \"truncated\": False,\n \"done\": i % 3 == 2,\n \"info\": {\"number\": {\"n\": i, \"t\": info_t}, \"extra\": None},\n }\n buffers[\"array\"].add(cast(RolloutBatchProtocol, Batch(kwargs)))\n buffers[\"prioritized\"].add(cast(RolloutBatchProtocol, Batch(kwargs)))\n\n # save\n paths = {}\n for k, buf in buffers.items():\n f, path = tempfile.mkstemp(suffix=\".hdf5\")\n os.close(f)\n buf.save_hdf5(path)\n paths[k] = path\n\n # load replay buffer\n _buffers = {k: buffer_types[k].load_hdf5(paths[k]) for k in paths}\n\n # compare\n for k in buffers:\n assert len(_buffers[k]) == len(buffers[k])\n assert np.allclose(_buffers[k].act, buffers[k].act)\n assert _buffers[k].stack_num == buffers[k].stack_num\n assert _buffers[k].maxsize == buffers[k].maxsize\n assert np.all(_buffers[k]._indices == buffers[k]._indices)\n for k in [\"array\", \"prioritized\"]:\n assert _buffers[k]._insertion_idx == buffers[k]._insertion_idx\n assert isinstance(buffers[k].get(0, \"info\"), Batch)\n assert isinstance(_buffers[k].get(0, \"info\"), Batch)\n for k in [\"array\"]:\n assert np.all(buffers[k][:][\"info\"].number.n == _buffers[k][:][\"info\"].number.n)\n assert np.all(buffers[k][:][\"info\"][\"extra\"] == _buffers[k][:][\"info\"][\"extra\"])\n\n # raise exception when value cannot be pickled\n data = {\"not_supported\": lambda x: x * x}\n grp = h5py.Group\n with pytest.raises(NotImplementedError):\n to_hdf5(data, grp) # type: ignore\n # ndarray with data type not supported by HDF5 that cannot be pickled\n data = {\"not_supported\": np.array(lambda x: x * x)}\n grp = h5py.Group\n with pytest.raises(RuntimeError):\n to_hdf5(data, grp) # type: ignore\n\n\ndef test_replaybuffermanager() -> None:\n buf = VectorReplayBuffer(20, 4)\n batch = cast(\n RolloutBatchProtocol,\n Batch(\n obs=[1, 2, 3],\n act=[1, 2, 3],\n rew=[1, 2, 3],\n terminated=[0, 0, 1],\n truncated=[0, 0, 0],\n ),\n )\n ptr, ep_rew, ep_len, ep_idx = buf.add(batch, buffer_ids=[0, 1, 2])\n assert np.all(ep_len == [0, 0, 1])\n assert np.all(ep_rew == [0, 0, 3])\n assert np.all(ptr == [0, 5, 10])\n assert np.all(ep_idx == [0, 5, 10])\n with pytest.raises(NotImplementedError):\n # ReplayBufferManager cannot be updated\n buf.update(buf)\n # sample index / prev / next / unfinished_index\n indices = buf.sample_indices(11000)\n assert np.bincount(indices)[[0, 5, 10]].min() >= 3000 # uniform sample\n batch, indices = buf.sample(0)\n assert np.allclose(indices, [0, 5, 10])\n indices_prev = buf.prev(indices)\n assert np.allclose(indices_prev, indices), indices_prev\n indices_next = buf.next(indices)\n assert np.allclose(indices_next, indices), indices_next\n assert np.allclose(buf.unfinished_index(), [0, 5])\n buf.add(\n cast(\n RolloutBatchProtocol,\n Batch(obs=[4], act=[4], rew=[4], terminated=[1], truncated=[0]),\n ),\n buffer_ids=[3],\n )\n assert np.allclose(buf.unfinished_index(), [0, 5])\n batch, indices = buf.sample(10)\n batch, indices = buf.sample(0)\n assert np.allclose(indices, [0, 5, 10, 15])\n indices_prev = buf.prev(indices)\n assert np.allclose(indices_prev, indices), indices_prev\n indices_next = buf.next(indices)\n assert np.allclose(indices_next, indices), indices_next\n data = np.array([0, 0, 0, 0])\n buf.add(\n cast(\n RolloutBatchProtocol,\n Batch(obs=data, act=data, rew=data, terminated=data, truncated=data),\n ),\n buffer_ids=[0, 1, 2, 3],\n )\n buf.add(\n cast(\n RolloutBatchProtocol,\n Batch(obs=data, act=data, rew=data, terminated=1 - data, truncated=data),\n ),\n buffer_ids=[0, 1, 2, 3],\n )\n assert len(buf) == 12\n buf.add(\n cast(\n RolloutBatchProtocol,\n Batch(obs=data, act=data, rew=data, terminated=data, truncated=data),\n ),\n buffer_ids=[0, 1, 2, 3],\n )\n buf.add(\n cast(\n RolloutBatchProtocol,\n Batch(obs=data, act=data, rew=data, terminated=[0, 1, 0, 1], truncated=data),\n ),\n buffer_ids=[0, 1, 2, 3],\n )\n assert len(buf) == 20\n indices = buf.sample_indices(120000)\n assert np.bincount(indices).min() >= 5000\n batch, indices = buf.sample(10)\n indices = buf.sample_indices(0)\n assert np.allclose(indices, np.arange(len(buf)))\n # check the actual data stored in buf._meta\n assert np.allclose(\n buf.done,\n [\n 0,\n 0,\n 1,\n 0,\n 0,\n 0,\n 0,\n 1,\n 0,\n 1,\n 1,\n 0,\n 1,\n 0,\n 0,\n 1,\n 0,\n 1,\n 0,\n 1,\n ],\n )\n assert np.allclose(\n buf.prev(indices),\n [\n 0,\n 0,\n 1,\n 3,\n 3,\n 5,\n 5,\n 6,\n 8,\n 8,\n 10,\n 11,\n 11,\n 13,\n 13,\n 15,\n 16,\n 16,\n 18,\n 18,\n ],\n )\n assert np.allclose(\n buf.next(indices),\n [\n 1,\n 2,\n 2,\n 4,\n 4,\n 6,\n 7,\n 7,\n 9,\n 9,\n 10,\n 12,\n 12,\n 14,\n 14,\n 15,\n 17,\n 17,\n 19,\n 19,\n ],\n )\n assert np.allclose(buf.unfinished_index(), [4, 14])\n ptr, ep_rew, ep_len, ep_idx = buf.add(\n cast(\n RolloutBatchProtocol,\n Batch(obs=[1], act=[1], rew=[1], terminated=[1], truncated=[0]),\n ),\n buffer_ids=[2],\n )\n assert np.all(ep_len == [3])\n assert np.all(ep_rew == [1])\n assert np.all(ptr == [10])\n assert np.all(ep_idx == [13])\n assert np.allclose(buf.unfinished_index(), [4])\n indices = np.array(sorted(buf.sample_indices(0)))\n assert np.allclose(indices, np.arange(len(buf)))\n assert np.allclose(\n buf.prev(indices),\n [\n 0,\n 0,\n 1,\n 3,\n 3,\n 5,\n 5,\n 6,\n 8,\n 8,\n 14,\n 11,\n 11,\n 13,\n 13,\n 15,\n 16,\n 16,\n 18,\n 18,\n ],\n )\n assert np.allclose(\n buf.next(indices),\n [\n 1,\n 2,\n 2,\n 4,\n 4,\n 6,\n 7,\n 7,\n 9,\n 9,\n 10,\n 12,\n 12,\n 14,\n 10,\n 15,\n 17,\n 17,\n 19,\n 19,\n ],\n )\n # corner case: list, int and -1\n assert buf.prev(-1) == buf.prev(np.array([buf.maxsize - 1]))[0]\n assert buf.next(-1) == buf.next(np.array([buf.maxsize - 1]))[0]\n batch = buf._meta\n batch.info = np.ones(buf.maxsize)\n buf.set_batch(batch)\n assert np.allclose(buf.buffers[-1].info, [1] * 5)\n assert buf.sample_indices(-1).tolist() == []\n assert np.array([ReplayBuffer(0, ignore_obs_next=True)]).dtype == object\n\n\ndef test_cachedbuffer() -> None:\n buf = CachedReplayBuffer(ReplayBuffer(10), 4, 5)\n assert buf.sample_indices(0).tolist() == []\n # check the normal function/usage/storage in CachedReplayBuffer\n ptr, ep_rew, ep_len, ep_idx = buf.add(\n cast(\n RolloutBatchProtocol,\n Batch(obs=[1], act=[1], rew=[1], terminated=[0], truncated=[0]),\n ),\n buffer_ids=[1],\n )\n obs = np.zeros(buf.maxsize)\n obs[15] = 1\n indices = buf.sample_indices(0)\n assert np.allclose(indices, [15])\n assert np.allclose(buf.prev(indices), [15])\n assert np.allclose(buf.next(indices), [15])\n assert np.allclose(buf.obs, obs)\n assert np.all(ep_len == [0])\n assert np.all(ep_rew == [0.0])\n assert np.all(ptr == [15])\n assert np.all(ep_idx == [15])\n ptr, ep_rew, ep_len, ep_idx = buf.add(\n cast(\n RolloutBatchProtocol,\n Batch(obs=[2], act=[2], rew=[2], terminated=[1], truncated=[0]),\n ),\n buffer_ids=[3],\n )\n obs[[0, 25]] = 2\n indices = buf.sample_indices(0)\n assert np.allclose(indices, [0, 15])\n assert np.allclose(buf.prev(indices), [0, 15])\n assert np.allclose(buf.next(indices), [0, 15])\n assert np.allclose(buf.obs, obs)\n assert np.all(ep_len == [1])\n assert np.all(ep_rew == [2.0])\n assert np.all(ptr == [0])\n assert np.all(ep_idx == [0])\n assert np.allclose(buf.unfinished_index(), [15])\n assert np.allclose(buf.sample_indices(0), [0, 15])\n ptr, ep_rew, ep_len, ep_idx = buf.add(\n cast(\n RolloutBatchProtocol,\n Batch(obs=[3, 4], act=[3, 4], rew=[3, 4], terminated=[0, 1], truncated=[0, 0]),\n ),\n buffer_ids=[3, 1], # TODO\n )\n assert np.all(ep_len == [0, 2])\n assert np.all(ep_rew == [0, 5.0])\n assert np.all(ptr == [25, 2])\n assert np.all(ep_idx == [25, 1])\n obs[[0, 1, 2, 15, 16, 25]] = [2, 1, 4, 1, 4, 3]\n assert np.allclose(buf.obs, obs)\n assert np.allclose(buf.unfinished_index(), [25])\n indices = buf.sample_indices(0)\n assert np.allclose(indices, [0, 1, 2, 25])\n assert np.allclose(buf.done[indices], [1, 0, 1, 0])\n assert np.allclose(buf.prev(indices), [0, 1, 1, 25])\n assert np.allclose(buf.next(indices), [0, 2, 2, 25])\n indices = buf.sample_indices(10000)\n assert np.bincount(indices)[[0, 1, 2, 25]].min() > 2000 # uniform sample\n # cached buffer with main_buffer size == 0 (no update)\n # used in test_collector\n buf = CachedReplayBuffer(ReplayBuffer(0, sample_avail=True), 4, 5)\n data = np.zeros(4)\n # TODO: this doesn't make any sense - why a matrix reward?!\n # See error message in ReplayBuffer._update_state_pre_add\n rew = np.ones([4, 4])\n buf.add(\n cast(\n RolloutBatchProtocol,\n Batch(\n obs=data,\n act=data,\n rew=rew,\n terminated=[0, 0, 1, 1],\n truncated=[0, 0, 0, 0],\n ),\n ),\n )\n buf.add(\n cast(\n RolloutBatchProtocol,\n Batch(\n obs=data,\n act=data,\n rew=rew,\n terminated=[0, 0, 0, 0],\n truncated=[0, 0, 0, 0],\n ),\n ),\n )\n buf.add(\n cast(\n RolloutBatchProtocol,\n Batch(\n obs=data,\n act=data,\n rew=rew,\n terminated=[1, 1, 1, 1],\n truncated=[0, 0, 0, 0],\n ),\n ),\n )\n buf.add(\n cast(\n RolloutBatchProtocol,\n Batch(\n obs=data,\n act=data,\n rew=rew,\n terminated=[0, 0, 0, 0],\n truncated=[0, 0, 0, 0],\n ),\n ),\n )\n ptr, ep_rew, ep_len, ep_idx = buf.add(\n cast(\n RolloutBatchProtocol,\n Batch(\n obs=data,\n act=data,\n rew=rew,\n terminated=[0, 1, 0, 1],\n truncated=[0, 0, 0, 0],\n ),\n ),\n )\n assert np.all(ptr == [1, -1, 11, -1])\n assert np.all(ep_idx == [0, -1, 10, -1])\n assert np.all(ep_len == [0, 2, 0, 2])\n assert np.all(ep_rew == [data, data + 2, data, data + 2])\n assert np.allclose(\n buf.done,\n [\n 0,\n 0,\n 1,\n 0,\n 0,\n 0,\n 1,\n 1,\n 0,\n 0,\n 0,\n 0,\n 0,\n 0,\n 0,\n 0,\n 1,\n 0,\n 0,\n 0,\n ],\n )\n indices = buf.sample_indices(0)\n assert np.allclose(indices, [0, 1, 10, 11])\n assert np.allclose(buf.prev(indices), [0, 0, 10, 10])\n assert np.allclose(buf.next(indices), [1, 1, 11, 11])\n\n\ndef test_multibuf_stack() -> None:\n size = 5\n bufsize = 9\n stack_num = 4\n cached_num = 3\n env = MoveToRightEnv(size)\n # test if CachedReplayBuffer can handle stack_num + ignore_obs_next\n buf4 = CachedReplayBuffer(\n ReplayBuffer(bufsize, stack_num=stack_num, ignore_obs_next=True),\n cached_num,\n size,\n )\n # test if CachedReplayBuffer can handle corner case:\n # buffer + stack_num + ignore_obs_next + sample_avail\n buf5 = CachedReplayBuffer(\n ReplayBuffer(bufsize, stack_num=stack_num, ignore_obs_next=True, sample_avail=True),\n cached_num,\n size,\n )\n obs, info = env.reset(options={\"state\": 1})\n obs = cast(np.ndarray, obs)\n for i in range(18):\n obs_next, rew, terminated, truncated, info = env.step(1)\n done = terminated or truncated\n obs_list = np.array([obs + size * i for i in range(cached_num)])\n act_list = [1] * cached_num\n rew_list = [rew] * cached_num\n terminated_list = [terminated] * cached_num\n truncated_list = [truncated] * cached_num\n obs_next_list = -obs_list\n info_list = [info] * cached_num\n batch = cast(\n RolloutBatchProtocol,\n Batch(\n obs=obs_list,\n act=act_list,\n rew=rew_list,\n terminated=terminated_list,\n truncated=truncated_list,\n obs_next=obs_next_list,\n info=info_list,\n ),\n )\n buf5.add(batch)\n buf4.add(batch)\n assert np.all(buf4.obs == buf5.obs)\n assert np.all(buf4.done == buf5.done)\n assert np.all(buf4.terminated == buf5.terminated)\n assert np.all(buf4.truncated == buf5.truncated)\n obs = obs_next\n if done:\n # obs is an array, but the env is malformed, so we can't properly type it\n obs, info = env.reset(options={\"state\": 1}) # type: ignore[assignment]\n # check the `add` order is correct\n assert np.allclose(\n buf4.obs.reshape(-1),\n [\n 12,\n 13,\n 14,\n 4,\n 6,\n 7,\n 8,\n 9,\n 11, # main_buffer\n 1,\n 2,\n 3,\n 4,\n 0, # cached_buffer[0]\n 6,\n 7,\n 8,\n 9,\n 0, # cached_buffer[1]\n 11,\n 12,\n 13,\n 14,\n 0, # cached_buffer[2]\n ],\n ), buf4.obs\n assert np.allclose(\n buf4.done,\n [\n 0,\n 0,\n 1,\n 1,\n 0,\n 0,\n 0,\n 1,\n 0, # main_buffer\n 0,\n 0,\n 0,\n 1,\n 0, # cached_buffer[0]\n 0,\n 0,\n 0,\n 1,\n 0, # cached_buffer[1]\n 0,\n 0,\n 0,\n 1,\n 0, # cached_buffer[2]\n ],\n ), buf4.done\n assert np.allclose(buf4.unfinished_index(), [10, 15, 20])\n indices = np.array(sorted(buf4.sample_indices(0)))\n assert np.allclose(indices, [*list(range(bufsize)), 9, 10, 14, 15, 19, 20])\n cur_obs = buf4[indices].obs\n assert isinstance(cur_obs, np.ndarray)\n assert np.allclose(\n cur_obs[..., 0],\n [\n [11, 11, 11, 12],\n [11, 11, 12, 13],\n [11, 12, 13, 14],\n [4, 4, 4, 4],\n [6, 6, 6, 6],\n [6, 6, 6, 7],\n [6, 6, 7, 8],\n [6, 7, 8, 9],\n [11, 11, 11, 11],\n [1, 1, 1, 1],\n [1, 1, 1, 2],\n [6, 6, 6, 6],\n [6, 6, 6, 7],\n [11, 11, 11, 11],\n [11, 11, 11, 12],\n ],\n )\n next_obs = buf4[indices].obs_next\n assert isinstance(next_obs, np.ndarray)\n assert np.allclose(\n next_obs[..., 0],\n [\n [11, 11, 12, 13],\n [11, 12, 13, 14],\n [11, 12, 13, 14],\n [4, 4, 4, 4],\n [6, 6, 6, 7],\n [6, 6, 7, 8],\n [6, 7, 8, 9],\n [6, 7, 8, 9],\n [11, 11, 11, 12],\n [1, 1, 1, 2],\n [1, 1, 1, 2],\n [6, 6, 6, 7],\n [6, 6, 6, 7],\n [11, 11, 11, 12],\n [11, 11, 11, 12],\n ],\n )\n indices = buf5.sample_indices(0)\n assert np.allclose(sorted(indices), [2, 7])\n assert np.all(np.isin(buf5.sample_indices(100), indices))\n # manually change the stack num\n buf5.stack_num = 2\n for buf in buf5.buffers:\n buf.stack_num = 2\n indices = buf5.sample_indices(0)\n assert np.allclose(sorted(indices), [0, 1, 2, 5, 6, 7, 10, 15, 20])\n batch_sample, _ = buf5.sample(0)\n # test Atari with CachedReplayBuffer, save_only_last_obs + ignore_obs_next\n buf6 = CachedReplayBuffer(\n ReplayBuffer(bufsize, stack_num=stack_num, save_only_last_obs=True, ignore_obs_next=True),\n cached_num,\n size,\n )\n obs = np.random.rand(size, 4, 84, 84)\n buf6.add(\n cast(\n RolloutBatchProtocol,\n Batch(\n obs=[obs[2], obs[0]],\n act=[1, 1],\n rew=[0, 0],\n terminated=[0, 1],\n truncated=[0, 0],\n obs_next=[obs[3], obs[1]],\n ),\n ),\n buffer_ids=[1, 2],\n )\n assert buf6.obs.shape == (buf6.maxsize, 84, 84)\n assert np.allclose(buf6.obs[0], obs[0, -1])\n assert np.allclose(buf6.obs[14], obs[2, -1])\n assert np.allclose(buf6.obs[19], obs[0, -1])\n assert buf6[0].obs.shape == (4, 84, 84)\n\n\ndef test_multibuf_hdf5() -> None:\n size = 100\n buffers = {\n \"vector\": VectorReplayBuffer(size * 4, 4),\n \"cached\": CachedReplayBuffer(ReplayBuffer(size), 4, size),\n }\n buffer_types = {k: b.__class__ for k, b in buffers.items()}\n device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n info_t = torch.tensor([1.0]).to(device)\n for i in range(4):\n kwargs = {\n \"obs\": Batch(index=np.array([i])),\n \"act\": i,\n \"rew\": np.array([1, 2]),\n \"terminated\": i % 3 == 2,\n \"truncated\": False,\n \"done\": i % 3 == 2,\n \"info\": {\"number\": {\"n\": i, \"t\": info_t}, \"extra\": None},\n }\n buffers[\"vector\"].add(Batch.stack([kwargs, kwargs, kwargs]), buffer_ids=[0, 1, 2])\n buffers[\"cached\"].add(Batch.stack([kwargs, kwargs, kwargs]), buffer_ids=[0, 1, 2])\n\n # save\n paths = {}\n for k, buf in buffers.items():\n f, path = tempfile.mkstemp(suffix=\".hdf5\")\n os.close(f)\n buf.save_hdf5(path)\n paths[k] = path\n\n # load replay buffer\n _buffers = {k: buffer_types[k].load_hdf5(paths[k]) for k in paths}\n\n # compare\n for k in buffers:\n assert len(_buffers[k]) == len(buffers[k])\n assert np.allclose(_buffers[k].act, buffers[k].act)\n assert _buffers[k].stack_num == buffers[k].stack_num\n assert _buffers[k].maxsize == buffers[k].maxsize\n assert np.all(_buffers[k]._indices == buffers[k]._indices)\n # check shallow copy in VectorReplayBuffer\n for k in [\"vector\", \"cached\"]:\n buffers[k].info.number.n[0] = -100\n assert buffers[k].buffers[0].info.number.n[0] == -100\n # check if still behave normally\n for k in [\"vector\", \"cached\"]:\n kwargs = {\n \"obs\": Batch(index=np.array([5])),\n \"act\": 5,\n \"rew\": np.array([2, 1]),\n \"terminated\": False,\n \"truncated\": False,\n \"done\": False,\n \"info\": {\"number\": {\"n\": i}, \"Timelimit.truncate\": True},\n }\n buffers[k].add(Batch.stack([kwargs, kwargs, kwargs, kwargs]))\n act = np.zeros(buffers[k].maxsize)\n if k == \"vector\":\n act[np.arange(5)] = np.array([0, 1, 2, 3, 5])\n act[np.arange(5) + size] = np.array([0, 1, 2, 3, 5])\n act[np.arange(5) + size * 2] = np.array([0, 1, 2, 3, 5])\n act[size * 3] = 5\n elif k == \"cached\":\n act[np.arange(9)] = np.array([0, 1, 2, 0, 1, 2, 0, 1, 2])\n act[np.arange(3) + size] = np.array([3, 5, 2])\n act[np.arange(3) + size * 2] = np.array([3, 5, 2])\n act[np.arange(3) + size * 3] = np.array([3, 5, 2])\n act[size * 4] = 5\n assert np.allclose(buffers[k].act, act)\n info_keys = [\"number\", \"extra\", \"Timelimit.truncate\"]\n assert set(buffers[k].info.keys()) == set(info_keys)\n\n for path in paths.values():\n os.remove(path)\n\n\ndef test_from_data() -> None:\n obs_data: npt.NDArray[np.uint8] = np.ndarray((10, 3, 3), dtype=\"uint8\")\n for i in range(10):\n obs_data[i] = i * np.ones((3, 3), dtype=\"uint8\")\n obs_next_data = np.zeros_like(obs_data)\n obs_next_data[:-1] = obs_data[1:]\n f, path = tempfile.mkstemp(suffix=\".hdf5\")\n os.close(f)\n with h5py.File(path, \"w\") as f:\n obs = f.create_dataset(\"obs\", data=obs_data)\n act = f.create_dataset(\"act\", data=np.arange(10, dtype=\"int32\"))\n rew = f.create_dataset(\"rew\", data=np.arange(10, dtype=\"float32\"))\n terminated = f.create_dataset(\"terminated\", data=np.zeros(10, dtype=\"bool\"))\n truncated = f.create_dataset(\"truncated\", data=np.zeros(10, dtype=\"bool\"))\n done = f.create_dataset(\"done\", data=np.zeros(10, dtype=\"bool\"))\n obs_next = f.create_dataset(\"obs_next\", data=obs_next_data)\n buf = ReplayBuffer.from_data(obs, act, rew, terminated, truncated, done, obs_next)\n assert len(buf) == 10\n batch = buf[3]\n cur_obs = batch.obs\n assert isinstance(cur_obs, np.ndarray)\n assert np.array_equal(cur_obs, 3 * np.ones((3, 3), dtype=\"uint8\"))\n assert batch.act == 3\n assert batch.rew == 3.0\n assert not batch.done\n next_obs = batch.obs_next\n assert isinstance(next_obs, np.ndarray)\n assert np.array_equal(next_obs, 4 * np.ones((3, 3), dtype=\"uint8\"))\n os.remove(path)\n\n\ndef test_custom_key() -> None:\n batch = cast(\n RolloutBatchProtocol,\n Batch(\n obs_next=np.array(\n [\n [\n 1.174,\n -0.1151,\n -0.609,\n -0.5205,\n -0.9316,\n 3.236,\n -2.418,\n 0.386,\n 0.2227,\n -0.5117,\n 2.293,\n ],\n ],\n ),\n rew=np.array([4.28125]),\n act=np.array([[-0.3088, -0.4636, 0.4956]]),\n truncated=np.array([False]),\n obs=np.array(\n [\n [\n 1.193,\n -0.1203,\n -0.6123,\n -0.519,\n -0.9434,\n 3.32,\n -2.266,\n 0.9116,\n 0.623,\n 0.1259,\n 0.363,\n ],\n ],\n ),\n terminated=np.array([False]),\n done=np.array([False]),\n returns=np.array([74.70343082]),\n info=Batch(),\n policy=Batch(),\n ),\n )\n buffer_size = len(batch.rew)\n buffer = ReplayBuffer(buffer_size)\n buffer.add(batch)\n sampled_batch, _ = buffer.sample(1)\n # Check if they have the same keys\n assert set(batch.get_keys()) == set(\n sampled_batch.get_keys(),\n ), f\"Batches have different keys: {set(batch.get_keys())} and {set(sampled_batch.get_keys())}\"\n # Compare the values for each key\n for key in batch.get_keys():\n if isinstance(batch.__dict__[key], np.ndarray) and isinstance(\n sampled_batch.__dict__[key],\n np.ndarray,\n ):\n assert np.allclose(\n batch.__dict__[key],\n sampled_batch.__dict__[key],\n ), f\"Value mismatch for key: {key}\"\n if isinstance(batch.__dict__[key], Batch) and isinstance(\n sampled_batch.__dict__[key],\n Batch,\n ):\n assert len(batch.__dict__[key].get_keys()) == 0\n assert len(sampled_batch.__dict__[key].get_keys()) == 0\n\n\ndef test_buffer_dropnull() -> None:\n size = 10\n buf = ReplayBuffer(size, ignore_obs_next=True)\n for i in range(4):\n buf.add(\n cast(\n RolloutBatchProtocol,\n Batch(\n obs={\n \"mask1\": i + 1,\n \"mask2\": i + 4,\n \"mask\": i,\n },\n act={\"act_id\": i, \"position_id\": i + 3},\n rew=i,\n terminated=i % 3 == 0,\n truncated=False,\n info={\"if\": i},\n ),\n ),\n )\n\n assert len(buf[:3]) == 3\n\n buf.set_array_at_key(np.array([1, 2, 3], float), \"newkey\", [0, 1, 2])\n assert np.array_equal(buf.newkey[:3], np.array([1, 2, 3], float))\n assert buf.hasnull()\n buf.dropnull()\n assert len(buf[:3]) == 3\n assert not buf.hasnull()\n\n\n@pytest.fixture\ndef dummy_rollout_batch() -> RolloutBatchProtocol:\n return cast(\n RolloutBatchProtocol,\n Batch(\n obs=np.arange(2),\n obs_next=np.arange(2),\n act=np.arange(5),\n rew=1,\n terminated=False,\n truncated=False,\n done=False,\n info={},\n ),\n )\n\n\ndef test_get_replay_buffer_indices(dummy_rollout_batch: RolloutBatchProtocol) -> None:\n buffer = ReplayBuffer(5)\n for _ in range(5):\n buffer.add(dummy_rollout_batch)\n assert np.array_equal(buffer.get_buffer_indices(0, 3), [0, 1, 2])\n assert np.array_equal(buffer.get_buffer_indices(3, 2), [3, 4, 0, 1])\n assert np.array_equal(buffer.get_buffer_indices(0, 5), np.arange(5))\n\n\ndef test_get_vector_replay_buffer_indices(\n dummy_rollout_batch: RolloutBatchProtocol,\n) -> None:\n stacked_batch = Batch.stack([dummy_rollout_batch, dummy_rollout_batch])\n buffer = VectorReplayBuffer(10, 2)\n for _ in range(5):\n buffer.add(stacked_batch)\n\n assert np.array_equal(buffer.get_buffer_indices(0, 3), [0, 1, 2])\n assert np.array_equal(buffer.get_buffer_indices(3, 2), [3, 4, 0, 1])\n\n assert np.array_equal(buffer.get_buffer_indices(6, 9), [6, 7, 8])\n assert np.array_equal(buffer.get_buffer_indices(8, 7), [8, 9, 5, 6])\n\n with pytest.raises(ValueError):\n buffer.get_buffer_indices(3, 6)\n with pytest.raises(ValueError):\n buffer.get_buffer_indices(6, 3)\n" }, { "path": "test/base/test_collector.py", "content": "from collections.abc import Callable, Sequence\nfrom typing import Any\n\nimport gymnasium as gym\nimport numpy as np\nimport pytest\nimport tqdm\n\nfrom test.base.env import MoveToRightEnv, NXEnv\nfrom tianshou.algorithm.algorithm_base import Policy, episode_mc_return_to_go\nfrom tianshou.data import (\n AsyncCollector,\n Batch,\n CachedReplayBuffer,\n Collector,\n CollectStats,\n PrioritizedReplayBuffer,\n ReplayBuffer,\n VectorReplayBuffer,\n)\nfrom tianshou.data.batch import BatchProtocol\nfrom tianshou.data.collector import (\n CollectActionBatchProtocol,\n EpisodeRolloutHookMCReturn,\n StepHook,\n)\nfrom tianshou.data.types import ObsBatchProtocol, RolloutBatchProtocol\nfrom tianshou.env import DummyVectorEnv, SubprocVectorEnv\n\ntry:\n import envpool\nexcept ImportError:\n envpool = None\n\n\nclass MaxActionPolicy(Policy):\n def __init__(\n self,\n action_space: gym.spaces.Space | None = None,\n dict_state: bool = False,\n need_state: bool = True,\n action_shape: Sequence[int] | int | None = None,\n ) -> None:\n \"\"\"Mock policy for testing, will always return an array of ones of the shape of the action space.\n Note that this doesn't make much sense for discrete action space (the output is then intepreted as\n logits, meaning all actions would be equally likely).\n\n :param action_space: the action space of the environment. If None, a dummy Box space will be used.\n :param bool dict_state: if the observation of the environment is a dict\n :param bool need_state: if the policy needs the hidden state (for RNN)\n \"\"\"\n action_space = action_space or gym.spaces.Box(-1, 1, (1,))\n super().__init__(action_space=action_space)\n self.dict_state = dict_state\n self.need_state = need_state\n self.action_shape = action_shape\n\n def forward(\n self,\n batch: ObsBatchProtocol,\n state: dict | BatchProtocol | np.ndarray | None = None,\n **kwargs: Any,\n ) -> Batch:\n if self.need_state:\n if state is None:\n state = np.zeros((len(batch.obs), 2))\n elif isinstance(state, np.ndarray | BatchProtocol):\n state += np.int_(1)\n elif isinstance(state, dict) and state.get(\"hidden\") is not None:\n state[\"hidden\"] += np.int_(1)\n if self.dict_state:\n if self.action_shape:\n action_shape = self.action_shape\n elif isinstance(batch.obs, Batch):\n action_shape = len(batch.obs[\"index\"])\n else:\n action_shape = len(batch.obs)\n return Batch(act=np.ones(action_shape), state=state)\n action_shape = self.action_shape if self.action_shape else len(batch.obs)\n return Batch(act=np.ones(action_shape), state=state)\n\n\n@pytest.fixture()\ndef collector_with_single_env() -> Collector[CollectStats]:\n \"\"\"The env will be a MoveToRightEnv with size 5, sleep 0.\"\"\"\n env = MoveToRightEnv(size=5, sleep=0)\n policy = MaxActionPolicy()\n collector = Collector[CollectStats](policy, env, ReplayBuffer(size=100))\n collector.reset()\n return collector\n\n\ndef test_collector() -> None:\n env_fns = [lambda x=i: MoveToRightEnv(size=x, sleep=0) for i in [2, 3, 4, 5]]\n\n subproc_venv_4_envs = SubprocVectorEnv(env_fns)\n dummy_venv_4_envs = DummyVectorEnv(env_fns)\n policy = MaxActionPolicy()\n single_env = env_fns[0]()\n c_single_env = Collector[CollectStats](\n policy,\n single_env,\n ReplayBuffer(size=100),\n )\n c_single_env.reset()\n c_single_env.collect(n_step=3)\n assert len(c_single_env.buffer) == 3\n # TODO: direct attr access is an arcane way of using the buffer, it should be never done\n # The placeholders for entries are all zeros, so buffer.obs is an array filled with 3\n # observations, and 97 zeros.\n # However, buffer[:] will have all attributes with length three... The non-filled entries are removed there\n\n # See above. For the single env, we start with obs=0, obs_next=1.\n # We move to obs=1, obs_next=2,\n # then the env is reset and we move to obs=0\n # Making one more step results in obs_next=1\n # The final 0 in the buffer.obs is because the buffer is initialized with zeros and the direct attr access\n assert np.allclose(c_single_env.buffer.obs[:4, 0], [0, 1, 0, 0])\n obs_next = c_single_env.buffer[:].obs_next[..., 0]\n assert isinstance(obs_next, np.ndarray)\n assert np.allclose(obs_next, [1, 2, 1])\n keys = np.zeros(100)\n keys[:3] = 1\n assert np.allclose(c_single_env.buffer.info[\"key\"], keys)\n for e in c_single_env.buffer.info[\"env\"][:3]:\n assert isinstance(e, MoveToRightEnv)\n assert np.allclose(c_single_env.buffer.info[\"env_id\"], 0)\n rews = np.zeros(100)\n rews[:3] = [0, 1, 0]\n assert np.allclose(c_single_env.buffer.rew, rews)\n # At this point, the buffer contains obs 0 -> 1 -> 0\n\n # At start we have 3 entries in the buffer\n # We collect 3 episodes, in addition to the transitions we have collected before\n # 0 -> 1 -> 0 -> 0 (reset at collection start) -> 1 -> done (0) -> 1 -> done(0)\n # obs_next: 1 -> 2 -> 1 -> 1 (reset at collection start) -> 2 -> 1 -> 2 -> 1 -> 2\n # In total, we will have 3 + 6 = 9 entries in the buffer\n c_single_env.collect(n_episode=3)\n assert len(c_single_env.buffer) == 8\n assert np.allclose(c_single_env.buffer.obs[:10, 0], [0, 1, 0, 1, 0, 1, 0, 1, 0, 0])\n obs_next = c_single_env.buffer[:].obs_next[..., 0]\n assert isinstance(obs_next, np.ndarray)\n assert np.allclose(obs_next, [1, 2, 1, 2, 1, 2, 1, 2])\n assert np.allclose(c_single_env.buffer.info[\"key\"][:8], 1)\n for e in c_single_env.buffer.info[\"env\"][:8]:\n assert isinstance(e, MoveToRightEnv)\n assert np.allclose(c_single_env.buffer.info[\"env_id\"][:8], 0)\n assert np.allclose(c_single_env.buffer.rew[:8], [0, 1, 0, 1, 0, 1, 0, 1])\n c_single_env.collect(n_step=3, random=True)\n\n c_subproc_venv_4_envs = Collector[CollectStats](\n policy,\n subproc_venv_4_envs,\n VectorReplayBuffer(total_size=100, buffer_num=4),\n )\n c_subproc_venv_4_envs.reset()\n\n # Collect some steps\n c_subproc_venv_4_envs.collect(n_step=8)\n obs = np.zeros(100)\n valid_indices = [0, 1, 25, 26, 50, 51, 75, 76]\n obs[valid_indices] = [0, 1, 0, 1, 0, 1, 0, 1]\n assert np.allclose(c_subproc_venv_4_envs.buffer.obs[:, 0], obs)\n obs_next = c_subproc_venv_4_envs.buffer[:].obs_next[..., 0]\n assert isinstance(obs_next, np.ndarray)\n assert np.allclose(obs_next, [1, 2, 1, 2, 1, 2, 1, 2])\n keys = np.zeros(100)\n keys[valid_indices] = [1, 1, 1, 1, 1, 1, 1, 1]\n assert np.allclose(c_subproc_venv_4_envs.buffer.info[\"key\"], keys)\n for e in c_subproc_venv_4_envs.buffer.info[\"env\"][valid_indices]:\n assert isinstance(e, MoveToRightEnv)\n env_ids = np.zeros(100)\n env_ids[valid_indices] = [0, 0, 1, 1, 2, 2, 3, 3]\n assert np.allclose(c_subproc_venv_4_envs.buffer.info[\"env_id\"], env_ids)\n rews = np.zeros(100)\n rews[valid_indices] = [0, 1, 0, 0, 0, 0, 0, 0]\n assert np.allclose(c_subproc_venv_4_envs.buffer.rew, rews)\n\n # we previously collected 8 steps, 2 from each env, now we collect 4 episodes\n # each env will contribute an episode, which will be of lens 2 (first env was reset), 1, 2, 3\n # So we get 8 + 2+1+2+3 = 16 steps\n c_subproc_venv_4_envs.collect(n_episode=4)\n assert len(c_subproc_venv_4_envs.buffer) == 16\n\n valid_indices = [2, 3, 27, 52, 53, 77, 78, 79]\n obs[valid_indices] = [0, 1, 2, 2, 3, 2, 3, 4]\n assert np.allclose(c_subproc_venv_4_envs.buffer.obs[:, 0], obs)\n obs_next = c_subproc_venv_4_envs.buffer[:].obs_next[..., 0]\n assert isinstance(obs_next, np.ndarray)\n assert np.allclose(\n obs_next,\n [1, 2, 1, 2, 1, 2, 3, 1, 2, 3, 4, 1, 2, 3, 4, 5],\n )\n keys[valid_indices] = [1, 1, 1, 1, 1, 1, 1, 1]\n assert np.allclose(c_subproc_venv_4_envs.buffer.info[\"key\"], keys)\n for e in c_subproc_venv_4_envs.buffer.info[\"env\"][valid_indices]:\n assert isinstance(e, MoveToRightEnv)\n env_ids[valid_indices] = [0, 0, 1, 2, 2, 3, 3, 3]\n assert np.allclose(c_subproc_venv_4_envs.buffer.info[\"env_id\"], env_ids)\n rews[valid_indices] = [0, 1, 1, 0, 1, 0, 0, 1]\n assert np.allclose(c_subproc_venv_4_envs.buffer.rew, rews)\n c_subproc_venv_4_envs.collect(n_episode=4, random=True)\n\n c_dummy_venv_4_envs = Collector[CollectStats](\n policy,\n dummy_venv_4_envs,\n VectorReplayBuffer(total_size=100, buffer_num=4),\n )\n c_dummy_venv_4_envs.reset()\n c_dummy_venv_4_envs.collect(n_episode=7)\n obs1 = obs.copy()\n obs1[[4, 5, 28, 29, 30]] = [0, 1, 0, 1, 2]\n obs2 = obs.copy()\n obs2[[28, 29, 30, 54, 55, 56, 57]] = [0, 1, 2, 0, 1, 2, 3]\n c2obs = c_dummy_venv_4_envs.buffer.obs[:, 0]\n assert np.all(c2obs == obs1) or np.all(c2obs == obs2)\n c_dummy_venv_4_envs.reset_env()\n c_dummy_venv_4_envs.reset_buffer()\n assert c_dummy_venv_4_envs.collect(n_episode=8).n_collected_episodes == 8\n valid_indices = [4, 5, 28, 29, 30, 54, 55, 56, 57]\n obs[valid_indices] = [0, 1, 0, 1, 2, 0, 1, 2, 3]\n assert np.all(c_dummy_venv_4_envs.buffer.obs[:, 0] == obs)\n keys[valid_indices] = [1, 1, 1, 1, 1, 1, 1, 1, 1]\n assert np.allclose(c_dummy_venv_4_envs.buffer.info[\"key\"], keys)\n for e in c_dummy_venv_4_envs.buffer.info[\"env\"][valid_indices]:\n assert isinstance(e, MoveToRightEnv)\n env_ids[valid_indices] = [0, 0, 1, 1, 1, 2, 2, 2, 2]\n assert np.allclose(c_dummy_venv_4_envs.buffer.info[\"env_id\"], env_ids)\n rews[valid_indices] = [0, 1, 0, 0, 1, 0, 0, 0, 1]\n assert np.allclose(c_dummy_venv_4_envs.buffer.rew, rews)\n c_dummy_venv_4_envs.collect(n_episode=4, random=True)\n\n # test corner case\n with pytest.raises(ValueError):\n Collector[CollectStats](policy, dummy_venv_4_envs, ReplayBuffer(10))\n with pytest.raises(ValueError):\n Collector[CollectStats](policy, dummy_venv_4_envs, PrioritizedReplayBuffer(10, 0.5, 0.5))\n with pytest.raises(ValueError):\n c_dummy_venv_4_envs.collect()\n\n def get_env_factory(i: int, t: str) -> Callable[[], NXEnv]:\n return lambda: NXEnv(i, t)\n\n # test NXEnv\n for obs_type in [\"array\", \"object\"]:\n envs = SubprocVectorEnv([get_env_factory(i=i, t=obs_type) for i in [5, 10, 15, 20]])\n c_suproc_new = Collector[CollectStats](\n policy,\n envs,\n VectorReplayBuffer(total_size=100, buffer_num=4),\n )\n c_suproc_new.reset()\n c_suproc_new.collect(n_step=6)\n assert c_suproc_new.buffer.obs.dtype == object\n\n\n@pytest.fixture()\ndef async_collector_and_env_lens() -> tuple[AsyncCollector, list[int]]:\n env_lens = [2, 3, 4, 5]\n env_fns = [lambda x=i: MoveToRightEnv(size=x, sleep=0.001, random_sleep=True) for i in env_lens]\n\n venv = SubprocVectorEnv(env_fns, wait_num=len(env_fns) - 1)\n policy = MaxActionPolicy()\n bufsize = 60\n async_collector = AsyncCollector(\n policy,\n venv,\n VectorReplayBuffer(total_size=bufsize * 4, buffer_num=4),\n )\n async_collector.reset()\n return async_collector, env_lens\n\n\nclass TestAsyncCollector:\n def test_collect_without_argument_gives_error(\n self,\n async_collector_and_env_lens: tuple[AsyncCollector, list[int]],\n ) -> None:\n c1, env_lens = async_collector_and_env_lens\n with pytest.raises(ValueError):\n c1.collect()\n\n def test_collect_one_episode_async(\n self,\n async_collector_and_env_lens: tuple[AsyncCollector, list[int]],\n ) -> None:\n c1, env_lens = async_collector_and_env_lens\n result = c1.collect(n_episode=1)\n assert result.n_collected_episodes >= 1\n\n def test_enough_episodes_two_collection_cycles_n_episode_without_reset(\n self,\n async_collector_and_env_lens: tuple[AsyncCollector, list[int]],\n ) -> None:\n c1, env_lens = async_collector_and_env_lens\n n_episode = 2\n result_c1 = c1.collect(n_episode=n_episode, reset_before_collect=False)\n assert result_c1.n_collected_episodes >= n_episode\n result_c2 = c1.collect(n_episode=n_episode, reset_before_collect=False)\n assert result_c2.n_collected_episodes >= n_episode\n\n def test_enough_episodes_two_collection_cycles_n_episode_with_reset(\n self,\n async_collector_and_env_lens: tuple[AsyncCollector, list[int]],\n ) -> None:\n c1, env_lens = async_collector_and_env_lens\n n_episode = 2\n result_c1 = c1.collect(n_episode=n_episode, reset_before_collect=True)\n assert result_c1.n_collected_episodes >= n_episode\n result_c2 = c1.collect(n_episode=n_episode, reset_before_collect=True)\n assert result_c2.n_collected_episodes >= n_episode\n\n def test_enough_episodes_and_correct_obs_indices_and_obs_next_iterative_collection_cycles_n_episode(\n self,\n async_collector_and_env_lens: tuple[AsyncCollector, list[int]],\n ) -> None:\n c1, env_lens = async_collector_and_env_lens\n ptr = [0, 0, 0, 0]\n bufsize = 60\n for n_episode in tqdm.trange(1, 30, desc=\"test async n_episode\"):\n result = c1.collect(n_episode=n_episode)\n assert result.n_collected_episodes >= n_episode\n # check buffer data, obs and obs_next, env_id\n for i, count in enumerate(np.bincount(result.lens, minlength=6)[2:]):\n env_len = i + 2\n total = env_len * count\n indices = np.arange(ptr[i], ptr[i] + total) % bufsize\n ptr[i] = (ptr[i] + total) % bufsize\n seq = np.arange(env_len)\n buf = c1.buffer.buffers[i]\n assert np.all(buf.info.env_id[indices] == i)\n assert np.all(buf.obs[indices].reshape(count, env_len) == seq)\n assert np.all(buf.obs_next[indices].reshape(count, env_len) == seq + 1)\n\n def test_enough_episodes_and_correct_obs_indices_and_obs_next_iterative_collection_cycles_n_step(\n self,\n async_collector_and_env_lens: tuple[AsyncCollector, list[int]],\n ) -> None:\n c1, env_lens = async_collector_and_env_lens\n bufsize = 60\n ptr = [0, 0, 0, 0]\n for n_step in tqdm.trange(1, 15, desc=\"test async n_step\"):\n result = c1.collect(n_step=n_step)\n assert result.n_collected_steps >= n_step\n for i, count in enumerate(np.bincount(result.lens, minlength=6)[2:]):\n env_len = i + 2\n total = env_len * count\n indices = np.arange(ptr[i], ptr[i] + total) % bufsize\n ptr[i] = (ptr[i] + total) % bufsize\n seq = np.arange(env_len)\n buf = c1.buffer.buffers[i]\n assert np.all(buf.info.env_id[indices] == i)\n assert np.all(buf.obs[indices].reshape(count, env_len) == seq)\n assert np.all(buf.obs_next[indices].reshape(count, env_len) == seq + 1)\n\n def test_enough_episodes_and_correct_obs_indices_and_obs_next_iterative_collection_cycles_first_n_episode_then_n_step(\n self,\n async_collector_and_env_lens: tuple[AsyncCollector, list[int]],\n ) -> None:\n c1, env_lens = async_collector_and_env_lens\n bufsize = 60\n ptr = [0, 0, 0, 0]\n for n_episode in tqdm.trange(1, 30, desc=\"test async n_episode\"):\n result = c1.collect(n_episode=n_episode)\n assert result.n_collected_episodes >= n_episode\n # check buffer data, obs and obs_next, env_id\n for i, count in enumerate(np.bincount(result.lens, minlength=6)[2:]):\n env_len = i + 2\n total = env_len * count\n indices = np.arange(ptr[i], ptr[i] + total) % bufsize\n ptr[i] = (ptr[i] + total) % bufsize\n seq = np.arange(env_len)\n buf = c1.buffer.buffers[i]\n assert np.all(buf.info.env_id[indices] == i)\n assert np.all(buf.obs[indices].reshape(count, env_len) == seq)\n assert np.all(buf.obs_next[indices].reshape(count, env_len) == seq + 1)\n # test async n_step, for now the buffer should be full of data, thus no bincount stuff as above\n for n_step in tqdm.trange(1, 15, desc=\"test async n_step\"):\n result = c1.collect(n_step=n_step)\n assert result.n_collected_steps >= n_step\n for i in range(4):\n env_len = i + 2\n seq = np.arange(env_len)\n buf = c1.buffer.buffers[i]\n assert np.all(buf.info.env_id == i)\n assert np.all(buf.obs.reshape(-1, env_len) == seq)\n assert np.all(buf.obs_next.reshape(-1, env_len) == seq + 1)\n\n\ndef test_collector_with_dict_state() -> None:\n env = MoveToRightEnv(size=5, sleep=0, dict_state=True)\n policy = MaxActionPolicy(dict_state=True)\n c0 = Collector[CollectStats](policy, env, ReplayBuffer(size=100))\n c0.reset()\n c0.collect(n_step=3)\n c0.collect(n_episode=2)\n assert len(c0.buffer) == 10 # 3 + two episodes with 5 steps each\n env_fns = [lambda x=i: MoveToRightEnv(size=x, sleep=0, dict_state=True) for i in [2, 3, 4, 5]]\n envs = DummyVectorEnv(env_fns)\n envs.seed(666)\n obs, info = envs.reset()\n assert not np.isclose(obs[0][\"rand\"], obs[1][\"rand\"])\n c1 = Collector[CollectStats](\n policy,\n envs,\n VectorReplayBuffer(total_size=100, buffer_num=4),\n )\n c1.reset()\n c1.collect(n_step=12)\n result = c1.collect(n_episode=8)\n assert result.n_collected_episodes == 8\n lens = np.bincount(result.lens)\n assert (result.n_collected_steps == 21 and np.all(lens == [0, 0, 2, 2, 2, 2])) or (\n result.n_collected_steps == 20 and np.all(lens == [0, 0, 3, 1, 2, 2])\n )\n batch, _ = c1.buffer.sample(10)\n c0.buffer.update(c1.buffer)\n assert len(c0.buffer) in [42, 43]\n cur_obs = c0.buffer[:].obs\n assert isinstance(cur_obs, Batch)\n if len(c0.buffer) == 42:\n assert np.all(\n cur_obs.index[..., 0]\n == [\n 0,\n 1,\n 2,\n 3,\n 4,\n 0,\n 1,\n 2,\n 3,\n 4,\n 0,\n 1,\n 0,\n 1,\n 0,\n 1,\n 0,\n 1,\n 0,\n 1,\n 2,\n 0,\n 1,\n 2,\n 0,\n 1,\n 2,\n 3,\n 0,\n 1,\n 2,\n 3,\n 0,\n 1,\n 2,\n 3,\n 4,\n 0,\n 1,\n 2,\n 3,\n 4,\n ],\n ), cur_obs.index[..., 0]\n else:\n assert np.all(\n cur_obs.index[..., 0]\n == [\n 0,\n 1,\n 2,\n 3,\n 4,\n 0,\n 1,\n 2,\n 3,\n 4,\n 0,\n 1,\n 0,\n 1,\n 0,\n 1,\n 0,\n 1,\n 2,\n 0,\n 1,\n 2,\n 0,\n 1,\n 2,\n 0,\n 1,\n 2,\n 3,\n 0,\n 1,\n 2,\n 3,\n 0,\n 1,\n 2,\n 3,\n 4,\n 0,\n 1,\n 2,\n 3,\n 4,\n ],\n ), cur_obs.index[..., 0]\n c2 = Collector[CollectStats](\n policy,\n envs,\n VectorReplayBuffer(total_size=100, buffer_num=4, stack_num=4),\n )\n c2.reset()\n c2.collect(n_episode=10)\n batch, _ = c2.buffer.sample(10)\n\n\ndef test_collector_with_multi_agent() -> None:\n multi_agent_env = MoveToRightEnv(size=5, sleep=0, ma_rew=4)\n policy = MaxActionPolicy()\n c_single_env = Collector[CollectStats](policy, multi_agent_env, ReplayBuffer(size=100))\n c_single_env.reset()\n multi_env_returns = c_single_env.collect(n_step=3).returns\n # c_single_env has length 3\n # We have no full episodes, so no returns yet\n assert len(multi_env_returns) == 0\n\n single_env_returns = c_single_env.collect(n_episode=2).returns\n # now two episodes. Since we have 4 a agents, the returns have shape (2, 4)\n assert single_env_returns.shape == (2, 4)\n assert np.all(single_env_returns == 1)\n\n env_fns = [lambda x=i: MoveToRightEnv(size=x, sleep=0, ma_rew=4) for i in [2, 3, 4, 5]]\n envs = DummyVectorEnv(env_fns)\n c_multi_env_ma = Collector[CollectStats](\n policy,\n envs,\n VectorReplayBuffer(total_size=100, buffer_num=4),\n )\n c_multi_env_ma.reset()\n multi_env_returns = c_multi_env_ma.collect(n_step=12).returns\n # each env makes 3 steps, the first two envs are done and result in two finished episodes\n assert multi_env_returns.shape == (2, 4) and np.all(multi_env_returns == 1), multi_env_returns\n multi_env_returns = c_multi_env_ma.collect(n_episode=8).returns\n assert multi_env_returns.shape == (8, 4)\n assert np.all(multi_env_returns == 1)\n batch, _ = c_multi_env_ma.buffer.sample(10)\n print(batch)\n c_single_env.buffer.update(c_multi_env_ma.buffer)\n assert len(c_single_env.buffer) in [42, 43]\n if len(c_single_env.buffer) == 42:\n multi_env_returns = np.array(\n [\n 0,\n 0,\n 0,\n 0,\n 1,\n 0,\n 0,\n 0,\n 0,\n 1,\n 0,\n 1,\n 0,\n 1,\n 0,\n 1,\n 0,\n 1,\n 0,\n 0,\n 1,\n 0,\n 0,\n 1,\n 0,\n 0,\n 0,\n 1,\n 0,\n 0,\n 0,\n 1,\n 0,\n 0,\n 0,\n 0,\n 1,\n 0,\n 0,\n 0,\n 0,\n 1,\n ],\n )\n else:\n multi_env_returns = np.array(\n [\n 0,\n 0,\n 0,\n 0,\n 1,\n 0,\n 0,\n 0,\n 0,\n 1,\n 0,\n 1,\n 0,\n 1,\n 0,\n 1,\n 0,\n 0,\n 1,\n 0,\n 0,\n 1,\n 0,\n 0,\n 1,\n 0,\n 0,\n 0,\n 1,\n 0,\n 0,\n 0,\n 1,\n 0,\n 0,\n 0,\n 0,\n 1,\n 0,\n 0,\n 0,\n 0,\n 1,\n ],\n )\n assert np.all(c_single_env.buffer[:].rew == [[x] * 4 for x in multi_env_returns])\n assert np.all(c_single_env.buffer[:].done == multi_env_returns)\n c2 = Collector[CollectStats](\n policy,\n envs,\n VectorReplayBuffer(total_size=100, buffer_num=4, stack_num=4),\n )\n c2.reset()\n multi_env_returns = c2.collect(n_episode=10).returns\n assert multi_env_returns.shape == (10, 4)\n assert np.all(multi_env_returns == 1)\n batch, _ = c2.buffer.sample(10)\n\n\ndef test_collector_with_atari_setting() -> None:\n reference_obs = np.zeros([6, 4, 84, 84])\n for i in range(6):\n reference_obs[i, 3, np.arange(84), np.arange(84)] = i\n reference_obs[i, 2, np.arange(84)] = i\n reference_obs[i, 1, :, np.arange(84)] = i\n reference_obs[i, 0] = i\n\n # atari single buffer\n env = MoveToRightEnv(size=5, sleep=0, array_state=True)\n policy = MaxActionPolicy()\n c0 = Collector[CollectStats](policy, env, ReplayBuffer(size=100))\n c0.reset()\n c0.collect(n_step=6)\n c0.collect(n_episode=2)\n assert c0.buffer.obs.shape == (100, 4, 84, 84)\n assert c0.buffer.obs_next.shape == (100, 4, 84, 84)\n assert len(c0.buffer) == 15 # 6 + 2 episodes with 5 steps each\n obs = np.zeros_like(c0.buffer.obs)\n obs[np.arange(15)] = reference_obs[np.arange(15) % 5]\n assert np.all(obs == c0.buffer.obs)\n\n c1 = Collector[CollectStats](policy, env, ReplayBuffer(size=100, ignore_obs_next=True))\n c1.collect(n_episode=3, reset_before_collect=True)\n assert np.allclose(c0.buffer.obs, c1.buffer.obs)\n with pytest.raises(AttributeError):\n c1.buffer.obs_next # noqa: B018\n assert np.all(reference_obs[[1, 2, 3, 4, 4] * 3] == c1.buffer[:].obs_next)\n\n c2 = Collector[CollectStats](\n policy,\n env,\n ReplayBuffer(size=100, ignore_obs_next=True, save_only_last_obs=True),\n )\n c2.reset()\n c2.collect(n_step=8)\n assert c2.buffer.obs.shape == (100, 84, 84)\n obs = np.zeros_like(c2.buffer.obs)\n obs[np.arange(8)] = reference_obs[[0, 1, 2, 3, 4, 0, 1, 2], -1]\n assert np.all(c2.buffer.obs == obs)\n obs_next = c2.buffer[:].obs_next\n assert isinstance(obs_next, np.ndarray)\n assert np.allclose(obs_next, reference_obs[[1, 2, 3, 4, 4, 1, 2, 2], -1])\n\n # atari multi buffer\n env_fns = [lambda x=i: MoveToRightEnv(size=x, sleep=0, array_state=True) for i in [2, 3, 4, 5]]\n envs = DummyVectorEnv(env_fns)\n c3 = Collector[CollectStats](policy, envs, VectorReplayBuffer(total_size=100, buffer_num=4))\n c3.reset()\n c3.collect(n_step=12)\n result_cached_buffer_collect_9_episodes = c3.collect(n_episode=9)\n assert result_cached_buffer_collect_9_episodes.n_collected_episodes == 9\n assert result_cached_buffer_collect_9_episodes.n_collected_steps == 23\n assert c3.buffer.obs.shape == (100, 4, 84, 84)\n obs = np.zeros_like(c3.buffer.obs)\n obs[np.arange(8)] = reference_obs[[0, 1, 0, 1, 0, 1, 0, 1]]\n obs[np.arange(25, 34)] = reference_obs[[0, 1, 2, 0, 1, 2, 0, 1, 2]]\n obs[np.arange(50, 58)] = reference_obs[[0, 1, 2, 3, 0, 1, 2, 3]]\n obs[np.arange(75, 85)] = reference_obs[[0, 1, 2, 3, 4, 0, 1, 2, 3, 4]]\n assert np.all(obs == c3.buffer.obs)\n obs_next = np.zeros_like(c3.buffer.obs_next)\n obs_next[np.arange(8)] = reference_obs[[1, 2, 1, 2, 1, 2, 1, 2]]\n obs_next[np.arange(25, 34)] = reference_obs[[1, 2, 3, 1, 2, 3, 1, 2, 3]]\n obs_next[np.arange(50, 58)] = reference_obs[[1, 2, 3, 4, 1, 2, 3, 4]]\n obs_next[np.arange(75, 85)] = reference_obs[[1, 2, 3, 4, 5, 1, 2, 3, 4, 5]]\n assert np.all(obs_next == c3.buffer.obs_next)\n c4 = Collector[CollectStats](\n policy,\n envs,\n VectorReplayBuffer(\n total_size=100,\n buffer_num=4,\n stack_num=4,\n ignore_obs_next=True,\n save_only_last_obs=True,\n ),\n )\n c4.reset()\n c4.collect(n_step=12)\n result_cached_buffer_collect_9_episodes = c4.collect(n_episode=9)\n assert result_cached_buffer_collect_9_episodes.n_collected_episodes == 9\n assert result_cached_buffer_collect_9_episodes.n_collected_steps == 23\n assert c4.buffer.obs.shape == (100, 84, 84)\n obs = np.zeros_like(c4.buffer.obs)\n slice_obs = reference_obs[:, -1]\n obs[np.arange(8)] = slice_obs[[0, 1, 0, 1, 0, 1, 0, 1]]\n obs[np.arange(25, 34)] = slice_obs[[0, 1, 2, 0, 1, 2, 0, 1, 2]]\n obs[np.arange(50, 58)] = slice_obs[[0, 1, 2, 3, 0, 1, 2, 3]]\n obs[np.arange(75, 85)] = slice_obs[[0, 1, 2, 3, 4, 0, 1, 2, 3, 4]]\n assert np.all(c4.buffer.obs == obs)\n obs_next = np.zeros([len(c4.buffer), 4, 84, 84])\n ref_index = np.array(\n [\n 1,\n 1,\n 1,\n 1,\n 1,\n 1,\n 1,\n 1,\n 1,\n 2,\n 2,\n 1,\n 2,\n 2,\n 1,\n 2,\n 2,\n 1,\n 2,\n 3,\n 3,\n 1,\n 2,\n 3,\n 3,\n 1,\n 2,\n 3,\n 4,\n 4,\n 1,\n 2,\n 3,\n 4,\n 4,\n ],\n )\n obs_next[:, -1] = slice_obs[ref_index]\n ref_index -= 1\n ref_index[ref_index < 0] = 0\n obs_next[:, -2] = slice_obs[ref_index]\n ref_index -= 1\n ref_index[ref_index < 0] = 0\n obs_next[:, -3] = slice_obs[ref_index]\n ref_index -= 1\n ref_index[ref_index < 0] = 0\n obs_next[:, -4] = slice_obs[ref_index]\n assert np.all(obs_next == c4.buffer[:].obs_next)\n\n buf = ReplayBuffer(100, stack_num=4, ignore_obs_next=True, save_only_last_obs=True)\n collector_cached_buffer = Collector[CollectStats](policy, envs, CachedReplayBuffer(buf, 4, 10))\n collector_cached_buffer.reset()\n result_cached_buffer_collect_12_steps = collector_cached_buffer.collect(n_step=12)\n assert len(buf) == 5\n assert len(collector_cached_buffer.buffer) == 12\n result_cached_buffer_collect_9_episodes = collector_cached_buffer.collect(n_episode=9)\n assert result_cached_buffer_collect_9_episodes.n_collected_episodes == 9\n assert result_cached_buffer_collect_9_episodes.n_collected_steps == 23\n assert len(buf) == 35\n assert np.all(\n buf.obs[: len(buf)]\n == slice_obs[\n [\n 0,\n 1,\n 0,\n 1,\n 2,\n 0,\n 1,\n 0,\n 1,\n 2,\n 3,\n 0,\n 1,\n 2,\n 3,\n 4,\n 0,\n 1,\n 0,\n 1,\n 2,\n 0,\n 1,\n 0,\n 1,\n 2,\n 3,\n 0,\n 1,\n 2,\n 0,\n 1,\n 2,\n 3,\n 4,\n ]\n ],\n )\n assert np.all(\n buf[:].obs_next[:, -1]\n == slice_obs[\n [\n 1,\n 1,\n 1,\n 2,\n 2,\n 1,\n 1,\n 1,\n 2,\n 3,\n 3,\n 1,\n 2,\n 3,\n 4,\n 4,\n 1,\n 1,\n 1,\n 2,\n 2,\n 1,\n 1,\n 1,\n 2,\n 3,\n 3,\n 1,\n 2,\n 2,\n 1,\n 2,\n 3,\n 4,\n 4,\n ]\n ],\n )\n assert len(buf) == len(collector_cached_buffer.buffer)\n\n # test buffer=None\n collector_default_buffer = Collector[CollectStats](policy, envs)\n collector_default_buffer.reset()\n result_default_buffer_collect_12_steps = collector_default_buffer.collect(n_step=12)\n for key in [\"n_collected_episodes\", \"n_collected_steps\", \"returns\", \"lens\"]:\n assert np.allclose(\n getattr(result_default_buffer_collect_12_steps, key),\n getattr(result_cached_buffer_collect_12_steps, key),\n )\n result2 = collector_default_buffer.collect(n_episode=9)\n for key in [\"n_collected_episodes\", \"n_collected_steps\", \"returns\", \"lens\"]:\n assert np.allclose(\n getattr(result2, key),\n getattr(result_cached_buffer_collect_9_episodes, key),\n )\n\n\n@pytest.mark.skipif(envpool is None, reason=\"EnvPool doesn't support this platform\")\ndef test_collector_envpool_gym_reset_return_info() -> None:\n envs = envpool.make_gymnasium(\"Pendulum-v1\", num_envs=4, gym_reset_return_info=True)\n policy = MaxActionPolicy(action_shape=(len(envs), 1))\n\n c0 = Collector[CollectStats](\n policy,\n envs,\n VectorReplayBuffer(len(envs) * 10, len(envs)),\n exploration_noise=True,\n )\n c0.reset()\n c0.collect(n_step=8)\n env_ids = np.zeros(len(envs) * 10)\n env_ids[[0, 1, 10, 11, 20, 21, 30, 31]] = [0, 0, 1, 1, 2, 2, 3, 3]\n assert np.allclose(c0.buffer.info[\"env_id\"], env_ids)\n\n\ndef test_collector_with_vector_env() -> None:\n env_fns = [lambda x=i: MoveToRightEnv(size=x, sleep=0) for i in [1, 8, 9, 10]]\n\n dum = DummyVectorEnv(env_fns)\n policy = MaxActionPolicy()\n\n c2 = Collector[CollectStats](\n policy,\n dum,\n VectorReplayBuffer(total_size=100, buffer_num=4),\n )\n\n c2.reset()\n\n c1r = c2.collect(n_episode=2)\n assert np.array_equal(np.array([1, 8]), c1r.lens)\n c2r = c2.collect(n_episode=10)\n assert np.array_equal(np.array([1, 1, 1, 1, 1, 1, 1, 8, 9, 10]), c2r.lens)\n c3r = c2.collect(n_step=20)\n assert np.array_equal(np.array([1, 1, 1, 1, 1]), c3r.lens)\n c4r = c2.collect(n_step=20)\n assert np.array_equal(np.array([1, 1, 1, 8, 1, 9, 1, 10]), c4r.lens)\n\n\ndef test_async_collector_with_vector_env() -> None:\n env_fns = [lambda x=i: MoveToRightEnv(size=x, sleep=0) for i in [1, 8, 9, 10]]\n\n dum = DummyVectorEnv(env_fns)\n policy = MaxActionPolicy()\n c1 = AsyncCollector(\n policy,\n dum,\n VectorReplayBuffer(total_size=100, buffer_num=4),\n )\n\n c1r = c1.collect(n_episode=10, reset_before_collect=True)\n assert np.array_equal(np.array([1, 1, 1, 1, 1, 1, 1, 1, 8, 1, 9]), c1r.lens)\n c2r = c1.collect(n_step=20)\n assert np.array_equal(np.array([1, 10, 1, 1, 1, 1]), c2r.lens)\n\n\nclass StepHookAddFieldToBatch(StepHook):\n def __call__(\n self,\n action_batch: CollectActionBatchProtocol,\n rollout_batch: RolloutBatchProtocol,\n ) -> None:\n rollout_batch.set_array_at_key(np.array([1]), \"added_by_hook\")\n\n\nclass TestCollectStatsAndHooks:\n @staticmethod\n def test_on_step_hook(collector_with_single_env: Collector) -> None:\n collector_with_single_env.set_on_step_hook(StepHookAddFieldToBatch())\n collect_stats = collector_with_single_env.collect(n_step=3)\n assert collect_stats.n_collected_steps == 3\n # a was added by the hook\n assert np.array_equal(\n collector_with_single_env.buffer[:].added_by_hook,\n np.array([1, 1, 1]),\n )\n\n @staticmethod\n def test_episode_mc_hook(collector_with_single_env: Collector) -> None:\n collector_with_single_env.set_on_episode_done_hook(EpisodeRolloutHookMCReturn())\n collector_with_single_env.collect(n_episode=1)\n collected_batch = collector_with_single_env.buffer[:]\n return_to_go = collected_batch.get(EpisodeRolloutHookMCReturn.MC_RETURN_TO_GO_KEY)\n full_return = collected_batch.get(EpisodeRolloutHookMCReturn.FULL_EPISODE_MC_RETURN_KEY)\n assert np.array_equal(return_to_go, episode_mc_return_to_go(collected_batch.rew))\n assert np.array_equal(full_return, np.ones(5) * return_to_go[0])\n" }, { "path": "test/base/test_env.py", "content": "import sys\nimport time\nfrom collections.abc import Callable\nfrom typing import Any, Literal\n\nimport gymnasium as gym\nimport numpy as np\nimport pytest\nfrom gymnasium.spaces.discrete import Discrete\n\nfrom test.base.env import MoveToRightEnv, NXEnv\nfrom tianshou.data import Batch\nfrom tianshou.env import (\n ContinuousToDiscrete,\n DummyVectorEnv,\n MultiDiscreteToDiscrete,\n RayVectorEnv,\n ShmemVectorEnv,\n SubprocVectorEnv,\n VectorEnvNormObs,\n)\nfrom tianshou.env.gym_wrappers import TruncatedAsTerminated\nfrom tianshou.env.venvs import BaseVectorEnv\nfrom tianshou.utils import RunningMeanStd\n\ntry:\n import envpool\nexcept ImportError:\n envpool = None\n\n\ndef has_ray() -> bool:\n try:\n import ray # noqa: F401\n\n return True\n except ImportError:\n return False\n\n\ndef recurse_comp(a: np.ndarray | list | tuple | dict, b: Any) -> np.bool_ | bool | None:\n try:\n if isinstance(a, np.ndarray):\n if a.dtype == object:\n return np.array([recurse_comp(m, n) for m, n in zip(a, b, strict=True)]).all()\n return np.allclose(a, b)\n if isinstance(a, list | tuple):\n return np.array([recurse_comp(m, n) for m, n in zip(a, b, strict=True)]).all()\n if isinstance(a, dict):\n return np.array([recurse_comp(a[k], b[k]) for k in a]).all()\n except Exception:\n return False\n\n\ndef test_async_env(size: int = 10000, num: int = 8, sleep: float = 0.1) -> None:\n # simplify the test case, just keep stepping\n env_fns = [\n lambda i=i: MoveToRightEnv(size=i, sleep=sleep, random_sleep=True)\n for i in range(size, size + num)\n ]\n test_cls = [SubprocVectorEnv, ShmemVectorEnv]\n if has_ray():\n test_cls += [RayVectorEnv]\n for cls in test_cls:\n v = cls(env_fns, wait_num=num // 2, timeout=1e-3)\n v.seed(None)\n v.reset()\n # for a random variable u ~ U[0, 1], let v = max{u1, u2, ..., un}\n # P(v <= x) = x^n (0 <= x <= 1), pdf of v is nx^{n-1}\n # expectation of v is n / (n + 1)\n # for a synchronous environment, the following actions should take\n # about 7 * sleep * num / (num + 1) seconds\n # for async simulation, the analysis is complicated, but the time cost\n # should be smaller\n action_list = [1] * num + [0] * (num * 2) + [1] * (num * 4)\n current_idx_start = 0\n act = action_list[:num]\n env_ids = list(range(num))\n o = []\n spent_time = time.time()\n while current_idx_start < len(action_list):\n (\n A,\n B,\n C,\n D,\n E,\n ) = v.step(action=act, id=env_ids)\n b = Batch({\"obs\": A, \"rew\": B, \"terminate\": C, \"truncated\": D, \"info\": E})\n env_ids = b.info.env_id\n o.append(b)\n current_idx_start += len(act)\n # len of action may be smaller than len(A) in the end\n act = action_list[current_idx_start : current_idx_start + len(A)]\n # truncate env_ids with the first terms\n # typically len(env_ids) == len(A) == len(action), except for the\n # last batch when actions are not enough\n env_ids = env_ids[: len(act)]\n spent_time = time.time() - spent_time\n Batch.cat(o)\n v.close()\n # assure 1/7 improvement\n if sys.platform == \"linux\" and cls != RayVectorEnv:\n # macOS/Windows cannot pass this check\n assert spent_time < 6.0 * sleep * num / (num + 1)\n\n\ndef test_async_check_id(\n size: int = 100,\n num: int = 4,\n sleep: float = 0.2,\n timeout: float = 0.7,\n) -> None:\n env_fns = [\n lambda: MoveToRightEnv(size=size, sleep=sleep * 2),\n lambda: MoveToRightEnv(size=size, sleep=sleep * 3),\n lambda: MoveToRightEnv(size=size, sleep=sleep * 5),\n lambda: MoveToRightEnv(size=size, sleep=sleep * 7),\n ]\n test_cls = [SubprocVectorEnv, ShmemVectorEnv]\n if has_ray():\n test_cls += [RayVectorEnv]\n total_pass = 0\n for cls in test_cls:\n pass_check = 1\n v = cls(env_fns, wait_num=num - 1, timeout=timeout)\n t = time.time()\n v.reset()\n t = time.time() - t\n print(f\"{cls} reset {t}\")\n if t > sleep * 9: # huge than maximum sleep time (7 sleep)\n pass_check = 0\n expect_result = [\n [0, 1],\n [0, 1, 2],\n [0, 1, 3],\n [0, 1, 2],\n [0, 1],\n [0, 2, 3],\n [0, 1],\n ]\n ids = np.arange(num)\n for res in expect_result:\n t = time.time()\n _, _, _, _, info = v.step([1] * len(ids), ids)\n t = time.time() - t\n ids = Batch(info).env_id\n print(ids, t)\n if not (\n len(ids) == len(res)\n and np.allclose(sorted(ids), res)\n and (t < timeout) == (len(res) == num - 1)\n ):\n pass_check = 0\n break\n total_pass += pass_check\n if sys.platform == \"linux\": # Windows/macOS may not pass this check\n assert total_pass >= 2\n\n\ndef test_vecenv(size: int = 10, num: int = 8, sleep: float = 0.001) -> None:\n env_fns = [\n lambda i=i: MoveToRightEnv(size=i, sleep=sleep, recurse_state=True)\n for i in range(size, size + num)\n ]\n venv = [\n DummyVectorEnv(env_fns),\n SubprocVectorEnv(env_fns),\n ShmemVectorEnv(env_fns),\n ]\n if has_ray() and sys.platform == \"linux\":\n venv += [RayVectorEnv(env_fns)]\n for v in venv:\n v.seed(0)\n action_list = [1] * 5 + [0] * 10 + [1] * 20\n for a in action_list:\n o = []\n for v in venv:\n A, B, C, D, E = v.step(np.array([a] * num))\n if sum(C + D):\n A, _ = v.reset(np.where(C + D)[0])\n o.append([A, B, C, D, E])\n for index, infos in enumerate(zip(*o, strict=True)):\n if index == 4: # do not check info here\n continue\n for info in infos:\n assert recurse_comp(infos[0], info)\n\n def assert_get(v: BaseVectorEnv, expected: list) -> None:\n assert v.get_env_attr(\"size\") == expected\n assert v.get_env_attr(\"size\", id=0) == [expected[0]]\n assert v.get_env_attr(\"size\", id=[0, 1, 2]) == expected[:3]\n\n for v in venv:\n assert_get(v, list(range(size, size + num)))\n assert v.env_num == num\n assert v.action_space == [Discrete(2)] * num\n\n v.set_env_attr(\"size\", 0)\n assert_get(v, [0] * num)\n\n v.set_env_attr(\"size\", 1, 0)\n assert_get(v, [1] + [0] * (num - 1))\n\n v.set_env_attr(\"size\", 2, [1, 2, 3])\n assert_get(v, [1] + [2] * 3 + [0] * (num - 4))\n\n for v in venv:\n v.close()\n\n\ndef test_attr_unwrapped() -> None:\n training_envs = DummyVectorEnv([lambda: gym.make(\"CartPole-v1\")])\n training_envs.set_env_attr(\"test_attribute\", 1337)\n assert training_envs.get_env_attr(\"test_attribute\") == [1337]\n assert hasattr(training_envs.workers[0].env.unwrapped, \"test_attribute\") # type: ignore\n\n\ndef test_env_obs_dtype() -> None:\n def create_env(i: int, t: str) -> Callable[[], NXEnv]:\n return lambda: NXEnv(i, t)\n\n for obs_type in [\"array\", \"object\"]:\n envs = SubprocVectorEnv([create_env(x, obs_type) for x in [5, 10, 15, 20]])\n obs, info = envs.reset()\n assert obs.dtype == object\n obs = envs.step(np.array([1, 1, 1, 1]))[0]\n assert obs.dtype == object\n\n\ndef test_env_reset_optional_kwargs(size: int = 10000, num: int = 8) -> None:\n env_fns = [lambda i=i: MoveToRightEnv(size=i) for i in range(size, size + num)]\n test_cls = [DummyVectorEnv, SubprocVectorEnv, ShmemVectorEnv]\n if has_ray():\n test_cls += [RayVectorEnv]\n for cls in test_cls:\n v = cls(env_fns, wait_num=num // 2, timeout=1e-3)\n _, info = v.reset(seed=1)\n assert len(info) == len(env_fns)\n assert isinstance(info[0], dict)\n\n\ndef test_venv_wrapper_gym(num_envs: int = 4) -> None:\n # Issue 697\n envs = DummyVectorEnv([lambda: gym.make(\"CartPole-v1\") for _ in range(num_envs)])\n envs = VectorEnvNormObs(envs)\n try:\n obs, info = envs.reset()\n except ValueError:\n obs, info = envs.reset(return_info=True)\n assert isinstance(obs, np.ndarray)\n assert isinstance(info, np.ndarray)\n assert isinstance(info[0], dict)\n assert obs.shape[0] == len(info) == num_envs\n\n\ndef run_align_norm_obs(\n raw_env: DummyVectorEnv,\n train_env: VectorEnvNormObs,\n test_env: VectorEnvNormObs,\n action_list: list[np.ndarray],\n) -> None:\n def reset_result_to_obs(\n reset_result: tuple[np.ndarray, dict | list[dict]],\n ) -> np.ndarray:\n \"\"\"Extract observation from reset result (result is possibly a tuple containing info).\"\"\"\n if isinstance(reset_result, tuple) and len(reset_result) == 2:\n obs, _ = reset_result\n else:\n obs = reset_result # type: ignore\n return obs\n\n eps = np.finfo(np.float32).eps.item()\n raw_reset_result = raw_env.reset()\n train_reset_result = train_env.reset()\n initial_raw_obs = reset_result_to_obs(raw_reset_result) # type: ignore\n initial_train_obs = reset_result_to_obs(train_reset_result) # type: ignore\n raw_obs, train_obs = [initial_raw_obs], [initial_train_obs]\n for action in action_list:\n step_result = raw_env.step(action)\n if len(step_result) == 5:\n obs, rew, terminated, truncated, info = step_result\n done = np.logical_or(terminated, truncated)\n else:\n obs, rew, done, info = step_result # type: ignore\n raw_obs.append(obs)\n if np.any(done):\n reset_result = raw_env.reset(np.where(done)[0])\n obs = reset_result_to_obs(reset_result) # type: ignore\n raw_obs.append(obs)\n step_result = train_env.step(action)\n if len(step_result) == 5:\n obs, rew, terminated, truncated, info = step_result\n done = np.logical_or(terminated, truncated)\n else:\n obs, rew, done, info = step_result # type: ignore\n train_obs.append(obs)\n if np.any(done):\n reset_result = train_env.reset(np.where(done)[0])\n obs = reset_result_to_obs(reset_result) # type: ignore\n train_obs.append(obs)\n ref_rms = RunningMeanStd()\n for ro, to in zip(raw_obs, train_obs, strict=True):\n ref_rms.update(ro)\n no = (ro - ref_rms.mean) / np.sqrt(ref_rms.var + eps)\n assert np.allclose(no, to)\n assert np.allclose(ref_rms.mean, train_env.get_obs_rms().mean)\n assert np.allclose(ref_rms.var, train_env.get_obs_rms().var)\n assert np.allclose(ref_rms.mean, test_env.get_obs_rms().mean)\n assert np.allclose(ref_rms.var, test_env.get_obs_rms().var)\n reset_result = test_env.reset()\n obs = reset_result_to_obs(reset_result) # type: ignore\n test_obs = [obs]\n for action in action_list:\n step_result = test_env.step(action)\n if len(step_result) == 5:\n obs, rew, terminated, truncated, info = step_result\n done = np.logical_or(terminated, truncated)\n else:\n obs, rew, done, info = step_result # type: ignore\n test_obs.append(obs)\n if np.any(done):\n reset_result = test_env.reset(np.where(done)[0])\n obs = reset_result_to_obs(reset_result) # type: ignore\n test_obs.append(obs)\n for ro, to in zip(raw_obs, test_obs, strict=True):\n no = (ro - ref_rms.mean) / np.sqrt(ref_rms.var + eps)\n assert np.allclose(no, to)\n\n\ndef test_venv_norm_obs() -> None:\n sizes = np.array([5, 10, 15, 20])\n action = np.array([1, 1, 1, 1])\n total_step = 30\n action_list = [action] * total_step\n env_fns = [lambda i=x: MoveToRightEnv(size=i, array_state=True) for x in sizes]\n raw = DummyVectorEnv(env_fns)\n train_env = VectorEnvNormObs(DummyVectorEnv(env_fns))\n print(train_env.observation_space)\n test_env = VectorEnvNormObs(DummyVectorEnv(env_fns), update_obs_rms=False)\n test_env.set_obs_rms(train_env.get_obs_rms())\n run_align_norm_obs(raw, train_env, test_env, action_list)\n\n\ndef test_gym_wrappers() -> None:\n class DummyEnv(gym.Env):\n def __init__(self) -> None:\n self.action_space = gym.spaces.Box(low=-1.0, high=2.0, shape=(4,), dtype=np.float32)\n self.observation_space = gym.spaces.Discrete(2)\n\n def step(self, act: Any) -> tuple[Any, Literal[-1], Literal[False], Literal[True], dict]:\n return self.observation_space.sample(), -1, False, True, {}\n\n bsz = 10\n action_per_branch = [4, 6, 10, 7]\n env = DummyEnv()\n assert isinstance(env.action_space, gym.spaces.Box)\n original_act = env.action_space.high\n # convert continous to multidiscrete action space\n # with different action number per dimension\n env_m = ContinuousToDiscrete(env, action_per_branch)\n assert isinstance(env_m.action_space, gym.spaces.MultiDiscrete)\n # check conversion is working properly for one action\n np.testing.assert_allclose(env_m.action(env_m.action_space.nvec - 1), original_act)\n # check conversion is working properly for a batch of actions\n np.testing.assert_allclose(\n env_m.action(np.array([env_m.action_space.nvec - 1] * bsz)),\n np.array([original_act] * bsz),\n )\n # convert multidiscrete with different action number per\n # dimension to discrete action space\n env_d = MultiDiscreteToDiscrete(env_m)\n assert isinstance(env_d.action_space, gym.spaces.Discrete)\n # check conversion is working properly for one action\n np.testing.assert_allclose(\n env_d.action(np.array(env_d.action_space.n - 1)),\n env_m.action_space.nvec - 1,\n )\n # check conversion is working properly for a batch of actions\n np.testing.assert_allclose(\n env_d.action(np.array([env_d.action_space.n - 1] * bsz)),\n np.array([env_m.action_space.nvec - 1] * bsz),\n )\n # check truncate is True when terminated\n try:\n env_t = TruncatedAsTerminated(env)\n except OSError:\n env_t = None\n if env_t is not None:\n _, _, truncated, _, _ = env_t.step(env_t.action_space.sample())\n assert truncated\n\n\n# TODO: old gym envs are no longer supported! Replace by Ant-v4 and fix assoticiated tests\n@pytest.mark.skipif(envpool is None, reason=\"EnvPool doesn't support this platform\")\ndef test_venv_wrapper_envpool() -> None:\n raw = envpool.make_gymnasium(\"Ant-v3\", num_envs=4)\n train = VectorEnvNormObs(envpool.make_gymnasium(\"Ant-v3\", num_envs=4))\n test = VectorEnvNormObs(envpool.make_gymnasium(\"Ant-v3\", num_envs=4), update_obs_rms=False)\n test.set_obs_rms(train.get_obs_rms())\n actions = [np.array([raw.action_space.sample() for _ in range(4)]) for i in range(30)]\n run_align_norm_obs(raw, train, test, actions)\n\n\n@pytest.mark.skipif(envpool is None, reason=\"EnvPool doesn't support this platform\")\ndef test_venv_wrapper_envpool_gym_reset_return_info() -> None:\n num_envs = 4\n env = VectorEnvNormObs(\n envpool.make_gymnasium(\"Ant-v3\", num_envs=num_envs, gym_reset_return_info=True),\n )\n obs, info = env.reset()\n assert obs.shape[0] == num_envs\n # This is not actually unreachable b/c envpool does not return info in the right format\n if isinstance(info, dict): # type: ignore[unreachable]\n for _, v in info.items(): # type: ignore[unreachable]\n if not isinstance(v, dict):\n assert v.shape[0] == num_envs\n else:\n for _info in info:\n for _, v in _info.items():\n if not isinstance(v, dict):\n assert v.shape[0] == num_envs\n" }, { "path": "test/base/test_env_finite.py", "content": "# see issue #322 for detail\n\nimport copy\nfrom collections import Counter\nfrom collections.abc import Callable, Iterator, Sequence\nfrom typing import Any, cast\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom gymnasium.spaces import Box\nfrom torch.utils.data import DataLoader, Dataset, DistributedSampler\n\nfrom tianshou.algorithm.algorithm_base import Policy\nfrom tianshou.data import Batch, Collector, CollectStats\nfrom tianshou.data.types import (\n ActBatchProtocol,\n BatchProtocol,\n ObsBatchProtocol,\n)\nfrom tianshou.env import BaseVectorEnv, DummyVectorEnv, SubprocVectorEnv\nfrom tianshou.env.utils import ENV_TYPE, gym_new_venv_step_type\n\n\nclass DummyDataset(Dataset):\n def __init__(self, length: int) -> None:\n self.length = length\n self.episodes = [3 * i % 5 + 1 for i in range(self.length)]\n\n def __getitem__(self, index: int) -> tuple[int, int]:\n assert 0 <= index < self.length\n return index, self.episodes[index]\n\n def __len__(self) -> int:\n return self.length\n\n\nclass FiniteEnv(gym.Env):\n def __init__(self, dataset: Dataset, num_replicas: int | None, rank: int | None) -> None:\n self.dataset = dataset\n self.num_replicas = num_replicas\n self.rank = rank\n self.loader = DataLoader(\n dataset,\n sampler=DistributedSampler(dataset, num_replicas, rank),\n batch_size=None,\n )\n self.iterator: Iterator | None = None\n\n def reset(\n self,\n *,\n seed: int | None = None,\n options: dict[str, Any] | None = None,\n ) -> tuple[Any, dict[str, Any]]:\n if self.iterator is None:\n self.iterator = iter(self.loader)\n try:\n self.current_sample, self.step_count = next(self.iterator)\n self.current_step = 0\n return self.current_sample, {}\n except StopIteration:\n self.iterator = None\n return None, {}\n\n def step(self, action: int) -> tuple[int, float, bool, bool, dict[str, Any]]:\n self.current_step += 1\n assert self.current_step <= self.step_count\n return (\n 0,\n 1.0,\n self.current_step >= self.step_count,\n False,\n {\"sample\": self.current_sample, \"action\": action, \"metric\": 2.0},\n )\n\n\nclass FiniteVectorEnv(BaseVectorEnv):\n def __init__(self, env_fns: Sequence[Callable[[], ENV_TYPE]], **kwargs: Any) -> None:\n super().__init__(env_fns, **kwargs)\n self._alive_env_ids: set[int] = set()\n self._reset_alive_envs()\n self._default_obs: np.ndarray | None = None\n self._default_info: dict | None = None\n self.tracker: MetricTracker\n\n def _reset_alive_envs(self) -> None:\n if not self._alive_env_ids:\n # starting or running out\n self._alive_env_ids = set(range(self.env_num))\n\n # to workaround with tianshou's buffer and batch\n def _set_default_obs(self, obs: np.ndarray) -> None:\n if obs is not None and self._default_obs is None:\n self._default_obs = copy.deepcopy(obs)\n\n def _set_default_info(self, info: dict) -> None:\n if info is not None and self._default_info is None:\n self._default_info = copy.deepcopy(info)\n\n def _get_default_obs(self) -> np.ndarray | None:\n return copy.deepcopy(self._default_obs)\n\n def _get_default_info(self) -> dict | None:\n return copy.deepcopy(self._default_info)\n\n # END\n\n def reset(\n self,\n env_id: int | list[int] | np.ndarray | None = None,\n **kwargs: Any,\n ) -> tuple[np.ndarray, np.ndarray]:\n env_id = self._wrap_id(env_id)\n self._reset_alive_envs()\n\n # ask super to reset alive envs and remap to current index\n request_id = list(filter(lambda i: i in self._alive_env_ids, env_id))\n obs_list: list[np.ndarray | None] = [None] * len(env_id)\n infos: list[dict | None] = [None] * len(env_id)\n id2idx = {i: k for k, i in enumerate(env_id)}\n if request_id:\n for k, o, info in zip(request_id, *super().reset(request_id), strict=True):\n obs_list[id2idx[k]] = o\n infos[id2idx[k]] = info\n for i, o in zip(env_id, obs_list, strict=True):\n if o is None and i in self._alive_env_ids:\n self._alive_env_ids.remove(i)\n\n # fill empty observation with default(fake) observation\n for o in obs_list:\n self._set_default_obs(o)\n\n for i in range(len(obs_list)):\n if obs_list[i] is None:\n obs_list[i] = self._get_default_obs()\n if infos[i] is None:\n infos[i] = self._get_default_info()\n\n if not self._alive_env_ids:\n self.reset()\n raise StopIteration\n\n obs_list = cast(list[np.ndarray], obs_list)\n infos = cast(list[dict], infos)\n\n return np.stack(obs_list), np.array(infos)\n\n def step(\n self,\n action: np.ndarray | torch.Tensor | None,\n id: int | list[int] | np.ndarray | None = None,\n ) -> gym_new_venv_step_type:\n ids: list[int] | np.ndarray = self._wrap_id(id)\n id2idx = {i: k for k, i in enumerate(ids)}\n request_id = list(filter(lambda i: i in self._alive_env_ids, ids))\n result: list[list] = [[None, 0.0, False, False, None] for _ in range(len(ids))]\n\n # ask super to step alive envs and remap to current index\n assert action is not None\n if request_id:\n valid_act = np.stack([action[id2idx[i]] for i in request_id])\n for i, (r_obs, r_reward, r_term, r_trunc, r_info) in zip(\n request_id,\n zip(*super().step(valid_act, request_id), strict=True),\n strict=True,\n ):\n result[id2idx[i]] = [r_obs, r_reward, r_term, r_trunc, r_info]\n\n # logging\n for i, r in zip(ids, result, strict=True):\n if i in self._alive_env_ids:\n self.tracker.log(*r)\n\n # fill empty observation/info with default(fake)\n for _, __, ___, ____, i in result:\n self._set_default_info(i)\n for i in range(len(result)):\n if result[i][0] is None:\n result[i][0] = self._get_default_obs()\n if result[i][-1] is None:\n result[i][-1] = self._get_default_info()\n\n obs_list, rew_list, term_list, trunc_list, info_list = zip(*result, strict=True)\n try:\n obs_stack = np.stack(obs_list)\n except ValueError: # different len(obs)\n obs_stack = np.array(obs_list, dtype=object)\n return (\n obs_stack,\n np.stack(rew_list),\n np.stack(term_list),\n np.stack(trunc_list),\n np.stack(info_list),\n )\n\n\nclass FiniteDummyVectorEnv(FiniteVectorEnv, DummyVectorEnv):\n pass\n\n\nclass FiniteSubprocVectorEnv(FiniteVectorEnv, SubprocVectorEnv):\n pass\n\n\nclass DummyPolicy(Policy):\n def __init__(self) -> None:\n super().__init__(action_space=Box(-1, 1, (1,)))\n\n def forward(\n self,\n batch: ObsBatchProtocol,\n state: dict | BatchProtocol | np.ndarray | None = None,\n **kwargs: Any,\n ) -> ActBatchProtocol:\n return cast(ActBatchProtocol, Batch(act=np.stack([1] * len(batch))))\n\n\ndef _finite_env_factory(dataset: Dataset, num_replicas: int, rank: int) -> Callable[[], FiniteEnv]:\n return lambda: FiniteEnv(dataset, num_replicas, rank)\n\n\nclass MetricTracker:\n def __init__(self) -> None:\n self.counter: Counter = Counter()\n self.finished: set[int] = set()\n\n def log(self, obs: Any, rew: float, terminated: bool, truncated: bool, info: dict) -> None:\n assert rew == 1.0\n done = terminated or truncated\n index = info[\"sample\"]\n if done:\n assert index not in self.finished\n self.finished.add(index)\n self.counter[index] += 1\n\n def validate(self) -> None:\n assert len(self.finished) == 100\n for k, v in self.counter.items():\n assert v == k * 3 % 5 + 1\n\n\ndef test_finite_dummy_vector_env() -> None:\n dataset = DummyDataset(100)\n envs = FiniteSubprocVectorEnv([_finite_env_factory(dataset, 5, i) for i in range(5)])\n policy = DummyPolicy()\n test_collector = Collector[CollectStats](policy, envs, exploration_noise=True)\n test_collector.reset()\n\n for _ in range(3):\n envs.tracker = MetricTracker()\n try:\n # TODO: why on earth 10**18?\n test_collector.collect(n_step=10**18)\n except StopIteration:\n envs.tracker.validate()\n\n\ndef test_finite_subproc_vector_env() -> None:\n dataset = DummyDataset(100)\n envs = FiniteSubprocVectorEnv([_finite_env_factory(dataset, 5, i) for i in range(5)])\n policy = DummyPolicy()\n test_collector = Collector[CollectStats](policy, envs, exploration_noise=True)\n test_collector.reset()\n\n for _ in range(3):\n envs.tracker = MetricTracker()\n try:\n test_collector.collect(n_step=10**18)\n except StopIteration:\n envs.tracker.validate()\n" }, { "path": "test/base/test_logger.py", "content": "from typing import Literal\n\nimport numpy as np\nimport pytest\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom tianshou.utils import TensorboardLogger\n\n\nclass TestTensorBoardLogger:\n @staticmethod\n @pytest.mark.parametrize(\n \"input_dict, expected_output\",\n [\n ({\"a\": 1, \"b\": {\"c\": 2, \"d\": {\"e\": 3}}}, {\"a\": 1, \"b/c\": 2, \"b/d/e\": 3}),\n ({\"a\": {\"b\": {\"c\": 1}}}, {\"a/b/c\": 1}),\n ],\n )\n def test_flatten_dict_basic(\n input_dict: dict[str, int | dict[str, int | dict[str, int]]]\n | dict[str, dict[str, dict[str, int]]],\n expected_output: dict[str, int],\n ) -> None:\n logger = TensorboardLogger(SummaryWriter(\"log/logger\"))\n result = logger.prepare_dict_for_logging(input_dict)\n assert result == expected_output\n\n @staticmethod\n @pytest.mark.parametrize(\n \"input_dict, delimiter, expected_output\",\n [\n ({\"a\": {\"b\": {\"c\": 1}}}, \"|\", {\"a|b|c\": 1}),\n ({\"a\": {\"b\": {\"c\": 1}}}, \".\", {\"a.b.c\": 1}),\n ],\n )\n def test_flatten_dict_custom_delimiter(\n input_dict: dict[str, dict[str, dict[str, int]]],\n delimiter: Literal[\"|\", \".\"],\n expected_output: dict[str, int],\n ) -> None:\n logger = TensorboardLogger(SummaryWriter(\"log/logger\"))\n result = logger.prepare_dict_for_logging(input_dict, delimiter=delimiter)\n assert result == expected_output\n\n @staticmethod\n @pytest.mark.parametrize(\n \"input_dict, exclude_arrays, expected_output\",\n [\n (\n {\"a\": np.array([1, 2, 3]), \"b\": {\"c\": np.array([4, 5, 6])}},\n False,\n {\"a\": np.array([1, 2, 3]), \"b/c\": np.array([4, 5, 6])},\n ),\n ({\"a\": np.array([1, 2, 3]), \"b\": {\"c\": np.array([4, 5, 6])}}, True, {}),\n ],\n )\n def test_flatten_dict_exclude_arrays(\n input_dict: dict[str, np.ndarray | dict[str, np.ndarray]],\n exclude_arrays: bool,\n expected_output: dict[str, np.ndarray],\n ) -> None:\n logger = TensorboardLogger(SummaryWriter(\"log/logger\"))\n result = logger.prepare_dict_for_logging(input_dict, exclude_arrays=exclude_arrays)\n assert result.keys() == expected_output.keys()\n for val1, val2 in zip(result.values(), expected_output.values(), strict=True):\n assert np.all(val1 == val2)\n\n @staticmethod\n @pytest.mark.parametrize(\n \"input_dict, expected_output\",\n [\n ({\"a\": (1,), \"b\": {\"c\": \"2\", \"d\": {\"e\": 3}}}, {\"b/d/e\": 3}),\n ],\n )\n def test_flatten_dict_invalid_values_filtered_out(\n input_dict: dict[str, tuple[Literal[1]] | dict[str, str | dict[str, int]]],\n expected_output: dict[str, int],\n ) -> None:\n logger = TensorboardLogger(SummaryWriter(\"log/logger\"))\n result = logger.prepare_dict_for_logging(input_dict)\n assert result == expected_output\n" }, { "path": "test/base/test_policy.py", "content": "import gymnasium as gym\nimport numpy as np\nimport pytest\nimport torch\nfrom torch.distributions import Categorical, Distribution, Independent, Normal\n\nfrom tianshou.algorithm import PPO\nfrom tianshou.algorithm.algorithm_base import (\n RandomActionPolicy,\n episode_mc_return_to_go,\n)\nfrom tianshou.algorithm.modelfree.reinforce import ProbabilisticActorPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Batch\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.continuous import ContinuousActorProbabilistic, ContinuousCritic\nfrom tianshou.utils.net.discrete import DiscreteActor\n\nobs_shape = (5,)\n\n\ndef _to_hashable(x: np.ndarray | int) -> int | tuple[list]:\n return x if isinstance(x, int) else tuple(x.tolist())\n\n\ndef test_calculate_discounted_returns() -> None:\n assert np.all(\n episode_mc_return_to_go([1, 1, 1], 0.9) == np.array([0.9**2 + 0.9 + 1, 0.9 + 1, 1]),\n )\n assert episode_mc_return_to_go([1, 2, 3], 0.5)[0] == 1 + 0.5 * (2 + 0.5 * 3)\n\n\n@pytest.fixture(params=[\"continuous\", \"discrete\"])\ndef algorithm(request: pytest.FixtureRequest) -> PPO:\n action_type = request.param\n action_space: gym.spaces.Box | gym.spaces.Discrete\n actor: DiscreteActor | ContinuousActorProbabilistic\n if action_type == \"continuous\":\n action_space = gym.spaces.Box(low=-1, high=1, shape=(3,))\n actor = ContinuousActorProbabilistic(\n preprocess_net=Net(\n state_shape=obs_shape,\n hidden_sizes=[64, 64],\n action_shape=action_space.shape,\n ),\n action_shape=action_space.shape,\n )\n\n def dist_fn(loc_scale: tuple[torch.Tensor, torch.Tensor]) -> Distribution:\n loc, scale = loc_scale\n return Independent(Normal(loc, scale), 1)\n\n elif action_type == \"discrete\":\n action_space = gym.spaces.Discrete(3)\n actor = DiscreteActor(\n preprocess_net=Net(\n state_shape=obs_shape,\n hidden_sizes=[64, 64],\n action_shape=action_space.n,\n ),\n action_shape=action_space.n,\n )\n dist_fn = Categorical\n else:\n raise ValueError(f\"Unknown action type: {action_type}\")\n\n critic = ContinuousCritic(\n preprocess_net=Net(state_shape=obs_shape, hidden_sizes=[64, 64]),\n )\n\n optim = AdamOptimizerFactory(lr=1e-3)\n\n algorithm: PPO\n policy = ProbabilisticActorPolicy(\n actor=actor,\n dist_fn=dist_fn,\n action_space=action_space,\n action_scaling=False,\n )\n algorithm = PPO(\n policy=policy,\n critic=critic,\n optim=optim,\n )\n algorithm.eval()\n return algorithm\n\n\nclass TestPolicyBasics:\n def test_get_action(self, algorithm: PPO) -> None:\n policy = algorithm.policy\n policy.is_within_training_step = False\n sample_obs = torch.randn(obs_shape)\n policy.deterministic_eval = False\n actions = [policy.compute_action(sample_obs) for _ in range(10)]\n assert all(policy.action_space.contains(a) for a in actions)\n\n # check that the actions are different in non-deterministic mode\n assert len(set(map(_to_hashable, actions))) > 1\n\n policy.deterministic_eval = True\n actions = [policy.compute_action(sample_obs) for _ in range(10)]\n # check that the actions are the same in deterministic mode\n assert len(set(map(_to_hashable, actions))) == 1\n\n @staticmethod\n def test_random_policy_discrete_actions() -> None:\n action_space = gym.spaces.Discrete(3)\n policy = RandomActionPolicy(action_space=action_space)\n\n # forward of actor returns discrete probabilities, in compliance with the overall discrete actor\n action_probs = policy.actor(np.zeros((10, 2)))[0]\n assert np.allclose(action_probs, 1 / 3 * np.ones((10, 3)))\n\n actions = []\n for _ in range(10):\n action = policy.compute_action(np.array([0]))\n assert action_space.contains(action)\n actions.append(action)\n\n # not all actions are the same\n assert len(set(actions)) > 1\n\n # test batched forward\n action_batch = policy(Batch(obs=np.zeros((10, 2))))\n assert action_batch.act.shape == (10,)\n assert len(set(action_batch.act.tolist())) > 1\n\n @staticmethod\n def test_random_policy_continuous_actions() -> None:\n action_space = gym.spaces.Box(low=-1, high=1, shape=(3,))\n policy = RandomActionPolicy(action_space=action_space)\n\n actions = []\n for _ in range(10):\n action = policy.compute_action(np.array([0]))\n assert action_space.contains(action)\n actions.append(action)\n\n # not all actions are the same\n assert len(set(map(_to_hashable, actions))) > 1\n\n # test batched forward\n action_batch = policy(Batch(obs=np.zeros((10, 2))))\n assert action_batch.act.shape == (10, 3)\n assert len(set(map(_to_hashable, action_batch.act))) > 1\n" }, { "path": "test/base/test_returns.py", "content": "from typing import cast\n\nimport numpy as np\nimport torch\n\nfrom tianshou.algorithm import Algorithm\nfrom tianshou.data import Batch, ReplayBuffer, to_numpy\nfrom tianshou.data.types import RolloutBatchProtocol\n\n\ndef compute_episodic_return_base(batch: Batch, gamma: float) -> Batch:\n returns = np.zeros_like(batch.rew)\n last = 0\n for i in reversed(range(len(batch.rew))):\n returns[i] = batch.rew[i]\n if not batch.done[i]:\n returns[i] += last * gamma\n last = returns[i]\n batch.returns = returns\n return batch\n\n\ndef test_episodic_returns(size: int = 2560) -> None:\n fn = Algorithm.compute_episodic_return\n buf = ReplayBuffer(20)\n batch = cast(\n RolloutBatchProtocol,\n Batch(\n terminated=np.array([1, 0, 0, 1, 0, 0, 0, 1.0]),\n truncated=np.array([0, 0, 0, 0, 0, 1, 0, 0]),\n rew=np.array([0, 1, 2, 3, 4, 5, 6, 7.0]),\n info=Batch(\n {\n \"TimeLimit.truncated\": np.array(\n [False, False, False, False, False, True, False, False],\n ),\n },\n ),\n ),\n )\n for b in iter(batch):\n b.obs = b.act = 1 # type: ignore[assignment]\n buf.add(b)\n returns, _ = fn(batch, buf, buf.sample_indices(0), gamma=0.1, gae_lambda=1)\n ans = np.array([0, 1.23, 2.3, 3, 4.5, 5, 6.7, 7])\n assert np.allclose(returns, ans)\n buf.reset()\n batch = cast(\n RolloutBatchProtocol,\n Batch(\n terminated=np.array([0, 1, 0, 1, 0, 1, 0.0]),\n truncated=np.array([0, 0, 0, 0, 0, 0, 0.0]),\n rew=np.array([7, 6, 1, 2, 3, 4, 5.0]),\n ),\n )\n for b in iter(batch):\n b.obs = b.act = 1 # type: ignore[assignment]\n buf.add(b)\n returns, _ = fn(batch, buf, buf.sample_indices(0), gamma=0.1, gae_lambda=1)\n ans = np.array([7.6, 6, 1.2, 2, 3.4, 4, 5])\n assert np.allclose(returns, ans)\n buf.reset()\n batch = cast(\n RolloutBatchProtocol,\n Batch(\n terminated=np.array([0, 1, 0, 1, 0, 0, 1.0]),\n truncated=np.array([0, 0, 0, 0, 0, 0, 0]),\n rew=np.array([7, 6, 1, 2, 3, 4, 5.0]),\n ),\n )\n for b in iter(batch):\n b.obs = b.act = 1 # type: ignore[assignment]\n buf.add(b)\n returns, _ = fn(batch, buf, buf.sample_indices(0), gamma=0.1, gae_lambda=1)\n ans = np.array([7.6, 6, 1.2, 2, 3.45, 4.5, 5])\n assert np.allclose(returns, ans)\n buf.reset()\n batch = cast(\n RolloutBatchProtocol,\n Batch(\n terminated=np.array([0, 0, 0, 1.0, 0, 0, 0, 1, 0, 0, 0, 1]),\n truncated=np.array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]),\n rew=np.array([101, 102, 103.0, 200, 104, 105, 106, 201, 107, 108, 109, 202]),\n ),\n )\n for b in batch:\n b.obs = b.act = 1 # type: ignore[assignment]\n buf.add(b)\n v = np.array([2.0, 3.0, 4, -1, 5.0, 6.0, 7, -2, 8.0, 9.0, 10, -3])\n returns, _ = fn(batch, buf, buf.sample_indices(0), v, gamma=0.99, gae_lambda=0.95)\n ground_truth = np.array(\n [\n 454.8344,\n 376.1143,\n 291.298,\n 200.0,\n 464.5610,\n 383.1085,\n 295.387,\n 201.0,\n 474.2876,\n 390.1027,\n 299.476,\n 202.0,\n ],\n )\n assert np.allclose(returns, ground_truth)\n buf.reset()\n batch = cast(\n RolloutBatchProtocol,\n Batch(\n terminated=np.array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1]),\n truncated=np.array([0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0]),\n rew=np.array([101, 102, 103.0, 200, 104, 105, 106, 201, 107, 108, 109, 202]),\n info=Batch(\n {\n \"TimeLimit.truncated\": np.array(\n [\n False,\n False,\n False,\n True,\n False,\n False,\n False,\n True,\n False,\n False,\n False,\n False,\n ],\n ),\n },\n ),\n ),\n )\n for b in iter(batch):\n b.obs = b.act = 1 # type: ignore[assignment]\n buf.add(b)\n v = np.array([2.0, 3.0, 4, -1, 5.0, 6.0, 7, -2, 8.0, 9.0, 10, -3])\n returns, _ = fn(batch, buf, buf.sample_indices(0), v, gamma=0.99, gae_lambda=0.95)\n ground_truth = np.array(\n [\n 454.0109,\n 375.2386,\n 290.3669,\n 199.01,\n 462.9138,\n 381.3571,\n 293.5248,\n 199.02,\n 474.2876,\n 390.1027,\n 299.476,\n 202.0,\n ],\n )\n assert np.allclose(returns, ground_truth)\n\n\ndef target_q_fn(buffer: ReplayBuffer, indices: np.ndarray) -> torch.Tensor:\n # return the next reward\n indices = buffer.next(indices)\n return torch.tensor(-buffer.rew[indices], dtype=torch.float32)\n\n\ndef target_q_fn_multidim(buffer: ReplayBuffer, indices: np.ndarray) -> torch.Tensor:\n return target_q_fn(buffer, indices).unsqueeze(1).repeat(1, 51)\n\n\ndef compute_nstep_return_base(\n nstep: int,\n gamma: float,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n) -> np.ndarray:\n returns = np.zeros_like(indices, dtype=float)\n buf_len = len(buffer)\n for i in range(len(indices)):\n flag, rew = False, 0.0\n real_step_n = nstep\n for n in range(nstep):\n idx = (indices[i] + n) % buf_len\n rew += buffer.rew[idx] * gamma**n\n if buffer.done[idx]:\n if not (hasattr(buffer, \"info\") and buffer.info[\"TimeLimit.truncated\"][idx]):\n flag = True\n real_step_n = n + 1\n break\n if not flag:\n idx = (indices[i] + real_step_n - 1) % buf_len\n rew += to_numpy(target_q_fn(buffer, idx)) * gamma**real_step_n\n returns[i] = rew\n return returns\n\n\ndef test_nstep_returns(size: int = 10000) -> None:\n buf = ReplayBuffer(10)\n for i in range(12):\n buf.add(\n cast(\n RolloutBatchProtocol,\n Batch(\n obs=0,\n act=0,\n rew=i + 1,\n terminated=i % 4 == 3,\n truncated=False,\n ),\n ),\n )\n batch, indices = buf.sample(0)\n assert np.allclose(indices, [2, 3, 4, 5, 6, 7, 8, 9, 0, 1])\n # rew: [11, 12, 3, 4, 5, 6, 7, 8, 9, 10]\n # done: [ 0, 1, 0, 1, 0, 0, 0, 1, 0, 0]\n # test nstep = 1\n returns = to_numpy(\n Algorithm.compute_nstep_return(batch, buf, indices, target_q_fn, gamma=0.1, n_step=1)\n .pop(\"returns\")\n .reshape(-1),\n )\n assert np.allclose(returns, [2.6, 4, 4.4, 5.3, 6.2, 8, 8, 8.9, 9.8, 12])\n r_ = compute_nstep_return_base(1, 0.1, buf, indices)\n assert np.allclose(returns, r_), (r_, returns)\n returns_multidim = to_numpy(\n Algorithm.compute_nstep_return(\n batch,\n buf,\n indices,\n target_q_fn_multidim,\n gamma=0.1,\n n_step=1,\n ).pop(\"returns\"),\n )\n assert np.allclose(returns_multidim, returns[:, np.newaxis])\n # test nstep = 2\n returns = to_numpy(\n Algorithm.compute_nstep_return(batch, buf, indices, target_q_fn, gamma=0.1, n_step=2)\n .pop(\"returns\")\n .reshape(-1),\n )\n assert np.allclose(returns, [3.4, 4, 5.53, 6.62, 7.8, 8, 9.89, 10.98, 12.2, 12])\n r_ = compute_nstep_return_base(2, 0.1, buf, indices)\n assert np.allclose(returns, r_)\n returns_multidim = to_numpy(\n Algorithm.compute_nstep_return(\n batch,\n buf,\n indices,\n target_q_fn_multidim,\n gamma=0.1,\n n_step=2,\n ).pop(\"returns\"),\n )\n assert np.allclose(returns_multidim, returns[:, np.newaxis])\n # test nstep = 10\n returns = to_numpy(\n Algorithm.compute_nstep_return(batch, buf, indices, target_q_fn, gamma=0.1, n_step=10)\n .pop(\"returns\")\n .reshape(-1),\n )\n assert np.allclose(returns, [3.4, 4, 5.678, 6.78, 7.8, 8, 10.122, 11.22, 12.2, 12])\n r_ = compute_nstep_return_base(10, 0.1, buf, indices)\n assert np.allclose(returns, r_)\n returns_multidim = to_numpy(\n Algorithm.compute_nstep_return(\n batch,\n buf,\n indices,\n target_q_fn_multidim,\n gamma=0.1,\n n_step=10,\n ).pop(\"returns\"),\n )\n assert np.allclose(returns_multidim, returns[:, np.newaxis])\n\n\ndef test_nstep_returns_with_timelimit(size: int = 10000) -> None:\n buf = ReplayBuffer(10)\n for i in range(12):\n buf.add(\n cast(\n RolloutBatchProtocol,\n Batch(\n obs=0,\n act=0,\n rew=i + 1,\n terminated=i % 4 == 3 and i != 3,\n truncated=i == 3,\n info={\"TimeLimit.truncated\": i == 3},\n ),\n ),\n )\n batch, indices = buf.sample(0)\n assert np.allclose(indices, [2, 3, 4, 5, 6, 7, 8, 9, 0, 1])\n # rew: [11, 12, 3, 4, 5, 6, 7, 8, 9, 10]\n # done: [ 0, 1, 0, 1, 0, 0, 0, 1, 0, 0]\n # test nstep = 1\n returns = to_numpy(\n Algorithm.compute_nstep_return(batch, buf, indices, target_q_fn, gamma=0.1, n_step=1)\n .pop(\"returns\")\n .reshape(-1),\n )\n assert np.allclose(returns, [2.6, 3.6, 4.4, 5.3, 6.2, 8, 8, 8.9, 9.8, 12])\n r_ = compute_nstep_return_base(1, 0.1, buf, indices)\n assert np.allclose(returns, r_), (r_, returns)\n returns_multidim = to_numpy(\n Algorithm.compute_nstep_return(\n batch,\n buf,\n indices,\n target_q_fn_multidim,\n gamma=0.1,\n n_step=1,\n ).pop(\"returns\"),\n )\n assert np.allclose(returns_multidim, returns[:, np.newaxis])\n # test nstep = 2\n returns = to_numpy(\n Algorithm.compute_nstep_return(batch, buf, indices, target_q_fn, gamma=0.1, n_step=2)\n .pop(\"returns\")\n .reshape(-1),\n )\n assert np.allclose(returns, [3.36, 3.6, 5.53, 6.62, 7.8, 8, 9.89, 10.98, 12.2, 12])\n r_ = compute_nstep_return_base(2, 0.1, buf, indices)\n assert np.allclose(returns, r_)\n returns_multidim = to_numpy(\n Algorithm.compute_nstep_return(\n batch,\n buf,\n indices,\n target_q_fn_multidim,\n gamma=0.1,\n n_step=2,\n ).pop(\"returns\"),\n )\n assert np.allclose(returns_multidim, returns[:, np.newaxis])\n # test nstep = 10\n returns = to_numpy(\n Algorithm.compute_nstep_return(batch, buf, indices, target_q_fn, gamma=0.1, n_step=10)\n .pop(\"returns\")\n .reshape(-1),\n )\n assert np.allclose(returns, [3.36, 3.6, 5.678, 6.78, 7.8, 8, 10.122, 11.22, 12.2, 12])\n r_ = compute_nstep_return_base(10, 0.1, buf, indices)\n assert np.allclose(returns, r_)\n returns_multidim = to_numpy(\n Algorithm.compute_nstep_return(\n batch,\n buf,\n indices,\n target_q_fn_multidim,\n gamma=0.1,\n n_step=10,\n ).pop(\"returns\"),\n )\n assert np.allclose(returns_multidim, returns[:, np.newaxis])\n" }, { "path": "test/base/test_stats.py", "content": "from typing import cast\n\nimport numpy as np\nimport pytest\nimport torch\nfrom torch.distributions import Categorical, Normal\n\nfrom tianshou.algorithm.algorithm_base import TrainingStats, TrainingStatsWrapper\nfrom tianshou.data import Batch, CollectStats\nfrom tianshou.data.collector import CollectStepBatchProtocol, get_stddev_from_dist\n\n\nclass DummyTrainingStatsWrapper(TrainingStatsWrapper):\n def __init__(self, wrapped_stats: TrainingStats, *, dummy_field: int) -> None:\n self.dummy_field = dummy_field\n super().__init__(wrapped_stats)\n\n\nclass TestStats:\n @staticmethod\n def test_training_stats_wrapper() -> None:\n train_stats = TrainingStats(train_time=1.0)\n\n setattr(train_stats, \"loss_field\", 12) # noqa: B010\n\n wrapped_train_stats = DummyTrainingStatsWrapper(train_stats, dummy_field=42)\n\n # basic readout\n assert wrapped_train_stats.train_time == 1.0\n assert wrapped_train_stats.loss_field == 12\n\n # mutation of TrainingStats fields\n wrapped_train_stats.train_time = 2.0\n wrapped_train_stats.smoothed_loss[\"foo\"] = 50\n assert wrapped_train_stats.train_time == 2.0\n assert wrapped_train_stats.smoothed_loss[\"foo\"] == 50\n\n # loss stats dict\n assert wrapped_train_stats.get_loss_stats_dict() == {\n \"loss_field\": 12,\n \"dummy_field\": 42,\n }\n\n # new fields can't be added\n with pytest.raises(AttributeError):\n wrapped_train_stats.new_loss_field = 90\n\n # existing fields, wrapped and not-wrapped, can be mutated\n wrapped_train_stats.loss_field = 13\n wrapped_train_stats.dummy_field = 43\n assert hasattr(\n wrapped_train_stats.wrapped_stats,\n \"loss_field\",\n ), \"Attribute `loss_field` not found in `wrapped_train_stats.wrapped_stats`.\"\n assert hasattr(\n wrapped_train_stats,\n \"loss_field\",\n ), \"Attribute `loss_field` not found in `wrapped_train_stats`.\"\n assert wrapped_train_stats.wrapped_stats.loss_field == wrapped_train_stats.loss_field == 13\n\n @staticmethod\n @pytest.mark.parametrize(\n \"act,dist\",\n (\n (np.array(1), Categorical(probs=torch.tensor([0.5, 0.5]))),\n (np.array([1, 2, 3]), Normal(torch.zeros(3), torch.ones(3))),\n ),\n )\n def test_collect_stats_update_at_step(\n act: np.ndarray,\n dist: torch.distributions.Distribution,\n ) -> None:\n step_batch = cast(\n CollectStepBatchProtocol,\n Batch(\n info={},\n obs=np.array([1, 2, 3]),\n obs_next=np.array([4, 5, 6]),\n act=act,\n rew=np.array(1.0),\n done=np.array(False),\n terminated=np.array(False),\n dist=dist,\n ).to_at_least_2d(),\n )\n stats = CollectStats()\n for _ in range(10):\n stats.update_at_step_batch(step_batch)\n stats.refresh_all_sequence_stats()\n assert stats.n_collected_steps == 10\n assert stats.pred_dist_std_array is not None\n assert np.allclose(stats.pred_dist_std_array, get_stddev_from_dist(dist))\n assert stats.pred_dist_std_array_stat is not None\n assert stats.pred_dist_std_array_stat[0].mean == get_stddev_from_dist(dist)[0].item()\n" }, { "path": "test/base/test_utils.py", "content": "from typing import cast\n\nimport numpy as np\nimport pytest\nimport torch\nimport torch.distributions as dist\nfrom gymnasium import spaces\nfrom torch import nn\n\nfrom tianshou.exploration import GaussianNoise, OUNoise\nfrom tianshou.utils import MovAvg, RunningMeanStd\nfrom tianshou.utils.net.common import MLP, Net\nfrom tianshou.utils.net.continuous import RecurrentActorProb, RecurrentCritic\nfrom tianshou.utils.torch_utils import create_uniform_action_dist, torch_train_mode\n\n\ndef test_noise() -> None:\n noise = GaussianNoise()\n size = (3, 4, 5)\n assert np.allclose(noise(size).shape, size)\n noise = OUNoise()\n noise.reset()\n assert np.allclose(noise(size).shape, size)\n\n\ndef test_moving_average() -> None:\n stat = MovAvg(10)\n assert np.allclose(stat.get(), 0)\n assert np.allclose(stat.mean(), 0)\n assert np.allclose(stat.std() ** 2, 0)\n stat.add(torch.tensor([1]))\n stat.add(np.array([2]))\n stat.add([3, 4])\n stat.add(5.0)\n assert np.allclose(stat.get(), 3)\n assert np.allclose(stat.mean(), 3)\n assert np.allclose(stat.std() ** 2, 2)\n\n\ndef test_rms() -> None:\n rms = RunningMeanStd()\n assert np.allclose(rms.mean, 0)\n assert np.allclose(rms.var, 1)\n rms.update(np.array([[[1, 2], [3, 5]]]))\n rms.update(np.array([[[1, 2], [3, 4]], [[1, 2], [0, 0]]]))\n assert np.allclose(rms.mean, np.array([[1, 2], [2, 3]]), atol=1e-3)\n assert np.allclose(rms.var, np.array([[0, 0], [2, 14 / 3.0]]), atol=1e-3)\n\n\ndef test_net() -> None:\n # here test the networks that does not appear in the other script\n bsz = 64\n # MLP\n data = torch.rand([bsz, 3])\n mlp = MLP(input_dim=3, output_dim=6, hidden_sizes=[128])\n assert list(mlp(data).shape) == [bsz, 6]\n # output == 0 and len(hidden_sizes) == 0 means identity model\n mlp = MLP(input_dim=6, output_dim=0)\n assert data.shape == mlp(data).shape\n # common net\n state_shape = (10, 2)\n action_shape = (5,)\n data = torch.rand([bsz, *state_shape])\n expect_output_shape = [bsz, *action_shape]\n net = Net(\n state_shape=state_shape,\n action_shape=action_shape,\n hidden_sizes=[128, 128],\n norm_layer=torch.nn.LayerNorm,\n activation=None,\n )\n assert list(net(data)[0].shape) == expect_output_shape\n assert str(net).count(\"LayerNorm\") == 2\n assert str(net).count(\"ReLU\") == 0\n Q_param = V_param = {\"hidden_sizes\": [128, 128]}\n net = Net(\n state_shape=state_shape,\n action_shape=action_shape,\n hidden_sizes=[128, 128],\n dueling_param=(Q_param, V_param),\n )\n assert list(net(data)[0].shape) == expect_output_shape\n # concat\n net = Net(\n state_shape=state_shape,\n action_shape=action_shape,\n hidden_sizes=[128],\n concat=True,\n )\n data = torch.rand([bsz, int(np.prod(state_shape)) + int(np.prod(action_shape))])\n expect_output_shape = [bsz, 128]\n assert list(net(data)[0].shape) == expect_output_shape\n net = Net(\n state_shape=state_shape,\n action_shape=action_shape,\n hidden_sizes=[128],\n concat=True,\n dueling_param=(Q_param, V_param),\n )\n assert list(net(data)[0].shape) == expect_output_shape\n # recurrent actor/critic\n data = torch.rand([bsz, *state_shape]).flatten(1)\n expect_output_shape = [bsz, *action_shape]\n net = RecurrentActorProb(layer_num=3, state_shape=state_shape, action_shape=action_shape)\n mu, sigma = net(data)[0]\n assert mu.shape == sigma.shape\n assert list(mu.shape) == [bsz, 5]\n net = RecurrentCritic(layer_num=3, state_shape=state_shape, action_shape=action_shape)\n data = torch.rand([bsz, 8, int(np.prod(state_shape))])\n act = torch.rand(expect_output_shape)\n assert list(net(data, act).shape) == [bsz, 1]\n\n\ndef test_in_eval_mode() -> None:\n module = nn.Linear(3, 4)\n module.train()\n with torch_train_mode(module, False):\n assert not module.training\n assert module.training\n\n\ndef test_in_train_mode() -> None:\n module = nn.Linear(3, 4)\n module.eval()\n with torch_train_mode(module):\n assert module.training\n assert not module.training\n\n\nclass TestCreateActionDistribution:\n @classmethod\n def setup_class(cls) -> None:\n # Set random seeds for reproducibility\n torch.manual_seed(0)\n np.random.seed(0)\n\n @pytest.mark.parametrize(\n \"action_space, batch_size\",\n [\n (spaces.Box(low=-1.0, high=1.0, shape=(3,)), 1),\n (spaces.Box(low=-1.0, high=1.0, shape=(3,)), 5),\n (spaces.Discrete(5), 1),\n (spaces.Discrete(5), 5),\n ],\n )\n def test_distribution_properties(\n self,\n action_space: spaces.Box | spaces.Discrete,\n batch_size: int,\n ) -> None:\n distribution = create_uniform_action_dist(action_space, batch_size)\n\n # Correct distribution type\n if isinstance(action_space, spaces.Box):\n assert isinstance(distribution, dist.Uniform)\n elif isinstance(action_space, spaces.Discrete):\n assert isinstance(distribution, dist.Categorical)\n\n # Samples are within correct range\n samples = distribution.sample()\n if isinstance(action_space, spaces.Box):\n low = torch.tensor(action_space.low, dtype=torch.float32)\n high = torch.tensor(action_space.high, dtype=torch.float32)\n assert torch.all(samples >= low)\n assert torch.all(samples <= high)\n elif isinstance(action_space, spaces.Discrete):\n assert torch.all(samples >= 0)\n assert torch.all(samples < action_space.n)\n\n @pytest.mark.parametrize(\n \"action_space, batch_size\",\n [\n (spaces.Box(low=-1.0, high=1.0, shape=(3,)), 1),\n (spaces.Box(low=-1.0, high=1.0, shape=(3,)), 5),\n (spaces.Discrete(5), 1),\n (spaces.Discrete(5), 5),\n ],\n )\n def test_distribution_uniformity(\n self,\n action_space: spaces.Box | spaces.Discrete,\n batch_size: int,\n ) -> None:\n distribution = create_uniform_action_dist(action_space, batch_size)\n\n # Test 7: Uniform distribution (statistical test)\n large_sample = distribution.sample(torch.Size((10000,)))\n if isinstance(action_space, spaces.Box):\n # For Box, check if mean is close to 0 and std is close to 1/sqrt(3)\n assert torch.allclose(large_sample.mean(), torch.tensor(0.0), atol=0.1)\n assert torch.allclose(large_sample.std(), torch.tensor(1 / 3**0.5), atol=0.1)\n elif isinstance(action_space, spaces.Discrete):\n # For Discrete, check if all actions are roughly equally likely\n n_actions = cast(int, action_space.n)\n counts = torch.bincount(large_sample.flatten(), minlength=n_actions).float()\n expected_count = 10000 * batch_size / n_actions\n assert torch.allclose(counts, torch.tensor(expected_count).float(), rtol=0.1)\n\n def test_unsupported_space(self) -> None:\n # Test 6: Raises ValueError for unsupported space\n with pytest.raises(ValueError):\n create_uniform_action_dist(spaces.MultiBinary(5)) # type: ignore\n\n @pytest.mark.parametrize(\n \"space, batch_size, expected_shape, distribution_type\",\n [\n (spaces.Box(low=-1.0, high=1.0, shape=(3,)), 1, (1, 3), dist.Uniform),\n (spaces.Box(low=-1.0, high=1.0, shape=(3,)), 5, (5, 3), dist.Uniform),\n (spaces.Box(low=-1.0, high=1.0, shape=(3,)), 10, (10, 3), dist.Uniform),\n (spaces.Discrete(5), 1, (1,), dist.Categorical),\n (spaces.Discrete(5), 5, (5,), dist.Categorical),\n (spaces.Discrete(5), 10, (10,), dist.Categorical),\n ],\n )\n def test_batch_sizes(\n self,\n space: spaces.Box | spaces.Discrete,\n batch_size: int,\n expected_shape: tuple[int, ...],\n distribution_type: type[dist.Distribution],\n ) -> None:\n distribution = create_uniform_action_dist(space, batch_size)\n\n # Check distribution type\n assert isinstance(distribution, distribution_type)\n\n # Check sample shape\n samples = distribution.sample()\n assert samples.shape == expected_shape\n\n # Check internal distribution shapes\n if isinstance(space, spaces.Box):\n distribution = cast(dist.Uniform, distribution)\n assert distribution.low.shape == expected_shape\n assert distribution.high.shape == expected_shape\n elif isinstance(space, spaces.Discrete):\n distribution = cast(dist.Categorical, distribution)\n assert distribution.probs.shape == (batch_size, space.n)\n" }, { "path": "test/continuous/__init__.py", "content": "" }, { "path": "test/continuous/test_ddpg.py", "content": "import argparse\nimport os\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom tianshou.algorithm import DDPG\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.ddpg import ContinuousDeterministicPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.exploration import GaussianNoise\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.continuous import ContinuousActorDeterministic, ContinuousCritic\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"Pendulum-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=0)\n parser.add_argument(\"--buffer_size\", type=int, default=20000)\n parser.add_argument(\"--actor_lr\", type=float, default=1e-4)\n parser.add_argument(\"--critic_lr\", type=float, default=1e-3)\n parser.add_argument(\"--gamma\", type=float, default=0.99)\n parser.add_argument(\"--tau\", type=float, default=0.005)\n parser.add_argument(\"--exploration_noise\", type=float, default=0.1)\n parser.add_argument(\"--epoch\", type=int, default=5)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=20000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=8)\n parser.add_argument(\"--update_per_step\", type=float, default=0.125)\n parser.add_argument(\"--batch_size\", type=int, default=128)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[128, 128])\n parser.add_argument(\"--num_training_envs\", type=int, default=8)\n parser.add_argument(\"--num_test_envs\", type=int, default=100)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\"--return_scaling\", action=\"store_true\", default=False)\n parser.add_argument(\"--n_step\", type=int, default=3)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n return parser.parse_known_args()[0]\n\n\ndef test_ddpg(args: argparse.Namespace = get_args(), enable_assertions: bool = True) -> None:\n env = gym.make(args.task)\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n args.max_action = space_info.action_info.max_action\n if args.reward_threshold is None:\n default_reward_threshold = {\"Pendulum-v0\": -250, \"Pendulum-v1\": -250}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n\n # model\n net = Net(state_shape=args.state_shape, hidden_sizes=args.hidden_sizes)\n actor = ContinuousActorDeterministic(\n preprocess_net=net, action_shape=args.action_shape, max_action=args.max_action\n ).to(\n args.device,\n )\n net = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n concat=True,\n )\n critic = ContinuousCritic(preprocess_net=net).to(args.device)\n critic_optim = AdamOptimizerFactory(lr=args.critic_lr)\n policy = ContinuousDeterministicPolicy(\n actor=actor,\n exploration_noise=GaussianNoise(sigma=args.exploration_noise),\n action_space=env.action_space,\n )\n policy_optim = AdamOptimizerFactory(lr=args.actor_lr)\n algorithm: DDPG = DDPG(\n policy=policy,\n policy_optim=policy_optim,\n critic=critic,\n critic_optim=critic_optim,\n tau=args.tau,\n gamma=args.gamma,\n n_step_return_horizon=args.n_step,\n )\n\n # collector\n training_collector = Collector[CollectStats](\n algorithm,\n training_envs,\n VectorReplayBuffer(args.buffer_size, len(training_envs)),\n exploration_noise=True,\n )\n test_collector = Collector[CollectStats](algorithm, test_envs)\n # log\n log_path = os.path.join(args.logdir, args.task, \"ddpg\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n # trainer\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n update_step_num_gradient_steps_per_sample=args.update_per_step,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=True,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_ddpg_determinism() -> None:\n main_fn = lambda args: test_ddpg(args, enable_assertions=False)\n AlgorithmDeterminismTest(\"continuous_ddpg\", main_fn, get_args()).run()\n" }, { "path": "test/continuous/test_npg.py", "content": "import argparse\nimport os\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch import nn\nfrom torch.distributions import Distribution, Independent, Normal\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom tianshou.algorithm import NPG\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.reinforce import ProbabilisticActorPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OnPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.continuous import ContinuousActorProbabilistic, ContinuousCritic\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"Pendulum-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=1)\n parser.add_argument(\"--buffer_size\", type=int, default=50000)\n parser.add_argument(\"--lr\", type=float, default=1e-3)\n parser.add_argument(\"--gamma\", type=float, default=0.95)\n parser.add_argument(\"--epoch\", type=int, default=5)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=50000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=2048)\n parser.add_argument(\n \"--update_step_num_repetitions\", type=int, default=2\n ) # theoretically it should be 1\n parser.add_argument(\"--batch_size\", type=int, default=99999)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[64, 64])\n parser.add_argument(\"--num_training_envs\", type=int, default=16)\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n # npg special\n parser.add_argument(\"--gae_lambda\", type=float, default=0.95)\n parser.add_argument(\"--return_scaling\", type=int, default=1)\n parser.add_argument(\"--advantage_normalization\", type=int, default=1)\n parser.add_argument(\"--optim_critic_iters\", type=int, default=5)\n parser.add_argument(\"--trust_region_size\", type=float, default=0.5)\n return parser.parse_known_args()[0]\n\n\ndef test_npg(args: argparse.Namespace = get_args(), enable_assertions: bool = True) -> None:\n env = gym.make(args.task)\n\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n args.max_action = space_info.action_info.max_action\n\n if args.reward_threshold is None:\n default_reward_threshold = {\"Pendulum-v0\": -250, \"Pendulum-v1\": -250}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n\n # model\n net = Net(\n state_shape=args.state_shape,\n hidden_sizes=args.hidden_sizes,\n activation=nn.Tanh,\n )\n actor = ContinuousActorProbabilistic(\n preprocess_net=net, action_shape=args.action_shape, unbounded=True\n ).to(args.device)\n critic = ContinuousCritic(\n preprocess_net=Net(\n state_shape=args.state_shape,\n hidden_sizes=args.hidden_sizes,\n activation=nn.Tanh,\n ),\n ).to(args.device)\n\n # orthogonal initialization\n for m in list(actor.modules()) + list(critic.modules()):\n if isinstance(m, torch.nn.Linear):\n torch.nn.init.orthogonal_(m.weight)\n torch.nn.init.zeros_(m.bias)\n\n # replace DiagGuassian with Independent(Normal) which is equivalent\n # pass *logits to be consistent with policy.forward\n def dist(loc_scale: tuple[torch.Tensor, torch.Tensor]) -> Distribution:\n loc, scale = loc_scale\n return Independent(Normal(loc, scale), 1)\n\n policy = ProbabilisticActorPolicy(\n actor=actor,\n dist_fn=dist,\n action_space=env.action_space,\n deterministic_eval=True,\n )\n algorithm: NPG = NPG(\n policy=policy,\n critic=critic,\n optim=AdamOptimizerFactory(lr=args.lr),\n gamma=args.gamma,\n return_scaling=args.return_scaling,\n advantage_normalization=args.advantage_normalization,\n gae_lambda=args.gae_lambda,\n optim_critic_iters=args.optim_critic_iters,\n trust_region_size=args.trust_region_size,\n )\n # collector\n training_collector = Collector[CollectStats](\n algorithm,\n training_envs,\n VectorReplayBuffer(args.buffer_size, len(training_envs)),\n )\n test_collector = Collector[CollectStats](algorithm, test_envs)\n # log\n log_path = os.path.join(args.logdir, args.task, \"npg\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n # trainer\n result = algorithm.run_training(\n OnPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n update_step_num_repetitions=args.update_step_num_repetitions,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=True,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_npg_determinism() -> None:\n main_fn = lambda args: test_npg(args, enable_assertions=False)\n AlgorithmDeterminismTest(\"continuous_npg\", main_fn, get_args()).run()\n" }, { "path": "test/continuous/test_ppo.py", "content": "import argparse\nimport os\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.distributions import Distribution, Independent, Normal\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom tianshou.algorithm import PPO\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.reinforce import ProbabilisticActorPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OnPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import ActorCritic, Net\nfrom tianshou.utils.net.continuous import ContinuousActorProbabilistic, ContinuousCritic\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"Pendulum-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=1)\n parser.add_argument(\"--buffer_size\", type=int, default=20000)\n parser.add_argument(\"--lr\", type=float, default=1e-3)\n parser.add_argument(\"--gamma\", type=float, default=0.95)\n parser.add_argument(\"--epoch\", type=int, default=5)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=150000)\n parser.add_argument(\"--collection_step_num_episodes\", type=int, default=16)\n parser.add_argument(\"--update_step_num_repetitions\", type=int, default=2)\n parser.add_argument(\"--batch_size\", type=int, default=128)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[64, 64])\n parser.add_argument(\"--num_training_envs\", type=int, default=16)\n parser.add_argument(\"--num_test_envs\", type=int, default=100)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n # ppo special\n parser.add_argument(\"--vf_coef\", type=float, default=0.25)\n parser.add_argument(\"--ent_coef\", type=float, default=0.0)\n parser.add_argument(\"--eps_clip\", type=float, default=0.2)\n parser.add_argument(\"--max_grad_norm\", type=float, default=0.5)\n parser.add_argument(\"--gae_lambda\", type=float, default=0.95)\n parser.add_argument(\"--return_scaling\", type=int, default=1)\n parser.add_argument(\"--dual_clip\", type=float, default=None)\n parser.add_argument(\"--value_clip\", type=int, default=1)\n parser.add_argument(\"--advantage_normalization\", type=int, default=1)\n parser.add_argument(\"--recompute_adv\", type=int, default=0)\n parser.add_argument(\"--resume\", action=\"store_true\")\n parser.add_argument(\"--save_interval\", type=int, default=4)\n return parser.parse_known_args()[0]\n\n\ndef test_ppo(args: argparse.Namespace = get_args(), enable_assertions: bool = True) -> None:\n env = gym.make(args.task)\n\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n args.max_action = space_info.action_info.max_action\n\n if args.reward_threshold is None:\n default_reward_threshold = {\"Pendulum-v0\": -250, \"Pendulum-v1\": -250}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n\n # model\n net = Net(state_shape=args.state_shape, hidden_sizes=args.hidden_sizes)\n actor = ContinuousActorProbabilistic(\n preprocess_net=net, action_shape=args.action_shape, unbounded=True\n ).to(args.device)\n critic = ContinuousCritic(\n preprocess_net=Net(state_shape=args.state_shape, hidden_sizes=args.hidden_sizes),\n ).to(args.device)\n actor_critic = ActorCritic(actor, critic)\n # orthogonal initialization\n for m in actor_critic.modules():\n if isinstance(m, torch.nn.Linear):\n torch.nn.init.orthogonal_(m.weight)\n torch.nn.init.zeros_(m.bias)\n optim = AdamOptimizerFactory(lr=args.lr)\n\n # replace DiagGuassian with Independent(Normal) which is equivalent\n # pass *logits to be consistent with policy.forward\n def dist(loc_scale: tuple[torch.Tensor, torch.Tensor]) -> Distribution:\n loc, scale = loc_scale\n return Independent(Normal(loc, scale), 1)\n\n policy = ProbabilisticActorPolicy(\n actor=actor,\n dist_fn=dist,\n action_space=env.action_space,\n )\n algorithm: PPO = PPO(\n policy=policy,\n critic=critic,\n optim=optim,\n gamma=args.gamma,\n max_grad_norm=args.max_grad_norm,\n eps_clip=args.eps_clip,\n vf_coef=args.vf_coef,\n ent_coef=args.ent_coef,\n return_scaling=args.return_scaling,\n advantage_normalization=args.advantage_normalization,\n recompute_advantage=args.recompute_adv,\n dual_clip=args.dual_clip,\n value_clip=args.value_clip,\n gae_lambda=args.gae_lambda,\n )\n # collector\n training_collector = Collector[CollectStats](\n algorithm,\n training_envs,\n VectorReplayBuffer(args.buffer_size, len(training_envs)),\n )\n test_collector = Collector[CollectStats](algorithm, test_envs)\n # log\n log_path = os.path.join(args.logdir, args.task, \"ppo\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer, save_interval=args.save_interval)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n def save_checkpoint_fn(epoch: int, env_step: int, gradient_step: int) -> str:\n # see also: https://pytorch.org/tutorials/beginner/saving_loading_models.html\n ckpt_path = os.path.join(log_path, \"checkpoint.pth\")\n # Example: saving by epoch num\n # ckpt_path = os.path.join(log_path, f\"checkpoint_{epoch}.pth\")\n torch.save(\n algorithm.state_dict(),\n ckpt_path,\n )\n return ckpt_path\n\n if args.resume:\n # load from existing checkpoint\n print(f\"Loading agent under {log_path}\")\n ckpt_path = os.path.join(log_path, \"checkpoint.pth\")\n if os.path.exists(ckpt_path):\n checkpoint = torch.load(ckpt_path, map_location=args.device)\n algorithm.load_state_dict(checkpoint)\n print(\"Successfully restore policy and optim.\")\n else:\n print(\"Fail to restore policy and optim.\")\n\n # train\n result = algorithm.run_training(\n OnPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n update_step_num_repetitions=args.update_step_num_repetitions,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n collection_step_num_episodes=args.collection_step_num_episodes,\n collection_step_num_env_steps=None,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n resume_from_log=args.resume,\n save_checkpoint_fn=save_checkpoint_fn,\n test_in_training=True,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_ppo_resume(args: argparse.Namespace = get_args()) -> None:\n args.resume = True\n test_ppo(args)\n\n\ndef test_ppo_determinism() -> None:\n main_fn = lambda args: test_ppo(args, enable_assertions=False)\n AlgorithmDeterminismTest(\"continuous_ppo\", main_fn, get_args()).run()\n" }, { "path": "test/continuous/test_redq.py", "content": "import argparse\nimport os\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nimport torch.nn as nn\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom tianshou.algorithm import REDQ\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.redq import REDQPolicy\nfrom tianshou.algorithm.modelfree.sac import AutoAlpha\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import EnsembleLinear, Net\nfrom tianshou.utils.net.continuous import ContinuousActorProbabilistic, ContinuousCritic\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"Pendulum-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=0)\n parser.add_argument(\"--buffer_size\", type=int, default=20000)\n parser.add_argument(\"--ensemble_size\", type=int, default=4)\n parser.add_argument(\"--subset_size\", type=int, default=2)\n parser.add_argument(\"--actor_lr\", type=float, default=1e-4)\n parser.add_argument(\"--critic_lr\", type=float, default=1e-3)\n parser.add_argument(\"--gamma\", type=float, default=0.99)\n parser.add_argument(\"--tau\", type=float, default=0.005)\n parser.add_argument(\"--alpha\", type=float, default=0.2)\n parser.add_argument(\"--auto_alpha\", action=\"store_true\", default=False)\n parser.add_argument(\"--alpha_lr\", type=float, default=3e-4)\n parser.add_argument(\"--start_timesteps\", type=int, default=1000)\n parser.add_argument(\"--epoch\", type=int, default=5)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=5000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=1)\n parser.add_argument(\"--update_per_step\", type=int, default=3)\n parser.add_argument(\"--n_step\", type=int, default=1)\n parser.add_argument(\"--batch_size\", type=int, default=64)\n parser.add_argument(\"--target_mode\", type=str, choices=(\"min\", \"mean\"), default=\"min\")\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[64, 64])\n parser.add_argument(\"--num_training_envs\", type=int, default=8)\n parser.add_argument(\"--num_test_envs\", type=int, default=100)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n return parser.parse_known_args()[0]\n\n\ndef test_redq(args: argparse.Namespace = get_args(), enable_assertions: bool = True) -> None:\n env = gym.make(args.task)\n assert isinstance(env.action_space, gym.spaces.Box)\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n if args.reward_threshold is None:\n default_reward_threshold = {\"Pendulum-v0\": -250, \"Pendulum-v1\": -250}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n # you can also use tianshou.env.SubprocVectorEnv\n # training_envs = gym.make(args.task)\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n # test_envs = gym.make(args.task)\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n # model\n net = Net(state_shape=args.state_shape, hidden_sizes=args.hidden_sizes)\n actor = ContinuousActorProbabilistic(\n preprocess_net=net,\n action_shape=args.action_shape,\n unbounded=True,\n conditioned_sigma=True,\n ).to(args.device)\n actor_optim = AdamOptimizerFactory(lr=args.actor_lr)\n\n def linear(x: int, y: int) -> nn.Module:\n return EnsembleLinear(args.ensemble_size, x, y)\n\n net_c = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n concat=True,\n linear_layer=linear,\n )\n critic = ContinuousCritic(preprocess_net=net_c, linear_layer=linear, flatten_input=False).to(\n args.device,\n )\n critic_optim = AdamOptimizerFactory(lr=args.critic_lr)\n\n action_dim = space_info.action_info.action_dim\n if args.auto_alpha:\n target_entropy = -action_dim\n log_alpha = 0.0\n alpha_optim = AdamOptimizerFactory(lr=args.alpha_lr)\n args.alpha = AutoAlpha(target_entropy, log_alpha, alpha_optim).to(args.device)\n\n policy = REDQPolicy(\n actor=actor,\n action_space=env.action_space,\n )\n algorithm: REDQ = REDQ(\n policy=policy,\n policy_optim=actor_optim,\n critic=critic,\n critic_optim=critic_optim,\n ensemble_size=args.ensemble_size,\n subset_size=args.subset_size,\n tau=args.tau,\n gamma=args.gamma,\n alpha=args.alpha,\n n_step_return_horizon=args.n_step,\n actor_delay=args.update_per_step,\n target_mode=args.target_mode,\n )\n # collector\n training_collector = Collector[CollectStats](\n algorithm,\n training_envs,\n VectorReplayBuffer(args.buffer_size, len(training_envs)),\n exploration_noise=True,\n )\n test_collector = Collector[CollectStats](algorithm, test_envs)\n training_collector.reset()\n training_collector.collect(n_step=args.start_timesteps, random=True)\n # log\n log_path = os.path.join(args.logdir, args.task, \"redq\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n update_step_num_gradient_steps_per_sample=args.update_per_step,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=True,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_redq_determinism() -> None:\n main_fn = lambda args: test_redq(args, enable_assertions=False)\n ignored_messages = [\n \"Params[actor_old]\",\n ] # actor_old only present in v1 (due to flawed inheritance)\n AlgorithmDeterminismTest(\n \"continuous_redq\",\n main_fn,\n get_args(),\n ignored_messages=ignored_messages,\n ).run()\n" }, { "path": "test/continuous/test_sac_with_il.py", "content": "import argparse\nimport os\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom tianshou.algorithm import SAC, OffPolicyImitationLearning\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.imitation.imitation_base import ImitationPolicy\nfrom tianshou.algorithm.modelfree.sac import AutoAlpha, SACPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.continuous import (\n ContinuousActorDeterministic,\n ContinuousActorProbabilistic,\n ContinuousCritic,\n)\nfrom tianshou.utils.space_info import SpaceInfo\n\ntry:\n import envpool\nexcept ImportError:\n envpool = None\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"Pendulum-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=1)\n parser.add_argument(\"--buffer_size\", type=int, default=20000)\n parser.add_argument(\"--actor_lr\", type=float, default=1e-3)\n parser.add_argument(\"--critic_lr\", type=float, default=1e-3)\n parser.add_argument(\"--il_lr\", type=float, default=1e-3)\n parser.add_argument(\"--gamma\", type=float, default=0.99)\n parser.add_argument(\"--tau\", type=float, default=0.005)\n parser.add_argument(\"--alpha\", type=float, default=0.2)\n parser.add_argument(\"--auto_alpha\", type=int, default=1)\n parser.add_argument(\"--alpha_lr\", type=float, default=3e-4)\n parser.add_argument(\"--epoch\", type=int, default=10)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=24000)\n parser.add_argument(\"--il_step_per_epoch\", type=int, default=500)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=10)\n parser.add_argument(\"--update_per_step\", type=float, default=0.1)\n parser.add_argument(\"--batch_size\", type=int, default=128)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[128, 128])\n parser.add_argument(\"--imitation_hidden_sizes\", type=int, nargs=\"*\", default=[128, 128])\n parser.add_argument(\"--num_training_envs\", type=int, default=10)\n parser.add_argument(\"--num_test_envs\", type=int, default=100)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\"--n_step\", type=int, default=3)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n return parser.parse_known_args()[0]\n\n\ndef test_sac_with_il(\n args: argparse.Namespace = get_args(),\n enable_assertions: bool = True,\n skip_il: bool = False,\n) -> None:\n # if you want to use python vector env, please refer to other test scripts\n # training_envs = env = envpool.make_gymnasium(args.task, num_envs=args.num_training_envs, seed=args.seed)\n # test_envs = envpool.make_gymnasium(args.task, num_envs=args.num_test_envs, seed=args.seed)\n env = gym.make(args.task)\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n args.max_action = space_info.action_info.max_action\n if args.reward_threshold is None:\n default_reward_threshold = {\"Pendulum-v0\": -250, \"Pendulum-v1\": -250}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed + args.num_training_envs)\n\n # model\n net = Net(state_shape=args.state_shape, hidden_sizes=args.hidden_sizes)\n actor = ContinuousActorProbabilistic(\n preprocess_net=net, action_shape=args.action_shape, unbounded=True\n ).to(args.device)\n actor_optim = AdamOptimizerFactory(lr=args.actor_lr)\n net_c1 = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n concat=True,\n )\n critic1 = ContinuousCritic(preprocess_net=net_c1).to(args.device)\n critic1_optim = AdamOptimizerFactory(lr=args.critic_lr)\n net_c2 = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n concat=True,\n )\n critic2 = ContinuousCritic(preprocess_net=net_c2).to(args.device)\n critic2_optim = AdamOptimizerFactory(lr=args.critic_lr)\n action_dim = space_info.action_info.action_dim\n if args.auto_alpha:\n target_entropy = -action_dim\n log_alpha = 0.0\n alpha_optim = AdamOptimizerFactory(lr=args.alpha_lr)\n args.alpha = AutoAlpha(target_entropy, log_alpha, alpha_optim)\n policy = SACPolicy(\n actor=actor,\n action_space=env.action_space,\n )\n algorithm: SAC = SAC(\n policy=policy,\n policy_optim=actor_optim,\n critic=critic1,\n critic_optim=critic1_optim,\n critic2=critic2,\n critic2_optim=critic2_optim,\n tau=args.tau,\n gamma=args.gamma,\n alpha=args.alpha,\n n_step_return_horizon=args.n_step,\n )\n # collector\n training_collector = Collector[CollectStats](\n algorithm,\n training_envs,\n VectorReplayBuffer(args.buffer_size, len(training_envs)),\n exploration_noise=True,\n )\n test_collector = Collector[CollectStats](algorithm, test_envs)\n # training_collector.collect(n_step=args.buffer_size)\n # log\n log_path = os.path.join(args.logdir, args.task, \"sac\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n update_step_num_gradient_steps_per_sample=args.update_per_step,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=True,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n if skip_il:\n return\n\n # here we define an imitation collector with a trivial policy\n if args.task.startswith(\"Pendulum\"):\n args.reward_threshold -= 50 # lower the goal\n il_net = Net(\n state_shape=args.state_shape,\n hidden_sizes=args.imitation_hidden_sizes,\n )\n il_actor = ContinuousActorDeterministic(\n preprocess_net=il_net,\n action_shape=args.action_shape,\n max_action=args.max_action,\n ).to(args.device)\n optim = AdamOptimizerFactory(lr=args.il_lr)\n il_policy = ImitationPolicy(\n actor=il_actor,\n action_space=env.action_space,\n action_scaling=True,\n action_bound_method=\"clip\",\n )\n il_algorithm: OffPolicyImitationLearning = OffPolicyImitationLearning(\n policy=il_policy,\n optim=optim,\n )\n il_test_env = gym.make(args.task)\n il_test_env.reset(seed=args.seed + args.num_training_envs + args.num_test_envs)\n il_test_collector = Collector[CollectStats](\n il_algorithm,\n # envpool.make_gymnasium(args.task, num_envs=args.num_test_envs, seed=args.seed),\n il_test_env,\n )\n training_collector.reset()\n result = il_algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=il_test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=True,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_sac_determinism() -> None:\n main_fn = lambda args: test_sac_with_il(args, enable_assertions=False, skip_il=True)\n AlgorithmDeterminismTest(\"continuous_sac\", main_fn, get_args()).run()\n" }, { "path": "test/continuous/test_td3.py", "content": "import argparse\nimport os\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom tianshou.algorithm import TD3\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.ddpg import ContinuousDeterministicPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.exploration import GaussianNoise\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.continuous import ContinuousActorDeterministic, ContinuousCritic\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"Pendulum-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=1)\n parser.add_argument(\"--buffer_size\", type=int, default=20000)\n parser.add_argument(\"--actor_lr\", type=float, default=1e-4)\n parser.add_argument(\"--critic_lr\", type=float, default=1e-3)\n parser.add_argument(\"--gamma\", type=float, default=0.99)\n parser.add_argument(\"--tau\", type=float, default=0.005)\n parser.add_argument(\"--exploration_noise\", type=float, default=0.1)\n parser.add_argument(\"--policy_noise\", type=float, default=0.2)\n parser.add_argument(\"--noise_clip\", type=float, default=0.5)\n parser.add_argument(\"--update_actor_freq\", type=int, default=2)\n parser.add_argument(\"--epoch\", type=int, default=5)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=20000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=8)\n parser.add_argument(\"--update_per_step\", type=float, default=0.125)\n parser.add_argument(\"--batch_size\", type=int, default=128)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[128, 128])\n parser.add_argument(\"--num_training_envs\", type=int, default=8)\n parser.add_argument(\"--num_test_envs\", type=int, default=100)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\"--n_step\", type=int, default=3)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n return parser.parse_known_args()[0]\n\n\ndef test_td3(args: argparse.Namespace = get_args(), enable_assertions: bool = True) -> None:\n env = gym.make(args.task)\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n args.max_action = space_info.action_info.max_action\n if args.reward_threshold is None:\n default_reward_threshold = {\"Pendulum-v0\": -250, \"Pendulum-v1\": -250}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n # you can also use tianshou.env.SubprocVectorEnv\n # training_envs = gym.make(args.task)\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n # test_envs = gym.make(args.task)\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n # model\n net = Net(state_shape=args.state_shape, hidden_sizes=args.hidden_sizes)\n actor = ContinuousActorDeterministic(\n preprocess_net=net, action_shape=args.action_shape, max_action=args.max_action\n ).to(\n args.device,\n )\n actor_optim = AdamOptimizerFactory(lr=args.actor_lr)\n net_c1 = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n concat=True,\n )\n critic1 = ContinuousCritic(preprocess_net=net_c1).to(args.device)\n critic1_optim = AdamOptimizerFactory(lr=args.critic_lr)\n net_c2 = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n concat=True,\n )\n critic2 = ContinuousCritic(preprocess_net=net_c2).to(args.device)\n critic2_optim = AdamOptimizerFactory(lr=args.critic_lr)\n policy = ContinuousDeterministicPolicy(\n actor=actor,\n action_space=env.action_space,\n exploration_noise=GaussianNoise(sigma=args.exploration_noise),\n )\n algorithm: TD3 = TD3(\n policy=policy,\n policy_optim=actor_optim,\n critic=critic1,\n critic_optim=critic1_optim,\n critic2=critic2,\n critic2_optim=critic2_optim,\n tau=args.tau,\n gamma=args.gamma,\n policy_noise=args.policy_noise,\n update_actor_freq=args.update_actor_freq,\n noise_clip=args.noise_clip,\n n_step_return_horizon=args.n_step,\n )\n # collector\n training_collector = Collector[CollectStats](\n algorithm,\n training_envs,\n VectorReplayBuffer(args.buffer_size, len(training_envs)),\n exploration_noise=True,\n )\n test_collector = Collector[CollectStats](algorithm, test_envs)\n # training_collector.collect(n_step=args.buffer_size)\n # log\n log_path = os.path.join(args.logdir, args.task, \"td3\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n update_step_num_gradient_steps_per_sample=args.update_per_step,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=True,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_td3_determinism() -> None:\n main_fn = lambda args: test_td3(args, enable_assertions=False)\n AlgorithmDeterminismTest(\"continuous_td3\", main_fn, get_args()).run()\n" }, { "path": "test/continuous/test_trpo.py", "content": "import argparse\nimport os\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch import nn\nfrom torch.distributions import Distribution, Independent, Normal\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom tianshou.algorithm import TRPO\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.reinforce import ProbabilisticActorPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OnPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.continuous import ContinuousActorProbabilistic, ContinuousCritic\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"Pendulum-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=1)\n parser.add_argument(\"--buffer_size\", type=int, default=50000)\n parser.add_argument(\"--lr\", type=float, default=1e-3)\n parser.add_argument(\"--gamma\", type=float, default=0.95)\n parser.add_argument(\"--epoch\", type=int, default=5)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=50000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=2048)\n parser.add_argument(\n \"--update_step_num_repetitions\", type=int, default=2\n ) # theoretically it should be 1\n parser.add_argument(\"--batch_size\", type=int, default=99999)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[64, 64])\n parser.add_argument(\"--num_training_envs\", type=int, default=16)\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n # trpo special\n parser.add_argument(\"--gae_lambda\", type=float, default=0.95)\n parser.add_argument(\"--return_scaling\", type=int, default=1)\n parser.add_argument(\"--advantage_normalization\", type=int, default=1)\n parser.add_argument(\"--optim_critic_iters\", type=int, default=5)\n parser.add_argument(\"--max_kl\", type=float, default=0.005)\n parser.add_argument(\"--backtrack_coeff\", type=float, default=0.8)\n parser.add_argument(\"--max_backtracks\", type=int, default=10)\n\n return parser.parse_known_args()[0]\n\n\ndef test_trpo(args: argparse.Namespace = get_args(), enable_assertions: bool = True) -> None:\n env = gym.make(args.task)\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n if args.reward_threshold is None:\n default_reward_threshold = {\"Pendulum-v0\": -250, \"Pendulum-v1\": -250}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n # you can also use tianshou.env.SubprocVectorEnv\n # training_envs = gym.make(args.task)\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n # test_envs = gym.make(args.task)\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n # model\n net = Net(\n state_shape=args.state_shape,\n hidden_sizes=args.hidden_sizes,\n activation=nn.Tanh,\n )\n actor = ContinuousActorProbabilistic(\n preprocess_net=net, action_shape=args.action_shape, unbounded=True\n ).to(args.device)\n critic = ContinuousCritic(\n preprocess_net=Net(\n state_shape=args.state_shape,\n hidden_sizes=args.hidden_sizes,\n activation=nn.Tanh,\n ),\n ).to(args.device)\n # orthogonal initialization\n for m in list(actor.modules()) + list(critic.modules()):\n if isinstance(m, torch.nn.Linear):\n torch.nn.init.orthogonal_(m.weight)\n torch.nn.init.zeros_(m.bias)\n optim = AdamOptimizerFactory(lr=args.lr)\n\n # replace DiagGuassian with Independent(Normal) which is equivalent\n # pass *logits to be consistent with policy.forward\n def dist(loc_scale: tuple[torch.Tensor, torch.Tensor]) -> Distribution:\n loc, scale = loc_scale\n return Independent(Normal(loc, scale), 1)\n\n policy = ProbabilisticActorPolicy(\n actor=actor,\n dist_fn=dist,\n action_space=env.action_space,\n )\n algorithm: TRPO = TRPO(\n policy=policy,\n critic=critic,\n optim=optim,\n gamma=args.gamma,\n return_scaling=args.return_scaling,\n advantage_normalization=args.advantage_normalization,\n gae_lambda=args.gae_lambda,\n optim_critic_iters=args.optim_critic_iters,\n max_kl=args.max_kl,\n backtrack_coeff=args.backtrack_coeff,\n max_backtracks=args.max_backtracks,\n )\n # collector\n training_collector = Collector[CollectStats](\n algorithm,\n training_envs,\n VectorReplayBuffer(args.buffer_size, len(training_envs)),\n )\n test_collector = Collector[CollectStats](algorithm, test_envs)\n # log\n log_path = os.path.join(args.logdir, args.task, \"trpo\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n # train\n result = algorithm.run_training(\n OnPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n update_step_num_repetitions=args.update_step_num_repetitions,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=True,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_trpo_determinism() -> None:\n main_fn = lambda args: test_trpo(args, enable_assertions=False)\n AlgorithmDeterminismTest(\"continuous_trpo\", main_fn, get_args()).run()\n" }, { "path": "test/determinism_test.py", "content": "from argparse import Namespace\nfrom collections.abc import Callable, Sequence\nfrom pathlib import Path\nfrom typing import Any\n\nimport pytest\nimport torch\n\nfrom tianshou.utils.determinism import TraceDeterminismTest, TraceLoggerContext\n\n\nclass TorchDeterministicModeContext:\n def __init__(self, mode: str | int = \"default\") -> None:\n self.new_mode = mode\n self.original_mode: str | int | None = None\n\n def __enter__(self) -> None:\n self.original_mode = torch.get_deterministic_debug_mode()\n torch.set_deterministic_debug_mode(self.new_mode)\n\n def __exit__(self, exc_type, exc_value, traceback): # type: ignore\n assert self.original_mode is not None\n torch.set_deterministic_debug_mode(self.original_mode)\n\n\nclass AlgorithmDeterminismTest:\n \"\"\"\n Represents a determinism test for Tianshou's RL algorithms.\n\n A test using this class should be added for every algorithm in Tianshou.\n Then, when making changes to one or more algorithms (e.g. refactoring), run the respective tests\n on the old branch (creating snapshots) and then on the new branch that contains the changes\n (comparing with the snapshots).\n\n Intended usage is therefore:\n\n 1. On the old branch: Set ENABLED=True and FORCE_SNAPSHOT_UPDATE=True and run the tests.\n 2. On the new branch: Set ENABLED=True and FORCE_SNAPSHOT_UPDATE=False and run the tests.\n 3. Inspect determinism_tests.log\n \"\"\"\n\n ENABLED = False\n \"\"\"\n whether determinism tests are enabled.\n \"\"\"\n FORCE_SNAPSHOT_UPDATE = False\n \"\"\"\n whether to force the update/creation of snapshots for every test.\n Enable this when running on the \"old\" branch and you want to prepare the snapshots\n for a comparison with the \"new\" branch.\n \"\"\"\n PASS_IF_CORE_MESSAGES_UNCHANGED = True\n \"\"\"\n whether to pass the test if only the core messages are unchanged.\n If this is False, then the full log is required to be equivalent, whereas if it is True,\n only the core messages need to be equivalent.\n The core messages test whether the algorithm produces the same network parameters.\n \"\"\"\n\n def __init__(\n self,\n name: str,\n main_fn: Callable[[Namespace], Any],\n args: Namespace,\n is_offline: bool = False,\n ignored_messages: Sequence[str] = (),\n ):\n \"\"\"\n :param name: the (unique!) name of the test\n :param main_fn: the function to be called for the test\n :param args: the arguments to be passed to the main function (some of which are overridden\n for the test)\n :param is_offline: whether the algorithm being tested is an offline algorithm and therefore\n does not configure the number of training environments (`num_training_envs`)\n :param ignored_messages: message fragments to ignore in the trace log (if any)\n \"\"\"\n self.determinism_test = TraceDeterminismTest(\n base_path=Path(__file__).parent / \"resources\" / \"determinism\",\n log_filename=\"determinism_tests.log\",\n core_messages=[\"Params\"],\n ignored_messages=ignored_messages,\n )\n self.name = name\n\n def set(attr: str, value: Any) -> None:\n old_value = getattr(args, attr)\n if old_value is None:\n raise ValueError(f\"Attribute '{attr}' is not defined for args: {args}\")\n setattr(args, attr, value)\n\n set(\"epoch\", 3)\n set(\"epoch_num_steps\", 100)\n set(\"device\", \"cpu\")\n if not is_offline:\n set(\"num_training_envs\", 1)\n set(\"num_test_envs\", 1)\n\n self.args = args\n self.main_fn = main_fn\n\n def run(self, update_snapshot: bool = False) -> None:\n \"\"\"\n :param update_snapshot: whether to update to snapshot (may be centrally overridden by\n FORCE_SNAPSHOT_UPDATE)\n \"\"\"\n if not self.ENABLED:\n pytest.skip(\"Algorithm determinism tests are disabled.\")\n\n if self.FORCE_SNAPSHOT_UPDATE:\n update_snapshot = True\n\n # run the actual process\n with TraceLoggerContext() as trace:\n with TorchDeterministicModeContext():\n self.main_fn(self.args)\n log = trace.get_log()\n\n self.determinism_test.check(\n log,\n self.name,\n create_reference_result=update_snapshot,\n pass_if_core_messages_unchanged=self.PASS_IF_CORE_MESSAGES_UNCHANGED,\n )\n" }, { "path": "test/discrete/__init__.py", "content": "" }, { "path": "test/discrete/test_a2c_with_il.py", "content": "import argparse\nimport os\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom gymnasium.spaces import Box\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom tianshou.algorithm import A2C, Algorithm, OffPolicyImitationLearning\nfrom tianshou.algorithm.imitation.imitation_base import ImitationPolicy\nfrom tianshou.algorithm.modelfree.reinforce import ProbabilisticActorPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OffPolicyTrainerParams, OnPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.discrete import DiscreteActor, DiscreteCritic\n\ntry:\n import envpool\nexcept ImportError:\n envpool = None\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"CartPole-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=1)\n parser.add_argument(\"--buffer_size\", type=int, default=20000)\n parser.add_argument(\"--lr\", type=float, default=1e-3)\n parser.add_argument(\"--il_lr\", type=float, default=1e-3)\n parser.add_argument(\"--gamma\", type=float, default=0.9)\n parser.add_argument(\"--epoch\", type=int, default=10)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=50000)\n parser.add_argument(\"--il_step_per_epoch\", type=int, default=1000)\n parser.add_argument(\"--collection_step_num_episodes\", type=int, default=16)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=16)\n parser.add_argument(\"--update_per_step\", type=float, default=1 / 16)\n parser.add_argument(\"--update_step_num_repetitions\", type=int, default=1)\n parser.add_argument(\"--batch_size\", type=int, default=64)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[64, 64])\n parser.add_argument(\"--imitation_hidden_sizes\", type=int, nargs=\"*\", default=[128])\n parser.add_argument(\"--num_training_envs\", type=int, default=16)\n parser.add_argument(\"--num_test_envs\", type=int, default=100)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n # a2c special\n parser.add_argument(\"--vf_coef\", type=float, default=0.5)\n parser.add_argument(\"--ent_coef\", type=float, default=0.0)\n parser.add_argument(\"--max_grad_norm\", type=float, default=None)\n parser.add_argument(\"--gae_lambda\", type=float, default=1.0)\n parser.add_argument(\"--return_scaling\", action=\"store_true\", default=False)\n return parser.parse_known_args()[0]\n\n\ndef test_a2c_with_il(\n args: argparse.Namespace = get_args(),\n enable_assertions: bool = True,\n skip_il: bool = False,\n) -> None:\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n\n if envpool is not None:\n training_envs = env = envpool.make(\n args.task,\n env_type=\"gymnasium\",\n num_envs=args.num_training_envs,\n seed=args.seed,\n )\n test_envs = envpool.make(\n args.task,\n env_type=\"gymnasium\",\n num_envs=args.num_test_envs,\n seed=args.seed,\n )\n else:\n env = gym.make(args.task)\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n args.state_shape = env.observation_space.shape or env.observation_space.n\n args.action_shape = env.action_space.shape or env.action_space.n\n if args.reward_threshold is None:\n default_reward_threshold = {\"CartPole-v1\": 195}\n args.reward_threshold = default_reward_threshold.get(args.task, env.spec.reward_threshold)\n # model\n net = Net(state_shape=args.state_shape, hidden_sizes=args.hidden_sizes)\n actor = DiscreteActor(preprocess_net=net, action_shape=args.action_shape).to(args.device)\n critic = DiscreteCritic(preprocess_net=net).to(args.device)\n optim = AdamOptimizerFactory(lr=args.lr)\n dist = torch.distributions.Categorical\n policy = ProbabilisticActorPolicy(\n actor=actor,\n dist_fn=dist,\n action_scaling=isinstance(env.action_space, Box),\n action_space=env.action_space,\n )\n algorithm: A2C = A2C(\n policy=policy,\n critic=critic,\n optim=optim,\n gamma=args.gamma,\n gae_lambda=args.gae_lambda,\n vf_coef=args.vf_coef,\n ent_coef=args.ent_coef,\n max_grad_norm=args.max_grad_norm,\n return_scaling=args.return_scaling,\n )\n # collector\n training_collector = Collector[CollectStats](\n algorithm,\n training_envs,\n VectorReplayBuffer(args.buffer_size, len(training_envs)),\n )\n training_collector.reset()\n test_collector = Collector[CollectStats](algorithm, test_envs)\n test_collector.reset()\n # log\n log_path = os.path.join(args.logdir, args.task, \"a2c\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n # trainer\n result = algorithm.run_training(\n OnPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n update_step_num_repetitions=args.update_step_num_repetitions,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n collection_step_num_episodes=args.collection_step_num_episodes,\n collection_step_num_env_steps=None,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=True,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n if skip_il:\n return\n\n # here we define an imitation collector with a trivial policy\n # if args.task == 'CartPole-v1':\n # env.spec.reward_threshold = 190 # lower the goal\n net = Net(state_shape=args.state_shape, hidden_sizes=args.hidden_sizes)\n actor = DiscreteActor(preprocess_net=net, action_shape=args.action_shape).to(args.device)\n optim = AdamOptimizerFactory(lr=args.il_lr)\n il_policy = ImitationPolicy(\n actor=actor,\n action_space=env.action_space,\n )\n il_algorithm: OffPolicyImitationLearning = OffPolicyImitationLearning(\n policy=il_policy,\n optim=optim,\n )\n if envpool is not None:\n il_env = envpool.make(\n args.task,\n env_type=\"gymnasium\",\n num_envs=args.num_test_envs,\n seed=args.seed,\n )\n else:\n il_env = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_test_envs)],\n )\n il_env.seed(args.seed)\n\n il_test_collector = Collector[CollectStats](\n il_algorithm,\n il_env,\n )\n training_collector.reset()\n result = il_algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=il_test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=True,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_ppo_determinism() -> None:\n main_fn = lambda args: test_a2c_with_il(args, enable_assertions=False, skip_il=True)\n AlgorithmDeterminismTest(\"discrete_a2c\", main_fn, get_args()).run()\n" }, { "path": "test/discrete/test_bdqn.py", "content": "import argparse\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom tianshou.algorithm import BDQN\nfrom tianshou.algorithm.modelfree.bdqn import BDQNPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env import ContinuousToDiscrete, DummyVectorEnv\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils.net.common import BranchingNet\nfrom tianshou.utils.torch_utils import policy_within_training_step\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n # task\n parser.add_argument(\"--task\", type=str, default=\"Pendulum-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n # network architecture\n parser.add_argument(\"--common_hidden_sizes\", type=int, nargs=\"*\", default=[64, 64])\n parser.add_argument(\"--action_hidden_sizes\", type=int, nargs=\"*\", default=[64])\n parser.add_argument(\"--value_hidden_sizes\", type=int, nargs=\"*\", default=[64])\n parser.add_argument(\"--action_per_branch\", type=int, default=40)\n # training hyperparameters\n parser.add_argument(\"--seed\", type=int, default=1626)\n parser.add_argument(\"--eps_test\", type=float, default=0.01)\n parser.add_argument(\"--eps_train\", type=float, default=0.76)\n parser.add_argument(\"--eps_decay\", type=float, default=1e-4)\n parser.add_argument(\"--buffer_size\", type=int, default=20000)\n parser.add_argument(\"--lr\", type=float, default=1e-3)\n parser.add_argument(\"--gamma\", type=float, default=0.9)\n parser.add_argument(\"--target_update_freq\", type=int, default=200)\n parser.add_argument(\"--epoch\", type=int, default=10)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=80000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=10)\n parser.add_argument(\"--update_per_step\", type=float, default=0.1)\n parser.add_argument(\"--batch_size\", type=int, default=128)\n parser.add_argument(\"--num_training_envs\", type=int, default=10)\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n return parser.parse_known_args()[0]\n\n\ndef test_bdq(args: argparse.Namespace = get_args(), enable_assertions: bool = True) -> None:\n env = gym.make(args.task)\n env = ContinuousToDiscrete(env, args.action_per_branch)\n\n if isinstance(env.observation_space, gym.spaces.Box):\n args.state_shape = env.observation_space.shape\n elif isinstance(env.observation_space, gym.spaces.Discrete):\n args.state_shape = int(env.observation_space.n)\n assert isinstance(env.action_space, gym.spaces.MultiDiscrete)\n args.num_branches = env.action_space.shape[0]\n\n if args.reward_threshold is None:\n default_reward_threshold = {\"Pendulum-v0\": -250, \"Pendulum-v1\": -250}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n\n print(\"Observations shape:\", args.state_shape)\n print(\"Num branches:\", args.num_branches)\n print(\"Actions per branch:\", args.action_per_branch)\n\n training_envs = DummyVectorEnv(\n [\n lambda: ContinuousToDiscrete(gym.make(args.task), args.action_per_branch)\n for _ in range(args.num_training_envs)\n ],\n )\n test_envs = DummyVectorEnv(\n [\n lambda: ContinuousToDiscrete(gym.make(args.task), args.action_per_branch)\n for _ in range(args.num_test_envs)\n ],\n )\n\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n # model\n net = BranchingNet(\n state_shape=args.state_shape,\n num_branches=args.num_branches,\n action_per_branch=args.action_per_branch,\n common_hidden_sizes=args.common_hidden_sizes,\n value_hidden_sizes=args.value_hidden_sizes,\n action_hidden_sizes=args.action_hidden_sizes,\n ).to(args.device)\n optim = AdamOptimizerFactory(lr=args.lr)\n policy = BDQNPolicy(\n model=net,\n action_space=env.action_space, # type: ignore[arg-type] # TODO: should `BranchingDQNPolicy` support also `MultiDiscrete` action spaces?\n eps_training=args.eps_train,\n eps_inference=args.eps_test,\n )\n algorithm: BDQN = BDQN(\n policy=policy,\n optim=optim,\n gamma=args.gamma,\n target_update_freq=args.target_update_freq,\n )\n # collector\n training_collector = Collector[CollectStats](\n algorithm,\n training_envs,\n VectorReplayBuffer(args.buffer_size, args.num_training_envs),\n exploration_noise=True,\n )\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=False)\n\n # initial data collection\n with policy_within_training_step(policy):\n training_collector.reset()\n training_collector.collect(n_step=args.batch_size * args.num_training_envs)\n\n def train_fn(epoch: int, env_step: int) -> None: # exp decay\n eps = max(args.eps_train * (1 - args.eps_decay) ** env_step, args.eps_test)\n policy.set_eps_training(eps)\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n update_step_num_gradient_steps_per_sample=args.update_per_step,\n training_fn=train_fn,\n stop_fn=stop_fn,\n test_in_training=True,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_bdq_determinism() -> None:\n main_fn = lambda args: test_bdq(args, enable_assertions=False)\n AlgorithmDeterminismTest(\"discrete_bdq\", main_fn, get_args()).run()\n" }, { "path": "test/discrete/test_c51.py", "content": "import argparse\nimport os\nimport pickle\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom tianshou.algorithm import C51\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.c51 import C51Policy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import (\n Collector,\n CollectStats,\n PrioritizedVectorReplayBuffer,\n ReplayBuffer,\n VectorReplayBuffer,\n)\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.space_info import SpaceInfo\nfrom tianshou.utils.torch_utils import policy_within_training_step\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"CartPole-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=1626)\n parser.add_argument(\"--eps_test\", type=float, default=0.05)\n parser.add_argument(\"--eps_train\", type=float, default=0.1)\n parser.add_argument(\"--buffer_size\", type=int, default=20000)\n parser.add_argument(\"--lr\", type=float, default=1e-3)\n parser.add_argument(\"--gamma\", type=float, default=0.9)\n parser.add_argument(\"--num_atoms\", type=int, default=51)\n parser.add_argument(\"--v_min\", type=float, default=-10.0)\n parser.add_argument(\"--v_max\", type=float, default=10.0)\n parser.add_argument(\"--n_step\", type=int, default=3)\n parser.add_argument(\"--target_update_freq\", type=int, default=320)\n parser.add_argument(\"--epoch\", type=int, default=10)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=8000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=8)\n parser.add_argument(\"--update_per_step\", type=float, default=0.125)\n parser.add_argument(\"--batch_size\", type=int, default=64)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[128, 128, 128, 128])\n parser.add_argument(\"--num_training_envs\", type=int, default=8)\n parser.add_argument(\"--num_test_envs\", type=int, default=100)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\"--prioritized_replay\", action=\"store_true\", default=False)\n parser.add_argument(\"--alpha\", type=float, default=0.6)\n parser.add_argument(\"--beta\", type=float, default=0.4)\n parser.add_argument(\"--resume\", action=\"store_true\")\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n parser.add_argument(\"--save_interval\", type=int, default=4)\n return parser.parse_known_args()[0]\n\n\ndef test_c51(args: argparse.Namespace = get_args(), enable_assertions: bool = True) -> None:\n env = gym.make(args.task)\n assert isinstance(env.action_space, gym.spaces.Discrete)\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n if args.reward_threshold is None:\n default_reward_threshold = {\"CartPole-v1\": 195}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n\n # model\n net = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n softmax=True,\n num_atoms=args.num_atoms,\n )\n optim = AdamOptimizerFactory(lr=args.lr)\n policy = C51Policy(\n model=net,\n action_space=env.action_space,\n observation_space=env.observation_space,\n num_atoms=args.num_atoms,\n v_min=args.v_min,\n v_max=args.v_max,\n eps_training=args.eps_train,\n eps_inference=args.eps_test,\n )\n algorithm: C51 = C51(\n policy=policy,\n optim=optim,\n gamma=args.gamma,\n n_step_return_horizon=args.n_step,\n target_update_freq=args.target_update_freq,\n ).to(args.device)\n\n # buffer\n buf: ReplayBuffer\n if args.prioritized_replay:\n buf = PrioritizedVectorReplayBuffer(\n args.buffer_size,\n buffer_num=len(training_envs),\n alpha=args.alpha,\n beta=args.beta,\n )\n else:\n buf = VectorReplayBuffer(args.buffer_size, buffer_num=len(training_envs))\n\n # collectors\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buf, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n\n # initial data collection\n with policy_within_training_step(policy):\n training_collector.reset()\n training_collector.collect(n_step=args.batch_size * args.num_training_envs)\n\n # logger\n log_path = os.path.join(args.logdir, args.task, \"c51\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer, save_interval=args.save_interval)\n\n def save_best_fn(algorithm: Algorithm) -> None:\n torch.save(algorithm.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n def train_fn(epoch: int, env_step: int) -> None:\n # eps annnealing, just a demo\n if env_step <= 10000:\n policy.set_eps_training(args.eps_train)\n elif env_step <= 50000:\n eps = args.eps_train - (env_step - 10000) / 40000 * (0.9 * args.eps_train)\n policy.set_eps_training(eps)\n else:\n policy.set_eps_training(0.1 * args.eps_train)\n\n def save_checkpoint_fn(epoch: int, env_step: int, gradient_step: int) -> str:\n # see also: https://pytorch.org/tutorials/beginner/saving_loading_models.html\n ckpt_path = os.path.join(log_path, \"checkpoint.pth\")\n # Example: saving by epoch num\n # ckpt_path = os.path.join(log_path, f\"checkpoint_{epoch}.pth\")\n torch.save(\n algorithm.state_dict(),\n ckpt_path,\n )\n buffer_path = os.path.join(log_path, \"train_buffer.pkl\")\n with open(buffer_path, \"wb\") as f:\n pickle.dump(training_collector.buffer, f)\n return ckpt_path\n\n if args.resume:\n # load from existing checkpoint\n print(f\"Loading agent under {log_path}\")\n ckpt_path = os.path.join(log_path, \"checkpoint.pth\")\n if os.path.exists(ckpt_path):\n checkpoint = torch.load(ckpt_path, map_location=args.device)\n algorithm.load_state_dict(checkpoint)\n print(\"Successfully restore policy and optim.\")\n else:\n print(\"Fail to restore policy and optim.\")\n buffer_path = os.path.join(log_path, \"train_buffer.pkl\")\n if os.path.exists(buffer_path):\n with open(buffer_path, \"rb\") as f:\n training_collector.buffer = pickle.load(f)\n print(\"Successfully restore buffer.\")\n else:\n print(\"Fail to restore buffer.\")\n\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n update_step_num_gradient_steps_per_sample=args.update_per_step,\n training_fn=train_fn,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n resume_from_log=args.resume,\n save_checkpoint_fn=save_checkpoint_fn,\n test_in_training=True,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_c51_resume(args: argparse.Namespace = get_args()) -> None:\n args.resume = True\n test_c51(args)\n\n\ndef test_pc51(args: argparse.Namespace = get_args()) -> None:\n args.prioritized_replay = True\n args.gamma = 0.95\n args.seed = 1\n test_c51(args)\n\n\ndef test_c51_determinism() -> None:\n main_fn = lambda args: test_c51(args, enable_assertions=False)\n AlgorithmDeterminismTest(\"discrete_c51\", main_fn, get_args()).run()\n" }, { "path": "test/discrete/test_discrete_sac.py", "content": "import argparse\nimport os\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom tianshou.algorithm import DiscreteSAC\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.discrete_sac import (\n DiscreteSACPolicy,\n)\nfrom tianshou.algorithm.modelfree.sac import AutoAlpha\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.discrete import DiscreteActor, DiscreteCritic\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"CartPole-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=1)\n parser.add_argument(\"--buffer_size\", type=int, default=20000)\n parser.add_argument(\"--actor_lr\", type=float, default=1e-4)\n parser.add_argument(\"--critic_lr\", type=float, default=1e-3)\n parser.add_argument(\"--alpha_lr\", type=float, default=3e-4)\n parser.add_argument(\"--gamma\", type=float, default=0.95)\n parser.add_argument(\"--tau\", type=float, default=0.005)\n parser.add_argument(\"--alpha\", type=float, default=0.05)\n parser.add_argument(\"--auto_alpha\", action=\"store_true\", default=False)\n parser.add_argument(\"--epoch\", type=int, default=5)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=10000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=10)\n parser.add_argument(\"--update_per_step\", type=float, default=0.1)\n parser.add_argument(\"--batch_size\", type=int, default=64)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[64, 64])\n parser.add_argument(\"--num_training_envs\", type=int, default=10)\n parser.add_argument(\"--num_test_envs\", type=int, default=100)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\"--n_step\", type=int, default=3)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n return parser.parse_known_args()[0]\n\n\ndef test_discrete_sac(\n args: argparse.Namespace = get_args(),\n enable_assertions: bool = True,\n) -> None:\n env = gym.make(args.task)\n assert isinstance(env.action_space, gym.spaces.Discrete)\n\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n\n if args.reward_threshold is None:\n default_reward_threshold = {\"CartPole-v1\": 170} # lower the goal\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n # model\n obs_dim = space_info.observation_info.obs_dim\n action_dim = space_info.action_info.action_dim\n net = Net(state_shape=args.state_shape, hidden_sizes=args.hidden_sizes)\n actor = DiscreteActor(\n preprocess_net=net, action_shape=args.action_shape, softmax_output=False\n ).to(args.device)\n actor_optim = AdamOptimizerFactory(lr=args.actor_lr)\n net_c1 = Net(state_shape=args.state_shape, hidden_sizes=args.hidden_sizes)\n critic1 = DiscreteCritic(preprocess_net=net_c1, last_size=action_dim).to(args.device)\n critic1_optim = AdamOptimizerFactory(lr=args.critic_lr)\n net_c2 = Net(state_shape=obs_dim, hidden_sizes=args.hidden_sizes)\n critic2 = DiscreteCritic(preprocess_net=net_c2, last_size=action_dim).to(args.device)\n critic2_optim = AdamOptimizerFactory(lr=args.critic_lr)\n\n # better not to use auto alpha in CartPole\n if args.auto_alpha:\n target_entropy = 0.98 * np.log(action_dim)\n log_alpha = 0.0\n alpha_optim = AdamOptimizerFactory(lr=args.alpha_lr)\n args.alpha = AutoAlpha(target_entropy, log_alpha, alpha_optim).to(args.device)\n\n policy = DiscreteSACPolicy(\n actor=actor,\n action_space=env.action_space,\n )\n algorithm = DiscreteSAC(\n policy=policy,\n policy_optim=actor_optim,\n critic=critic1,\n critic_optim=critic1_optim,\n critic2=critic2,\n critic2_optim=critic2_optim,\n tau=args.tau,\n gamma=args.gamma,\n alpha=args.alpha,\n n_step_return_horizon=args.n_step,\n )\n # collector\n training_collector = Collector[CollectStats](\n algorithm,\n training_envs,\n VectorReplayBuffer(args.buffer_size, len(training_envs)),\n )\n test_collector = Collector[CollectStats](algorithm, test_envs)\n # training_collector.collect(n_step=args.buffer_size)\n # log\n log_path = os.path.join(args.logdir, args.task, \"discrete_sac\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n update_step_num_gradient_steps_per_sample=args.update_per_step,\n test_in_training=False,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_discrete_sac_determinism() -> None:\n main_fn = lambda args: test_discrete_sac(args, enable_assertions=False)\n ignored_messages = [\n \"Params[actor_old]\", # actor_old only present in v1 (due to flawed inheritance)\n ]\n AlgorithmDeterminismTest(\n \"discrete_sac\", main_fn, get_args(), ignored_messages=ignored_messages\n ).run()\n" }, { "path": "test/discrete/test_dqn.py", "content": "import argparse\nimport os\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom tianshou.algorithm import DQN\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.dqn import DiscreteQLearningPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import (\n Collector,\n CollectStats,\n PrioritizedVectorReplayBuffer,\n ReplayBuffer,\n VectorReplayBuffer,\n)\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.space_info import SpaceInfo\nfrom tianshou.utils.torch_utils import policy_within_training_step\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"CartPole-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=1626)\n parser.add_argument(\"--eps_test\", type=float, default=0.05)\n parser.add_argument(\"--eps_train\", type=float, default=0.1)\n parser.add_argument(\"--buffer_size\", type=int, default=20000)\n parser.add_argument(\"--lr\", type=float, default=1e-3)\n parser.add_argument(\"--gamma\", type=float, default=0.9)\n parser.add_argument(\"--n_step\", type=int, default=3)\n parser.add_argument(\"--target_update_freq\", type=int, default=320)\n parser.add_argument(\"--epoch\", type=int, default=20)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=10000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=10)\n parser.add_argument(\"--update_per_step\", type=float, default=0.1)\n parser.add_argument(\"--batch_size\", type=int, default=64)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[128, 128, 128, 128])\n parser.add_argument(\"--num_training_envs\", type=int, default=10)\n parser.add_argument(\"--num_test_envs\", type=int, default=100)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\"--prioritized_replay\", action=\"store_true\", default=False)\n parser.add_argument(\"--alpha\", type=float, default=0.6)\n parser.add_argument(\"--beta\", type=float, default=0.4)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n return parser.parse_known_args()[0]\n\n\ndef test_dqn(args: argparse.Namespace = get_args(), enable_assertions: bool = True) -> None:\n env = gym.make(args.task)\n assert isinstance(env.action_space, gym.spaces.Discrete)\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n if args.reward_threshold is None:\n default_reward_threshold = {\"CartPole-v1\": 195}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n\n # Q_param = V_param = {\"hidden_sizes\": [128]}\n # model\n net = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n # dueling=(Q_param, V_param),\n ).to(args.device)\n optim = AdamOptimizerFactory(lr=args.lr)\n policy = DiscreteQLearningPolicy(\n model=net,\n action_space=env.action_space,\n observation_space=env.observation_space,\n eps_training=args.eps_train,\n eps_inference=args.eps_test,\n )\n algorithm: DQN = DQN(\n policy=policy,\n optim=optim,\n gamma=args.gamma,\n n_step_return_horizon=args.n_step,\n target_update_freq=args.target_update_freq,\n )\n\n # buffer\n buf: ReplayBuffer\n if args.prioritized_replay:\n buf = PrioritizedVectorReplayBuffer(\n args.buffer_size,\n buffer_num=len(training_envs),\n alpha=args.alpha,\n beta=args.beta,\n )\n else:\n buf = VectorReplayBuffer(args.buffer_size, buffer_num=len(training_envs))\n\n # collectors\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buf, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n\n # initial data collection\n with policy_within_training_step(policy):\n training_collector.reset()\n training_collector.collect(n_step=args.batch_size * args.num_training_envs)\n\n # logger\n log_path = os.path.join(args.logdir, args.task, \"dqn\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n def train_fn(epoch: int, env_step: int) -> None:\n # eps annnealing, just a demo\n if env_step <= 10000:\n policy.set_eps_training(args.eps_train)\n elif env_step <= 50000:\n eps = args.eps_train - (env_step - 10000) / 40000 * (0.9 * args.eps_train)\n policy.set_eps_training(eps)\n else:\n policy.set_eps_training(0.1 * args.eps_train)\n\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n update_step_num_gradient_steps_per_sample=args.update_per_step,\n training_fn=train_fn,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=True,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_dqn_determinism() -> None:\n main_fn = lambda args: test_dqn(args, enable_assertions=False)\n AlgorithmDeterminismTest(\"discrete_dqn\", main_fn, get_args()).run()\n\n\ndef test_pdqn(args: argparse.Namespace = get_args()) -> None:\n args.prioritized_replay = True\n args.gamma = 0.95\n args.seed = 1\n test_dqn(args)\n" }, { "path": "test/discrete/test_drqn.py", "content": "import argparse\nimport os\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom tianshou.algorithm import DQN\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.dqn import DiscreteQLearningPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Recurrent\nfrom tianshou.utils.space_info import SpaceInfo\nfrom tianshou.utils.torch_utils import policy_within_training_step\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"CartPole-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=1)\n parser.add_argument(\"--eps_test\", type=float, default=0.05)\n parser.add_argument(\"--eps_train\", type=float, default=0.1)\n parser.add_argument(\"--buffer_size\", type=int, default=20000)\n parser.add_argument(\"--stack_num\", type=int, default=4)\n parser.add_argument(\"--lr\", type=float, default=1e-3)\n parser.add_argument(\"--gamma\", type=float, default=0.95)\n parser.add_argument(\"--n_step\", type=int, default=3)\n parser.add_argument(\"--target_update_freq\", type=int, default=320)\n parser.add_argument(\"--epoch\", type=int, default=5)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=20000)\n parser.add_argument(\"--update_per_step\", type=float, default=1 / 16)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=16)\n parser.add_argument(\"--batch_size\", type=int, default=128)\n parser.add_argument(\"--layer_num\", type=int, default=2)\n parser.add_argument(\"--num_training_envs\", type=int, default=16)\n parser.add_argument(\"--num_test_envs\", type=int, default=100)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n return parser.parse_known_args()[0]\n\n\ndef test_drqn(args: argparse.Namespace = get_args(), enable_assertions: bool = True) -> None:\n env = gym.make(args.task)\n assert isinstance(env.action_space, gym.spaces.Discrete)\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n if args.reward_threshold is None:\n default_reward_threshold = {\"CartPole-v1\": 195}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n # training_envs = gym.make(args.task)\n # you can also use tianshou.env.SubprocVectorEnv\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n # test_envs = gym.make(args.task)\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n # model\n net = Recurrent(\n layer_num=args.layer_num,\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n ).to(\n args.device,\n )\n optim = AdamOptimizerFactory(lr=args.lr)\n policy = DiscreteQLearningPolicy(\n model=net,\n action_space=env.action_space,\n eps_training=args.eps_train,\n eps_inference=args.eps_test,\n )\n algorithm: DQN = DQN(\n policy=policy,\n optim=optim,\n gamma=args.gamma,\n n_step_return_horizon=args.n_step,\n target_update_freq=args.target_update_freq,\n )\n\n # collector\n buffer = VectorReplayBuffer(\n args.buffer_size,\n buffer_num=len(training_envs),\n stack_num=args.stack_num,\n ignore_obs_next=True,\n )\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buffer, exploration_noise=True\n )\n # the stack_num is for RNN training: sample framestack obs\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n\n # initial data collection\n with policy_within_training_step(policy):\n training_collector.reset()\n training_collector.collect(n_step=args.batch_size * args.num_training_envs)\n\n # log\n log_path = os.path.join(args.logdir, args.task, \"drqn\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n update_step_num_gradient_steps_per_sample=args.update_per_step,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=True,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_drqn_determinism() -> None:\n main_fn = lambda args: test_drqn(args, enable_assertions=False)\n AlgorithmDeterminismTest(\"discrete_drqn\", main_fn, get_args()).run()\n" }, { "path": "test/discrete/test_fqf.py", "content": "import argparse\nimport os\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom tianshou.algorithm import FQF\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.fqf import FQFPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory, RMSpropOptimizerFactory\nfrom tianshou.data import (\n Collector,\n CollectStats,\n PrioritizedVectorReplayBuffer,\n ReplayBuffer,\n VectorReplayBuffer,\n)\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.discrete import FractionProposalNetwork, FullQuantileFunction\nfrom tianshou.utils.space_info import SpaceInfo\nfrom tianshou.utils.torch_utils import policy_within_training_step\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"CartPole-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=1)\n parser.add_argument(\"--eps_test\", type=float, default=0.05)\n parser.add_argument(\"--eps_train\", type=float, default=0.1)\n parser.add_argument(\"--buffer_size\", type=int, default=20000)\n parser.add_argument(\"--lr\", type=float, default=3e-3)\n parser.add_argument(\"--fraction_lr\", type=float, default=2.5e-9)\n parser.add_argument(\"--gamma\", type=float, default=0.9)\n parser.add_argument(\"--num_fractions\", type=int, default=32)\n parser.add_argument(\"--num_cosines\", type=int, default=64)\n parser.add_argument(\"--ent_coef\", type=float, default=10.0)\n parser.add_argument(\"--n_step\", type=int, default=3)\n parser.add_argument(\"--target_update_freq\", type=int, default=320)\n parser.add_argument(\"--epoch\", type=int, default=10)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=10000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=10)\n parser.add_argument(\"--update_per_step\", type=float, default=0.1)\n parser.add_argument(\"--batch_size\", type=int, default=64)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[64, 64, 64])\n parser.add_argument(\"--num_training_envs\", type=int, default=10)\n parser.add_argument(\"--num_test_envs\", type=int, default=100)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\"--prioritized_replay\", action=\"store_true\", default=False)\n parser.add_argument(\"--alpha\", type=float, default=0.6)\n parser.add_argument(\"--beta\", type=float, default=0.4)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n return parser.parse_known_args()[0]\n\n\ndef test_fqf(args: argparse.Namespace = get_args(), enable_assertions: bool = True) -> None:\n env = gym.make(args.task)\n space_info = SpaceInfo.from_env(env)\n assert isinstance(env.action_space, gym.spaces.Discrete)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n if args.reward_threshold is None:\n default_reward_threshold = {\"CartPole-v1\": 195}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n\n # model\n feature_net = Net(\n state_shape=args.state_shape,\n action_shape=args.hidden_sizes[-1],\n hidden_sizes=args.hidden_sizes[:-1],\n softmax=False,\n )\n net = FullQuantileFunction(\n preprocess_net=feature_net,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n num_cosines=args.num_cosines,\n )\n optim = AdamOptimizerFactory(lr=args.lr)\n fraction_net = FractionProposalNetwork(args.num_fractions, net.input_dim)\n fraction_optim = RMSpropOptimizerFactory(lr=args.fraction_lr)\n policy = FQFPolicy(\n model=net,\n fraction_model=fraction_net,\n action_space=env.action_space,\n eps_training=args.eps_train,\n eps_inference=args.eps_test,\n )\n algorithm: FQF = FQF(\n policy=policy,\n optim=optim,\n fraction_optim=fraction_optim,\n gamma=args.gamma,\n num_fractions=args.num_fractions,\n ent_coef=args.ent_coef,\n n_step_return_horizon=args.n_step,\n target_update_freq=args.target_update_freq,\n ).to(args.device)\n\n # buffer\n buf: ReplayBuffer\n if args.prioritized_replay:\n buf = PrioritizedVectorReplayBuffer(\n args.buffer_size,\n buffer_num=len(training_envs),\n alpha=args.alpha,\n beta=args.beta,\n )\n else:\n buf = VectorReplayBuffer(args.buffer_size, buffer_num=len(training_envs))\n\n # collectors\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buf, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n\n # initial data collection\n with policy_within_training_step(policy):\n training_collector.reset()\n training_collector.collect(n_step=args.batch_size * args.num_training_envs)\n\n # logger\n log_path = os.path.join(args.logdir, args.task, \"fqf\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n def train_fn(epoch: int, env_step: int) -> None:\n # eps annnealing, just a demo\n if env_step <= 10000:\n policy.set_eps_training(args.eps_train)\n elif env_step <= 50000:\n eps = args.eps_train - (env_step - 10000) / 40000 * (0.9 * args.eps_train)\n policy.set_eps_training(eps)\n else:\n policy.set_eps_training(0.1 * args.eps_train)\n\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n training_fn=train_fn,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n update_step_num_gradient_steps_per_sample=args.update_per_step,\n test_in_training=True,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_pfqf(args: argparse.Namespace = get_args()) -> None:\n args.prioritized_replay = True\n args.gamma = 0.95\n test_fqf(args)\n\n\ndef test_fqf_determinism() -> None:\n main_fn = lambda args: test_fqf(args, enable_assertions=False)\n AlgorithmDeterminismTest(\"discrete_fqf\", main_fn, get_args()).run()\n" }, { "path": "test/discrete/test_iqn.py", "content": "import argparse\nimport os\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom tianshou.algorithm import IQN\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.iqn import IQNPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import (\n Collector,\n CollectStats,\n PrioritizedVectorReplayBuffer,\n ReplayBuffer,\n VectorReplayBuffer,\n)\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.discrete import ImplicitQuantileNetwork\nfrom tianshou.utils.space_info import SpaceInfo\nfrom tianshou.utils.torch_utils import policy_within_training_step\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"CartPole-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=0)\n parser.add_argument(\"--eps_test\", type=float, default=0.05)\n parser.add_argument(\"--eps_train\", type=float, default=0.1)\n parser.add_argument(\"--buffer_size\", type=int, default=20000)\n parser.add_argument(\"--lr\", type=float, default=3e-3)\n parser.add_argument(\"--gamma\", type=float, default=0.9)\n parser.add_argument(\"--sample_size\", type=int, default=32)\n parser.add_argument(\"--online_sample_size\", type=int, default=8)\n parser.add_argument(\"--target_sample_size\", type=int, default=8)\n parser.add_argument(\"--num_cosines\", type=int, default=64)\n parser.add_argument(\"--n_step\", type=int, default=3)\n parser.add_argument(\"--target_update_freq\", type=int, default=320)\n parser.add_argument(\"--epoch\", type=int, default=10)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=10000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=10)\n parser.add_argument(\"--update_per_step\", type=float, default=0.1)\n parser.add_argument(\"--batch_size\", type=int, default=64)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[64, 64, 64])\n parser.add_argument(\"--num_training_envs\", type=int, default=10)\n parser.add_argument(\"--num_test_envs\", type=int, default=100)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\"--prioritized_replay\", action=\"store_true\", default=False)\n parser.add_argument(\"--alpha\", type=float, default=0.6)\n parser.add_argument(\"--beta\", type=float, default=0.4)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n return parser.parse_known_args()[0]\n\n\ndef test_iqn(args: argparse.Namespace = get_args(), enable_assertions: bool = True) -> None:\n env = gym.make(args.task)\n space_info = SpaceInfo.from_env(env)\n assert isinstance(env.action_space, gym.spaces.Discrete)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n if args.reward_threshold is None:\n default_reward_threshold = {\"CartPole-v1\": 195}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n\n # model\n feature_net = Net(\n state_shape=args.state_shape,\n action_shape=args.hidden_sizes[-1],\n hidden_sizes=args.hidden_sizes[:-1],\n softmax=False,\n )\n net = ImplicitQuantileNetwork(\n preprocess_net=feature_net,\n action_shape=args.action_shape,\n num_cosines=args.num_cosines,\n )\n optim = AdamOptimizerFactory(lr=args.lr)\n policy = IQNPolicy(\n model=net,\n action_space=env.action_space,\n sample_size=args.sample_size,\n online_sample_size=args.online_sample_size,\n target_sample_size=args.target_sample_size,\n eps_training=args.eps_train,\n eps_inference=args.eps_test,\n )\n algorithm: IQN = IQN(\n policy=policy,\n optim=optim,\n gamma=args.gamma,\n n_step_return_horizon=args.n_step,\n target_update_freq=args.target_update_freq,\n ).to(args.device)\n\n # buffer\n buf: ReplayBuffer\n if args.prioritized_replay:\n buf = PrioritizedVectorReplayBuffer(\n args.buffer_size,\n buffer_num=len(training_envs),\n alpha=args.alpha,\n beta=args.beta,\n )\n else:\n buf = VectorReplayBuffer(args.buffer_size, buffer_num=len(training_envs))\n\n # collectors\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buf, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n\n # initial data collection\n with policy_within_training_step(policy):\n training_collector.reset()\n training_collector.collect(n_step=args.batch_size * args.num_training_envs)\n\n # logger\n log_path = os.path.join(args.logdir, args.task, \"iqn\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n def train_fn(epoch: int, env_step: int) -> None:\n # eps annnealing, just a demo\n if env_step <= 10000:\n policy.set_eps_training(args.eps_train)\n elif env_step <= 50000:\n eps = args.eps_train - (env_step - 10000) / 40000 * (0.9 * args.eps_train)\n policy.set_eps_training(eps)\n else:\n policy.set_eps_training(0.1 * args.eps_train)\n\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n training_fn=train_fn,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n update_step_num_gradient_steps_per_sample=args.update_per_step,\n test_in_training=True,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_piqn(args: argparse.Namespace = get_args()) -> None:\n args.prioritized_replay = True\n args.gamma = 0.95\n test_iqn(args)\n\n\ndef test_iqn_determinism() -> None:\n main_fn = lambda args: test_iqn(args, enable_assertions=False)\n AlgorithmDeterminismTest(\"discrete_iqn\", main_fn, get_args()).run()\n" }, { "path": "test/discrete/test_ppo_discrete.py", "content": "import argparse\nimport os\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom tianshou.algorithm import PPO\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.reinforce import DiscreteActorPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OnPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import (\n ActionReprNet,\n ActionReprNetDataParallelWrapper,\n ActorCritic,\n DataParallelNet,\n Net,\n)\nfrom tianshou.utils.net.discrete import DiscreteActor, DiscreteCritic\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"CartPole-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=1626)\n parser.add_argument(\"--buffer_size\", type=int, default=20000)\n parser.add_argument(\"--lr\", type=float, default=3e-4)\n parser.add_argument(\"--gamma\", type=float, default=0.99)\n parser.add_argument(\"--epoch\", type=int, default=10)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=50000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=2000)\n parser.add_argument(\"--update_step_num_repetitions\", type=int, default=10)\n parser.add_argument(\"--batch_size\", type=int, default=64)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[64, 64])\n parser.add_argument(\"--num_training_envs\", type=int, default=20)\n parser.add_argument(\"--num_test_envs\", type=int, default=100)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n # ppo special\n parser.add_argument(\"--vf_coef\", type=float, default=0.5)\n parser.add_argument(\"--ent_coef\", type=float, default=0.0)\n parser.add_argument(\"--eps_clip\", type=float, default=0.2)\n parser.add_argument(\"--max_grad_norm\", type=float, default=0.5)\n parser.add_argument(\"--gae_lambda\", type=float, default=0.95)\n parser.add_argument(\"--return_scaling\", type=int, default=0)\n parser.add_argument(\"--advantage_normalization\", type=int, default=0)\n parser.add_argument(\"--recompute_adv\", type=int, default=0)\n parser.add_argument(\"--dual_clip\", type=float, default=None)\n parser.add_argument(\"--value_clip\", type=int, default=0)\n return parser.parse_known_args()[0]\n\n\ndef test_ppo(args: argparse.Namespace = get_args(), enable_assertions: bool = True) -> None:\n env = gym.make(args.task)\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n if args.reward_threshold is None:\n default_reward_threshold = {\"CartPole-v1\": 195}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n # training_envs = gym.make(args.task)\n # you can also use tianshou.env.SubprocVectorEnv\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n # test_envs = gym.make(args.task)\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n # model\n net = Net(state_shape=args.state_shape, hidden_sizes=args.hidden_sizes)\n critic: DiscreteCritic | DataParallelNet\n actor: ActionReprNet\n if torch.cuda.is_available():\n actor = ActionReprNetDataParallelWrapper(\n DiscreteActor(preprocess_net=net, action_shape=args.action_shape).to(args.device)\n )\n critic = DataParallelNet(DiscreteCritic(preprocess_net=net).to(args.device))\n else:\n actor = DiscreteActor(preprocess_net=net, action_shape=args.action_shape).to(args.device)\n critic = DiscreteCritic(preprocess_net=net).to(args.device)\n actor_critic = ActorCritic(actor, critic)\n # orthogonal initialization\n for m in actor_critic.modules():\n if isinstance(m, torch.nn.Linear):\n torch.nn.init.orthogonal_(m.weight)\n torch.nn.init.zeros_(m.bias)\n optim = AdamOptimizerFactory(lr=args.lr)\n dist = torch.distributions.Categorical\n policy = DiscreteActorPolicy(\n actor=actor,\n dist_fn=dist,\n action_space=env.action_space,\n deterministic_eval=True,\n )\n algorithm: PPO = PPO(\n policy=policy,\n critic=critic,\n optim=optim,\n gamma=args.gamma,\n max_grad_norm=args.max_grad_norm,\n eps_clip=args.eps_clip,\n vf_coef=args.vf_coef,\n ent_coef=args.ent_coef,\n gae_lambda=args.gae_lambda,\n return_scaling=args.return_scaling,\n dual_clip=args.dual_clip,\n value_clip=args.value_clip,\n advantage_normalization=args.advantage_normalization,\n recompute_advantage=args.recompute_adv,\n )\n # collector\n training_collector = Collector[CollectStats](\n algorithm,\n training_envs,\n VectorReplayBuffer(args.buffer_size, len(training_envs)),\n )\n test_collector = Collector[CollectStats](algorithm, test_envs)\n # log\n log_path = os.path.join(args.logdir, args.task, \"ppo\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n # trainer\n result = algorithm.run_training(\n OnPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n update_step_num_repetitions=args.update_step_num_repetitions,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=True,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_ppo_determinism() -> None:\n main_fn = lambda args: test_ppo(args, enable_assertions=False)\n AlgorithmDeterminismTest(\"discrete_ppo\", main_fn, get_args()).run()\n" }, { "path": "test/discrete/test_qrdqn.py", "content": "import argparse\nimport os\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom tianshou.algorithm import QRDQN\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.qrdqn import QRDQNPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import (\n Collector,\n CollectStats,\n PrioritizedVectorReplayBuffer,\n VectorReplayBuffer,\n)\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.space_info import SpaceInfo\nfrom tianshou.utils.torch_utils import policy_within_training_step\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"CartPole-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=1)\n parser.add_argument(\"--eps_test\", type=float, default=0.05)\n parser.add_argument(\"--eps_train\", type=float, default=0.1)\n parser.add_argument(\"--buffer_size\", type=int, default=20000)\n parser.add_argument(\"--lr\", type=float, default=1e-3)\n parser.add_argument(\"--gamma\", type=float, default=0.9)\n parser.add_argument(\"--num_quantiles\", type=int, default=200)\n parser.add_argument(\"--n_step\", type=int, default=3)\n parser.add_argument(\"--target_update_freq\", type=int, default=320)\n parser.add_argument(\"--epoch\", type=int, default=10)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=10000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=10)\n parser.add_argument(\"--update_per_step\", type=float, default=0.1)\n parser.add_argument(\"--batch_size\", type=int, default=64)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[128, 128, 128, 128])\n parser.add_argument(\"--num_training_envs\", type=int, default=10)\n parser.add_argument(\"--num_test_envs\", type=int, default=100)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\"--prioritized_replay\", action=\"store_true\", default=False)\n parser.add_argument(\"--alpha\", type=float, default=0.6)\n parser.add_argument(\"--beta\", type=float, default=0.4)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n return parser.parse_known_args()[0]\n\n\ndef test_qrdqn(args: argparse.Namespace = get_args(), enable_assertions: bool = True) -> None:\n env = gym.make(args.task)\n assert isinstance(env.action_space, gym.spaces.Discrete)\n\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n\n if args.task == \"CartPole-v1\" and env.spec:\n env.spec.reward_threshold = 190 # lower the goal\n if args.reward_threshold is None:\n default_reward_threshold = {\"CartPole-v1\": 195}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n # training_envs = gym.make(args.task)\n # you can also use tianshou.env.SubprocVectorEnv\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n # test_envs = gym.make(args.task)\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n # model\n net = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n softmax=False,\n num_atoms=args.num_quantiles,\n )\n optim = AdamOptimizerFactory(lr=args.lr)\n policy = QRDQNPolicy(\n model=net,\n action_space=env.action_space,\n observation_space=env.observation_space,\n eps_training=args.eps_train,\n eps_inference=args.eps_test,\n )\n algorithm: QRDQN = QRDQN(\n policy=policy,\n optim=optim,\n gamma=args.gamma,\n num_quantiles=args.num_quantiles,\n n_step_return_horizon=args.n_step,\n target_update_freq=args.target_update_freq,\n ).to(args.device)\n\n # buffer\n buf: PrioritizedVectorReplayBuffer | VectorReplayBuffer\n if args.prioritized_replay:\n buf = PrioritizedVectorReplayBuffer(\n args.buffer_size,\n buffer_num=len(training_envs),\n alpha=args.alpha,\n beta=args.beta,\n )\n else:\n buf = VectorReplayBuffer(args.buffer_size, buffer_num=len(training_envs))\n\n # collectors\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buf, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n\n # initial data collection\n with policy_within_training_step(policy):\n training_collector.reset()\n training_collector.collect(n_step=args.batch_size * args.num_training_envs)\n\n # logger\n log_path = os.path.join(args.logdir, args.task, \"qrdqn\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(algo: Algorithm) -> None:\n torch.save(algo.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n def train_fn(epoch: int, env_step: int) -> None:\n # eps annnealing, just a demo\n if env_step <= 10000:\n policy.set_eps_training(args.eps_train)\n elif env_step <= 50000:\n eps = args.eps_train - (env_step - 10000) / 40000 * (0.9 * args.eps_train)\n policy.set_eps_training(eps)\n else:\n policy.set_eps_training(0.1 * args.eps_train)\n\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n training_fn=train_fn,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n update_step_num_gradient_steps_per_sample=args.update_per_step,\n test_in_training=True,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_pqrdqn(args: argparse.Namespace = get_args()) -> None:\n args.prioritized_replay = True\n args.gamma = 0.95\n test_qrdqn(args)\n\n\ndef test_qrdqn_determinism() -> None:\n main_fn = lambda args: test_qrdqn(args, enable_assertions=False)\n AlgorithmDeterminismTest(\"discrete_qrdqn\", main_fn, get_args()).run()\n" }, { "path": "test/discrete/test_rainbow.py", "content": "import argparse\nimport os\nimport pickle\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom tianshou.algorithm import RainbowDQN\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.c51 import C51Policy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import (\n Collector,\n CollectStats,\n PrioritizedVectorReplayBuffer,\n VectorReplayBuffer,\n)\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.discrete import NoisyLinear\nfrom tianshou.utils.space_info import SpaceInfo\nfrom tianshou.utils.torch_utils import policy_within_training_step\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"CartPole-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=1626)\n parser.add_argument(\"--eps_test\", type=float, default=0.05)\n parser.add_argument(\"--eps_train\", type=float, default=0.1)\n parser.add_argument(\"--buffer_size\", type=int, default=20000)\n parser.add_argument(\"--lr\", type=float, default=1e-3)\n parser.add_argument(\"--gamma\", type=float, default=0.9)\n parser.add_argument(\"--num_atoms\", type=int, default=51)\n parser.add_argument(\"--v_min\", type=float, default=-10.0)\n parser.add_argument(\"--v_max\", type=float, default=10.0)\n parser.add_argument(\"--noisy_std\", type=float, default=0.1)\n parser.add_argument(\"--n_step\", type=int, default=3)\n parser.add_argument(\"--target_update_freq\", type=int, default=320)\n parser.add_argument(\"--epoch\", type=int, default=10)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=8000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=8)\n parser.add_argument(\"--update_per_step\", type=float, default=0.125)\n parser.add_argument(\"--batch_size\", type=int, default=64)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[128, 128, 128, 128])\n parser.add_argument(\"--num_training_envs\", type=int, default=8)\n parser.add_argument(\"--num_test_envs\", type=int, default=100)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\"--prioritized_replay\", action=\"store_true\", default=False)\n parser.add_argument(\"--alpha\", type=float, default=0.6)\n parser.add_argument(\"--beta\", type=float, default=0.4)\n parser.add_argument(\"--beta_final\", type=float, default=1.0)\n parser.add_argument(\"--resume\", action=\"store_true\")\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n parser.add_argument(\"--save_interval\", type=int, default=4)\n return parser.parse_known_args()[0]\n\n\ndef test_rainbow(args: argparse.Namespace = get_args(), enable_assertions: bool = True) -> None:\n env = gym.make(args.task)\n assert isinstance(env.action_space, gym.spaces.Discrete)\n\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n\n if args.reward_threshold is None:\n default_reward_threshold = {\"CartPole-v1\": 195}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n # training_envs = gym.make(args.task)\n # you can also use tianshou.env.SubprocVectorEnv\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n # test_envs = gym.make(args.task)\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n\n def noisy_linear(x: int, y: int) -> NoisyLinear:\n return NoisyLinear(x, y, args.noisy_std)\n\n # model\n net = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n softmax=True,\n num_atoms=args.num_atoms,\n dueling_param=({\"linear_layer\": noisy_linear}, {\"linear_layer\": noisy_linear}),\n )\n optim = AdamOptimizerFactory(lr=args.lr)\n policy = C51Policy(\n model=net,\n action_space=env.action_space,\n num_atoms=args.num_atoms,\n v_min=args.v_min,\n v_max=args.v_max,\n eps_training=args.eps_train,\n eps_inference=args.eps_test,\n )\n algorithm: RainbowDQN = RainbowDQN(\n policy=policy,\n optim=optim,\n gamma=args.gamma,\n n_step_return_horizon=args.n_step,\n target_update_freq=args.target_update_freq,\n ).to(args.device)\n\n # buffer\n buf: PrioritizedVectorReplayBuffer | VectorReplayBuffer\n if args.prioritized_replay:\n buf = PrioritizedVectorReplayBuffer(\n args.buffer_size,\n buffer_num=len(training_envs),\n alpha=args.alpha,\n beta=args.beta,\n weight_norm=True,\n )\n else:\n buf = VectorReplayBuffer(args.buffer_size, buffer_num=len(training_envs))\n\n # collectors\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buf, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n\n # initial data collection\n with policy_within_training_step(policy):\n training_collector.reset()\n training_collector.collect(n_step=args.batch_size * args.num_training_envs)\n\n # logger\n log_path = os.path.join(args.logdir, args.task, \"rainbow\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer, save_interval=args.save_interval)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n def train_fn(epoch: int, env_step: int) -> None:\n # eps annealing, just a demo\n if env_step <= 10000:\n policy.set_eps_training(args.eps_train)\n elif env_step <= 50000:\n eps = args.eps_train - (env_step - 10000) / 40000 * (0.9 * args.eps_train)\n policy.set_eps_training(eps)\n else:\n policy.set_eps_training(0.1 * args.eps_train)\n # beta annealing, just a demo\n if args.prioritized_replay:\n if env_step <= 10000:\n beta = args.beta\n elif env_step <= 50000:\n beta = args.beta - (env_step - 10000) / 40000 * (args.beta - args.beta_final)\n else:\n beta = args.beta_final\n buf.set_beta(beta)\n\n def save_checkpoint_fn(epoch: int, env_step: int, gradient_step: int) -> str:\n # see also: https://pytorch.org/tutorials/beginner/saving_loading_models.html\n ckpt_path = os.path.join(log_path, \"checkpoint.pth\")\n # Example: saving by epoch num\n # ckpt_path = os.path.join(log_path, f\"checkpoint_{epoch}.pth\")\n torch.save(\n algorithm.state_dict(),\n ckpt_path,\n )\n buffer_path = os.path.join(log_path, \"train_buffer.pkl\")\n with open(buffer_path, \"wb\") as f:\n pickle.dump(training_collector.buffer, f)\n return ckpt_path\n\n if args.resume:\n # load from existing checkpoint\n print(f\"Loading agent under {log_path}\")\n ckpt_path = os.path.join(log_path, \"checkpoint.pth\")\n if os.path.exists(ckpt_path):\n checkpoint = torch.load(ckpt_path, map_location=args.device)\n algorithm.load_state_dict(checkpoint)\n print(\"Successfully restore policy and optim.\")\n else:\n print(\"Fail to restore policy and optim.\")\n buffer_path = os.path.join(log_path, \"train_buffer.pkl\")\n if os.path.exists(buffer_path):\n with open(buffer_path, \"rb\") as f:\n training_collector.buffer = pickle.load(f)\n print(\"Successfully restore buffer.\")\n else:\n print(\"Fail to restore buffer.\")\n\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n update_step_num_gradient_steps_per_sample=args.update_per_step,\n training_fn=train_fn,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n resume_from_log=args.resume,\n save_checkpoint_fn=save_checkpoint_fn,\n test_in_training=True,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_rainbow_resume(args: argparse.Namespace = get_args()) -> None:\n args.resume = True\n test_rainbow(args)\n\n\ndef test_prainbow(args: argparse.Namespace = get_args()) -> None:\n args.prioritized_replay = True\n args.gamma = 0.95\n args.seed = 1\n test_rainbow(args)\n\n\ndef test_rainbow_determinism() -> None:\n main_fn = lambda args: test_rainbow(args, enable_assertions=False)\n AlgorithmDeterminismTest(\"discrete_rainbow\", main_fn, get_args()).run()\n" }, { "path": "test/discrete/test_reinforce.py", "content": "import argparse\nimport os\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom gymnasium.spaces import Box\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom tianshou.algorithm import Reinforce\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.reinforce import ProbabilisticActorPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OnPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"CartPole-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=1)\n parser.add_argument(\"--buffer_size\", type=int, default=20000)\n parser.add_argument(\"--lr\", type=float, default=1e-3)\n parser.add_argument(\"--gamma\", type=float, default=0.95)\n parser.add_argument(\"--epoch\", type=int, default=10)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=40000)\n parser.add_argument(\"--collection_step_num_episodes\", type=int, default=8)\n parser.add_argument(\"--update_step_num_repetitions\", type=int, default=2)\n parser.add_argument(\"--batch_size\", type=int, default=64)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[64, 64])\n parser.add_argument(\"--num_training_envs\", type=int, default=8)\n parser.add_argument(\"--num_test_envs\", type=int, default=100)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\"--return_scaling\", type=int, default=1)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n return parser.parse_known_args()[0]\n\n\ndef test_reinforce(args: argparse.Namespace = get_args(), enable_assertions: bool = True) -> None:\n env = gym.make(args.task)\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n if args.reward_threshold is None:\n default_reward_threshold = {\"CartPole-v1\": 195}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n\n # model\n net = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n softmax=True,\n ).to(args.device)\n optim = AdamOptimizerFactory(lr=args.lr)\n dist_fn = torch.distributions.Categorical\n policy = ProbabilisticActorPolicy(\n actor=net,\n dist_fn=dist_fn,\n action_space=env.action_space,\n action_scaling=isinstance(env.action_space, Box),\n )\n algorithm: Reinforce = Reinforce(\n policy=policy,\n optim=optim,\n gamma=args.gamma,\n return_standardization=args.return_scaling,\n )\n for m in net.modules():\n if isinstance(m, torch.nn.Linear):\n # orthogonal initialization\n torch.nn.init.orthogonal_(m.weight, gain=np.sqrt(2))\n torch.nn.init.zeros_(m.bias)\n\n # collector\n training_collector = Collector[CollectStats](\n algorithm,\n training_envs,\n VectorReplayBuffer(args.buffer_size, len(training_envs)),\n )\n test_collector = Collector[CollectStats](algorithm, test_envs)\n\n # log\n log_path = os.path.join(args.logdir, args.task, \"pg\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(algorithm: Algorithm) -> None:\n torch.save(algorithm.policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n # train\n training_config = OnPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n update_step_num_repetitions=args.update_step_num_repetitions,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n collection_step_num_episodes=args.collection_step_num_episodes,\n collection_step_num_env_steps=None,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=True,\n )\n result = algorithm.run_training(training_config)\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_reinforce_determinism() -> None:\n main_fn = lambda args: test_reinforce(args, enable_assertions=False)\n AlgorithmDeterminismTest(\"discrete_reinforce\", main_fn, get_args()).run()\n" }, { "path": "test/highlevel/__init__.py", "content": "" }, { "path": "test/highlevel/env_factory.py", "content": "from tianshou.highlevel.env import (\n EnvFactoryRegistered,\n VectorEnvType,\n)\n\n\nclass DiscreteTestEnvFactory(EnvFactoryRegistered):\n def __init__(self) -> None:\n super().__init__(\n task=\"CartPole-v1\",\n venv_type=VectorEnvType.DUMMY,\n )\n\n\nclass ContinuousTestEnvFactory(EnvFactoryRegistered):\n def __init__(self) -> None:\n super().__init__(\n task=\"Pendulum-v1\",\n venv_type=VectorEnvType.DUMMY,\n )\n" }, { "path": "test/highlevel/test_experiment_builder.py", "content": "import pytest\n\nfrom test.highlevel.env_factory import ContinuousTestEnvFactory, DiscreteTestEnvFactory\nfrom tianshou.highlevel.config import (\n OffPolicyTrainingConfig,\n OnPolicyTrainingConfig,\n)\nfrom tianshou.highlevel.experiment import (\n A2CExperimentBuilder,\n DDPGExperimentBuilder,\n DiscreteSACExperimentBuilder,\n DQNExperimentBuilder,\n ExperimentBuilder,\n ExperimentConfig,\n IQNExperimentBuilder,\n OffPolicyExperimentBuilder,\n OnPolicyExperimentBuilder,\n PPOExperimentBuilder,\n REDQExperimentBuilder,\n ReinforceExperimentBuilder,\n SACExperimentBuilder,\n TD3ExperimentBuilder,\n TRPOExperimentBuilder,\n)\n\n\ndef create_training_config(\n builder_cls: type[ExperimentBuilder],\n num_epochs: int = 1,\n epoch_num_steps: int = 100,\n num_training_envs: int = 2,\n num_test_envs: int = 2,\n) -> OffPolicyTrainingConfig | OnPolicyTrainingConfig:\n if issubclass(builder_cls, OffPolicyExperimentBuilder):\n return OffPolicyTrainingConfig(\n max_epochs=num_epochs,\n epoch_num_steps=epoch_num_steps,\n num_training_envs=num_training_envs,\n num_test_envs=num_test_envs,\n )\n elif issubclass(builder_cls, OnPolicyExperimentBuilder):\n return OnPolicyTrainingConfig(\n max_epochs=num_epochs,\n epoch_num_steps=epoch_num_steps,\n num_training_envs=num_training_envs,\n num_test_envs=num_test_envs,\n )\n else:\n raise ValueError\n\n\n@pytest.mark.parametrize(\n \"builder_cls\",\n [\n PPOExperimentBuilder,\n A2CExperimentBuilder,\n SACExperimentBuilder,\n DDPGExperimentBuilder,\n TD3ExperimentBuilder,\n # NPGExperimentBuilder, # TODO test fails non-deterministically\n REDQExperimentBuilder,\n TRPOExperimentBuilder,\n ReinforceExperimentBuilder,\n ],\n)\ndef test_experiment_builder_continuous_default_params(\n builder_cls: type[ExperimentBuilder],\n) -> None:\n env_factory = ContinuousTestEnvFactory()\n training_config = create_training_config(\n builder_cls,\n num_epochs=1,\n epoch_num_steps=100,\n num_training_envs=2,\n num_test_envs=2,\n )\n experiment_config = ExperimentConfig(persistence_enabled=False)\n builder = builder_cls(\n experiment_config=experiment_config,\n env_factory=env_factory,\n training_config=training_config,\n )\n experiment = builder.build()\n experiment.run(run_name=\"test\")\n print(experiment)\n\n\n@pytest.mark.parametrize(\n \"builder_cls\",\n [\n ReinforceExperimentBuilder,\n PPOExperimentBuilder,\n A2CExperimentBuilder,\n DQNExperimentBuilder,\n DiscreteSACExperimentBuilder,\n IQNExperimentBuilder,\n ],\n)\ndef test_experiment_builder_discrete_default_params(\n builder_cls: type[ExperimentBuilder],\n) -> None:\n env_factory = DiscreteTestEnvFactory()\n training_config = create_training_config(\n builder_cls,\n num_epochs=1,\n epoch_num_steps=100,\n num_training_envs=2,\n num_test_envs=2,\n )\n builder = builder_cls(\n experiment_config=ExperimentConfig(persistence_enabled=False),\n env_factory=env_factory,\n training_config=training_config,\n )\n experiment = builder.build()\n experiment.run(run_name=\"test\")\n print(experiment)\n" }, { "path": "test/modelbased/__init__.py", "content": "" }, { "path": "test/modelbased/test_dqn_icm.py", "content": "import argparse\nimport os\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom tianshou.algorithm import DQN, Algorithm, ICMOffPolicyWrapper\nfrom tianshou.algorithm.modelfree.dqn import DiscreteQLearningPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import (\n Collector,\n CollectStats,\n PrioritizedVectorReplayBuffer,\n VectorReplayBuffer,\n)\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import MLP, Net\nfrom tianshou.utils.net.discrete import IntrinsicCuriosityModule\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"CartPole-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=1626)\n parser.add_argument(\"--eps_test\", type=float, default=0.05)\n parser.add_argument(\"--eps_train\", type=float, default=0.1)\n parser.add_argument(\"--buffer_size\", type=int, default=20000)\n parser.add_argument(\"--lr\", type=float, default=1e-3)\n parser.add_argument(\"--gamma\", type=float, default=0.9)\n parser.add_argument(\"--n_step\", type=int, default=3)\n parser.add_argument(\"--target_update_freq\", type=int, default=320)\n parser.add_argument(\"--epoch\", type=int, default=20)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=10000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=10)\n parser.add_argument(\"--update_per_step\", type=float, default=0.1)\n parser.add_argument(\"--batch_size\", type=int, default=64)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[128, 128, 128, 128])\n parser.add_argument(\"--num_training_envs\", type=int, default=10)\n parser.add_argument(\"--num_test_envs\", type=int, default=100)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\"--prioritized_replay\", action=\"store_true\", default=False)\n parser.add_argument(\"--alpha\", type=float, default=0.6)\n parser.add_argument(\"--beta\", type=float, default=0.4)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n parser.add_argument(\n \"--lr_scale\",\n type=float,\n default=1.0,\n help=\"use intrinsic curiosity module with this lr scale\",\n )\n parser.add_argument(\n \"--reward_scale\",\n type=float,\n default=0.01,\n help=\"scaling factor for intrinsic curiosity reward\",\n )\n parser.add_argument(\n \"--forward_loss_weight\",\n type=float,\n default=0.2,\n help=\"weight for the forward model loss in ICM\",\n )\n return parser.parse_known_args()[0]\n\n\ndef test_dqn_icm(args: argparse.Namespace = get_args()) -> None:\n env = gym.make(args.task)\n assert isinstance(env.action_space, gym.spaces.Discrete)\n\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n\n if args.reward_threshold is None:\n default_reward_threshold = {\"CartPole-v1\": 195}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n\n # Q_param = V_param = {\"hidden_sizes\": [128]}\n # model\n net = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n # dueling=(Q_param, V_param),\n ).to(args.device)\n optim = AdamOptimizerFactory(lr=args.lr)\n policy = DiscreteQLearningPolicy(\n model=net,\n action_space=env.action_space,\n eps_training=args.eps_train,\n eps_inference=args.eps_test,\n )\n algorithm: DQN = DQN(\n policy=policy,\n optim=optim,\n gamma=args.gamma,\n n_step_return_horizon=args.n_step,\n target_update_freq=args.target_update_freq,\n )\n\n # ICM wrapper\n feature_dim = args.hidden_sizes[-1]\n obs_dim = space_info.observation_info.obs_dim\n feature_net = MLP(\n input_dim=obs_dim,\n output_dim=feature_dim,\n hidden_sizes=args.hidden_sizes[:-1],\n )\n action_dim = space_info.action_info.action_dim\n icm_net = IntrinsicCuriosityModule(\n feature_net=feature_net,\n feature_dim=feature_dim,\n action_dim=action_dim,\n hidden_sizes=args.hidden_sizes[-1:],\n ).to(args.device)\n icm_optim = AdamOptimizerFactory(lr=args.lr)\n icm_algorithm = ICMOffPolicyWrapper(\n wrapped_algorithm=algorithm,\n model=icm_net,\n optim=icm_optim,\n lr_scale=args.lr_scale,\n reward_scale=args.reward_scale,\n forward_loss_weight=args.forward_loss_weight,\n )\n\n # buffer\n buf: PrioritizedVectorReplayBuffer | VectorReplayBuffer\n if args.prioritized_replay:\n buf = PrioritizedVectorReplayBuffer(\n args.buffer_size,\n buffer_num=len(training_envs),\n alpha=args.alpha,\n beta=args.beta,\n )\n else:\n buf = VectorReplayBuffer(args.buffer_size, buffer_num=len(training_envs))\n\n # collector\n training_collector = Collector[CollectStats](\n icm_algorithm, training_envs, buf, exploration_noise=True\n )\n test_collector = Collector[CollectStats](icm_algorithm, test_envs, exploration_noise=True)\n training_collector.reset()\n training_collector.collect(n_step=args.batch_size * args.num_training_envs)\n\n # log\n log_path = str(os.path.join(args.logdir, args.task, \"dqn_icm\"))\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n def train_fn(epoch: int, env_step: int) -> None:\n # eps annnealing, just a demo\n if env_step <= 10000:\n policy.set_eps_training(args.eps_train)\n elif env_step <= 50000:\n eps = args.eps_train - (env_step - 10000) / 40000 * (0.9 * args.eps_train)\n policy.set_eps_training(eps)\n else:\n policy.set_eps_training(0.1 * args.eps_train)\n\n # train\n result = icm_algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n update_step_num_gradient_steps_per_sample=args.update_per_step,\n training_fn=train_fn,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=True,\n )\n )\n assert stop_fn(result.best_reward)\n" }, { "path": "test/modelbased/test_ppo_icm.py", "content": "import argparse\nimport os\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom gymnasium.spaces import Box\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom tianshou.algorithm import PPO\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelbased.icm import ICMOnPolicyWrapper\nfrom tianshou.algorithm.modelfree.reinforce import ProbabilisticActorPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OnPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import MLP, ActorCritic, Net\nfrom tianshou.utils.net.discrete import (\n DiscreteActor,\n DiscreteCritic,\n IntrinsicCuriosityModule,\n)\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"CartPole-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=1626)\n parser.add_argument(\"--buffer_size\", type=int, default=20000)\n parser.add_argument(\"--lr\", type=float, default=3e-4)\n parser.add_argument(\"--gamma\", type=float, default=0.99)\n parser.add_argument(\"--epoch\", type=int, default=10)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=50000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=2000)\n parser.add_argument(\"--update_step_num_repetitions\", type=int, default=10)\n parser.add_argument(\"--batch_size\", type=int, default=64)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[64, 64])\n parser.add_argument(\"--num_training_envs\", type=int, default=20)\n parser.add_argument(\"--num_test_envs\", type=int, default=100)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n # ppo special\n parser.add_argument(\"--vf_coef\", type=float, default=0.5)\n parser.add_argument(\"--ent_coef\", type=float, default=0.0)\n parser.add_argument(\"--eps_clip\", type=float, default=0.2)\n parser.add_argument(\"--max_grad_norm\", type=float, default=0.5)\n parser.add_argument(\"--gae_lambda\", type=float, default=0.95)\n parser.add_argument(\"--return_scaling\", type=int, default=0)\n parser.add_argument(\"--advantage_normalization\", type=int, default=0)\n parser.add_argument(\"--recompute_adv\", type=int, default=0)\n parser.add_argument(\"--dual_clip\", type=float, default=None)\n parser.add_argument(\"--value_clip\", type=int, default=0)\n parser.add_argument(\n \"--lr_scale\",\n type=float,\n default=1.0,\n help=\"use intrinsic curiosity module with this lr scale\",\n )\n parser.add_argument(\n \"--reward_scale\",\n type=float,\n default=0.01,\n help=\"scaling factor for intrinsic curiosity reward\",\n )\n parser.add_argument(\n \"--forward_loss_weight\",\n type=float,\n default=0.2,\n help=\"weight for the forward model loss in ICM\",\n )\n return parser.parse_known_args()[0]\n\n\ndef test_ppo(args: argparse.Namespace = get_args()) -> None:\n env = gym.make(args.task)\n\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n\n if args.reward_threshold is None:\n default_reward_threshold = {\"CartPole-v1\": 195}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n\n # model\n net = Net(state_shape=args.state_shape, hidden_sizes=args.hidden_sizes)\n actor = DiscreteActor(preprocess_net=net, action_shape=args.action_shape).to(args.device)\n critic = DiscreteCritic(preprocess_net=net).to(args.device)\n actor_critic = ActorCritic(actor, critic)\n\n # orthogonal initialization\n for m in actor_critic.modules():\n if isinstance(m, torch.nn.Linear):\n torch.nn.init.orthogonal_(m.weight)\n torch.nn.init.zeros_(m.bias)\n\n # base algorithm: PPO\n optim = AdamOptimizerFactory(lr=args.lr)\n dist = torch.distributions.Categorical\n policy = ProbabilisticActorPolicy(\n actor=actor,\n dist_fn=dist,\n action_scaling=isinstance(env.action_space, Box),\n action_space=env.action_space,\n deterministic_eval=True,\n )\n algorithm: PPO = PPO(\n policy=policy,\n critic=critic,\n optim=optim,\n gamma=args.gamma,\n max_grad_norm=args.max_grad_norm,\n eps_clip=args.eps_clip,\n vf_coef=args.vf_coef,\n ent_coef=args.ent_coef,\n gae_lambda=args.gae_lambda,\n return_scaling=args.return_scaling,\n dual_clip=args.dual_clip,\n value_clip=args.value_clip,\n advantage_normalization=args.advantage_normalization,\n recompute_advantage=args.recompute_adv,\n )\n\n # ICM wrapper\n feature_dim = args.hidden_sizes[-1]\n feature_net = MLP(\n input_dim=space_info.observation_info.obs_dim,\n output_dim=feature_dim,\n hidden_sizes=args.hidden_sizes[:-1],\n )\n action_dim = space_info.action_info.action_dim\n icm_net = IntrinsicCuriosityModule(\n feature_net=feature_net,\n feature_dim=feature_dim,\n action_dim=action_dim,\n hidden_sizes=args.hidden_sizes[-1:],\n ).to(args.device)\n icm_optim = AdamOptimizerFactory(lr=args.lr)\n icm_algorithm = ICMOnPolicyWrapper(\n wrapped_algorithm=algorithm,\n model=icm_net,\n optim=icm_optim,\n lr_scale=args.lr_scale,\n reward_scale=args.reward_scale,\n forward_loss_weight=args.forward_loss_weight,\n )\n\n # collector\n training_collector = Collector[CollectStats](\n icm_algorithm,\n training_envs,\n VectorReplayBuffer(args.buffer_size, len(training_envs)),\n )\n test_collector = Collector[CollectStats](icm_algorithm, test_envs)\n\n # log\n log_path = os.path.join(args.logdir, args.task, \"ppo_icm\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(alg: Algorithm) -> None:\n torch.save(alg.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n # train\n result = icm_algorithm.run_training(\n OnPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n update_step_num_repetitions=args.update_step_num_repetitions,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n test_in_training=True,\n )\n )\n assert stop_fn(result.best_reward)\n" }, { "path": "test/modelbased/test_psrl.py", "content": "import argparse\nimport os\n\nimport numpy as np\nimport pytest\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom tianshou.algorithm import PSRL\nfrom tianshou.algorithm.modelbased.psrl import PSRLPolicy\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.trainer import OnPolicyTrainerParams\nfrom tianshou.utils import LazyLogger, TensorboardLogger, WandbLogger\n\ntry:\n import envpool\nexcept ImportError:\n envpool = None\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"NChain-v0\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=1)\n parser.add_argument(\"--buffer_size\", type=int, default=50000)\n parser.add_argument(\"--epoch\", type=int, default=5)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=1000)\n parser.add_argument(\"--collection_step_num_episodes\", type=int, default=1)\n parser.add_argument(\"--num_training_envs\", type=int, default=1)\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\"--rew_mean_prior\", type=float, default=0.0)\n parser.add_argument(\"--rew_std_prior\", type=float, default=1.0)\n parser.add_argument(\"--gamma\", type=float, default=0.99)\n parser.add_argument(\"--eps\", type=float, default=0.01)\n parser.add_argument(\"--add_done_loop\", action=\"store_true\", default=False)\n parser.add_argument(\n \"--logger\",\n type=str,\n default=\"none\", # TODO: Change to \"wandb\" once wandb supports Gym >=0.26.0\n choices=[\"wandb\", \"tensorboard\", \"none\"],\n )\n return parser.parse_known_args()[0]\n\n\n@pytest.mark.skipif(\n envpool is None,\n reason=\"EnvPool is not installed. If on linux, please install it (e.g. as poetry extra)\",\n)\ndef test_psrl(args: argparse.Namespace = get_args()) -> None:\n training_envs = env = envpool.make_gymnasium(\n args.task, num_envs=args.num_training_envs, seed=args.seed\n )\n test_envs = envpool.make_gymnasium(args.task, num_envs=args.num_test_envs, seed=args.seed)\n if args.reward_threshold is None:\n default_reward_threshold = {\"NChain-v0\": 3400}\n args.reward_threshold = default_reward_threshold.get(args.task, env.spec.reward_threshold)\n print(\"reward threshold:\", args.reward_threshold)\n args.state_shape = env.observation_space.shape or env.observation_space.n\n args.action_shape = env.action_space.shape or env.action_space.n\n\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n\n # model\n n_action = args.action_shape\n n_state = args.state_shape\n trans_count_prior = np.ones((n_state, n_action, n_state))\n rew_mean_prior = np.full((n_state, n_action), args.rew_mean_prior)\n rew_std_prior = np.full((n_state, n_action), args.rew_std_prior)\n policy = PSRLPolicy(\n trans_count_prior=trans_count_prior,\n rew_mean_prior=rew_mean_prior,\n rew_std_prior=rew_std_prior,\n action_space=env.action_space,\n discount_factor=args.gamma,\n epsilon=args.eps,\n )\n algorithm: PSRL = PSRL(\n policy=policy,\n add_done_loop=args.add_done_loop,\n )\n\n # collector\n training_collector = Collector[CollectStats](\n algorithm,\n training_envs,\n VectorReplayBuffer(args.buffer_size, len(training_envs)),\n exploration_noise=True,\n )\n training_collector.reset()\n test_collector = Collector[CollectStats](algorithm, test_envs)\n test_collector.reset()\n\n # Logger\n log_path = os.path.join(args.logdir, args.task, \"psrl\")\n writer = SummaryWriter(log_path)\n writer.add_text(\"args\", str(args))\n logger: WandbLogger | TensorboardLogger | LazyLogger\n if args.logger == \"wandb\":\n logger = WandbLogger(save_interval=1, project=\"psrl\", name=\"wandb_test\", config=args)\n logger.load(writer)\n elif args.logger == \"tensorboard\":\n logger = TensorboardLogger(writer)\n else:\n logger = LazyLogger()\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n training_collector.collect(n_step=args.buffer_size, random=True)\n\n # train\n result = algorithm.run_training(\n OnPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n update_step_num_repetitions=1,\n test_step_num_episodes=args.num_test_envs,\n batch_size=0,\n collection_step_num_episodes=args.collection_step_num_episodes,\n collection_step_num_env_steps=None,\n stop_fn=stop_fn,\n logger=logger,\n test_in_training=False,\n )\n )\n assert result.best_reward >= args.reward_threshold\n" }, { "path": "test/offline/__init__.py", "content": "" }, { "path": "test/offline/gather_cartpole_data.py", "content": "import argparse\nimport os\nimport pickle\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom tianshou.algorithm import QRDQN\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.qrdqn import QRDQNPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import (\n Collector,\n CollectStats,\n PrioritizedVectorReplayBuffer,\n VectorReplayBuffer,\n)\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef expert_file_name() -> str:\n return os.path.join(os.path.dirname(__file__), \"expert_QRDQN_CartPole-v1.pkl\")\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"CartPole-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=1)\n parser.add_argument(\"--eps_test\", type=float, default=0.05)\n parser.add_argument(\"--eps_train\", type=float, default=0.1)\n parser.add_argument(\"--buffer_size\", type=int, default=20000)\n parser.add_argument(\"--lr\", type=float, default=1e-3)\n parser.add_argument(\"--gamma\", type=float, default=0.9)\n parser.add_argument(\"--num_quantiles\", type=int, default=200)\n parser.add_argument(\"--n_step\", type=int, default=3)\n parser.add_argument(\"--target_update_freq\", type=int, default=320)\n parser.add_argument(\"--epoch\", type=int, default=10)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=10000)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=10)\n parser.add_argument(\"--update_per_step\", type=float, default=0.1)\n parser.add_argument(\"--batch_size\", type=int, default=64)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[128, 128, 128, 128])\n parser.add_argument(\"--num_training_envs\", type=int, default=10)\n parser.add_argument(\"--num_test_envs\", type=int, default=100)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\"--prioritized_replay\", action=\"store_true\", default=False)\n parser.add_argument(\"--alpha\", type=float, default=0.6)\n parser.add_argument(\"--beta\", type=float, default=0.4)\n parser.add_argument(\"--save_buffer_name\", type=str, default=expert_file_name())\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n return parser.parse_known_args()[0]\n\n\ndef gather_data() -> VectorReplayBuffer | PrioritizedVectorReplayBuffer:\n args = get_args()\n env = gym.make(args.task)\n assert isinstance(env.action_space, gym.spaces.Discrete)\n\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n\n if args.reward_threshold is None:\n default_reward_threshold = {\"CartPole-v1\": 190}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n # training_envs = gym.make(args.task)\n # you can also use tianshou.env.SubprocVectorEnv\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n # test_envs = gym.make(args.task)\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n # model\n net = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n softmax=False,\n num_atoms=args.num_quantiles,\n )\n optim = AdamOptimizerFactory(lr=args.lr)\n policy = QRDQNPolicy(\n model=net,\n action_space=env.action_space,\n eps_training=args.eps_train,\n eps_inference=args.eps_test,\n )\n algorithm: QRDQN = QRDQN(\n policy=policy,\n optim=optim,\n gamma=args.gamma,\n num_quantiles=args.num_quantiles,\n n_step_return_horizon=args.n_step,\n target_update_freq=args.target_update_freq,\n ).to(args.device)\n # buffer\n buf: VectorReplayBuffer | PrioritizedVectorReplayBuffer\n if args.prioritized_replay:\n buf = PrioritizedVectorReplayBuffer(\n args.buffer_size,\n buffer_num=len(training_envs),\n alpha=args.alpha,\n beta=args.beta,\n )\n else:\n buf = VectorReplayBuffer(args.buffer_size, buffer_num=len(training_envs))\n # collector\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buf, exploration_noise=True\n )\n training_collector.reset()\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n test_collector.reset()\n training_collector.collect(n_step=args.batch_size * args.num_training_envs)\n # log\n log_path = os.path.join(args.logdir, args.task, \"qrdqn\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n def train_fn(epoch: int, env_step: int) -> None:\n # eps annnealing, just a demo\n if env_step <= 10000:\n policy.set_eps_training(args.eps_train)\n elif env_step <= 50000:\n eps = args.eps_train - (env_step - 10000) / 40000 * (0.9 * args.eps_train)\n policy.set_eps_training(eps)\n else:\n policy.set_eps_training(0.1 * args.eps_train)\n\n # train\n result = algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n training_fn=train_fn,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n update_step_num_gradient_steps_per_sample=args.update_per_step,\n test_in_training=True,\n )\n )\n assert stop_fn(result.best_reward)\n\n # save buffer in pickle format, for imitation learning unittest\n buf = VectorReplayBuffer(args.buffer_size, buffer_num=len(test_envs))\n policy.set_eps_inference(0.2)\n collector = Collector[CollectStats](algorithm, test_envs, buf, exploration_noise=True)\n collector.reset()\n collector_stats = collector.collect(n_step=args.buffer_size)\n if args.save_buffer_name.endswith(\".hdf5\"):\n buf.save_hdf5(args.save_buffer_name)\n else:\n with open(args.save_buffer_name, \"wb\") as f:\n pickle.dump(buf, f)\n print(collector_stats)\n return buf\n" }, { "path": "test/offline/gather_pendulum_data.py", "content": "import argparse\nimport os\nimport pickle\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom tianshou.algorithm import SAC\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.sac import AutoAlpha, SACPolicy, SACTrainingStats\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OffPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.continuous import ContinuousActorProbabilistic, ContinuousCritic\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef expert_file_name() -> str:\n return os.path.join(os.path.dirname(__file__), \"expert_SAC_Pendulum-v1.pkl\")\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"Pendulum-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=0)\n parser.add_argument(\"--buffer_size\", type=int, default=20000)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[128, 128])\n parser.add_argument(\"--actor_lr\", type=float, default=1e-3)\n parser.add_argument(\"--critic_lr\", type=float, default=1e-3)\n parser.add_argument(\"--epoch\", type=int, default=7)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=8000)\n parser.add_argument(\"--batch_size\", type=int, default=256)\n parser.add_argument(\"--num_training_envs\", type=int, default=10)\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=10)\n parser.add_argument(\"--update_per_step\", type=float, default=0.125)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\"--gamma\", default=0.99)\n parser.add_argument(\"--tau\", default=0.005)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n parser.add_argument(\"--resume_path\", type=str, default=None)\n parser.add_argument(\n \"--watch\",\n default=False,\n action=\"store_true\",\n help=\"watch the play of pre-trained policy only\",\n )\n # sac:\n parser.add_argument(\"--alpha\", type=float, default=0.2)\n parser.add_argument(\"--auto_alpha\", type=int, default=1)\n parser.add_argument(\"--alpha_lr\", type=float, default=3e-4)\n parser.add_argument(\"--n_step\", type=int, default=3)\n parser.add_argument(\"--save_buffer_name\", type=str, default=expert_file_name())\n return parser.parse_known_args()[0]\n\n\ndef gather_data() -> VectorReplayBuffer:\n \"\"\"Return expert buffer data.\"\"\"\n args = get_args()\n env = gym.make(args.task)\n\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n\n if args.reward_threshold is None:\n default_reward_threshold = {\"Pendulum-v0\": -250, \"Pendulum-v1\": -250}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n # you can also use tianshou.env.SubprocVectorEnv\n # training_envs = gym.make(args.task)\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n # test_envs = gym.make(args.task)\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n # model\n net = Net(state_shape=args.state_shape, hidden_sizes=args.hidden_sizes)\n actor = ContinuousActorProbabilistic(\n preprocess_net=net,\n action_shape=args.action_shape,\n unbounded=True,\n ).to(args.device)\n actor_optim = AdamOptimizerFactory(lr=args.actor_lr)\n net_c = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n concat=True,\n )\n critic = ContinuousCritic(preprocess_net=net_c).to(args.device)\n critic_optim = AdamOptimizerFactory(lr=args.critic_lr)\n\n action_dim = space_info.action_info.action_dim\n if args.auto_alpha:\n target_entropy = -action_dim\n log_alpha = 0.0\n alpha_optim = AdamOptimizerFactory(lr=args.alpha_lr)\n args.alpha = AutoAlpha(target_entropy, log_alpha, alpha_optim).to(args.device)\n\n policy = SACPolicy(\n actor=actor,\n action_space=env.action_space,\n )\n algorithm: SAC[SACTrainingStats] = SAC(\n policy=policy,\n policy_optim=actor_optim,\n critic=critic,\n critic_optim=critic_optim,\n tau=args.tau,\n gamma=args.gamma,\n alpha=args.alpha,\n n_step_return_horizon=args.n_step,\n )\n # collector\n buffer = VectorReplayBuffer(args.buffer_size, len(training_envs))\n training_collector = Collector[CollectStats](\n algorithm, training_envs, buffer, exploration_noise=True\n )\n test_collector = Collector[CollectStats](algorithm, test_envs)\n # training_collector.collect(n_step=args.buffer_size)\n # log\n log_path = os.path.join(args.logdir, args.task, \"sac\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n # trainer\n algorithm.run_training(\n OffPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n collection_step_num_env_steps=args.collection_step_num_env_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n update_step_num_gradient_steps_per_sample=args.update_per_step,\n save_best_fn=save_best_fn,\n stop_fn=stop_fn,\n logger=logger,\n test_in_training=True,\n )\n )\n training_collector.reset()\n collector_stats = training_collector.collect(n_step=args.buffer_size)\n print(collector_stats)\n if args.save_buffer_name.endswith(\".hdf5\"):\n buffer.save_hdf5(args.save_buffer_name)\n else:\n with open(args.save_buffer_name, \"wb\") as f:\n pickle.dump(buffer, f)\n return buffer\n" }, { "path": "test/offline/test_bcq.py", "content": "import argparse\nimport datetime\nimport os\nimport pickle\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom test.offline.gather_pendulum_data import expert_file_name, gather_data\nfrom tianshou.algorithm import BCQ, Algorithm\nfrom tianshou.algorithm.imitation.bcq import BCQPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OfflineTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import MLP, Net\nfrom tianshou.utils.net.continuous import VAE, ContinuousCritic, Perturbation\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"Pendulum-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=0)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[64])\n parser.add_argument(\"--actor_lr\", type=float, default=1e-3)\n parser.add_argument(\"--critic_lr\", type=float, default=1e-3)\n parser.add_argument(\"--epoch\", type=int, default=5)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=500)\n parser.add_argument(\"--batch_size\", type=int, default=32)\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=1 / 35)\n\n parser.add_argument(\"--vae_hidden_sizes\", type=int, nargs=\"*\", default=[32, 32])\n # default to 2 * action_dim\n parser.add_argument(\"--latent_dim\", type=int, default=None)\n parser.add_argument(\"--gamma\", default=0.99)\n parser.add_argument(\"--tau\", default=0.005)\n # Weighting for Clipped Double Q-learning in BCQ\n parser.add_argument(\"--lmbda\", default=0.75)\n # Max perturbation hyper-parameter for BCQ\n parser.add_argument(\"--phi\", default=0.05)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n parser.add_argument(\"--resume_path\", type=str, default=None)\n parser.add_argument(\n \"--watch\",\n default=False,\n action=\"store_true\",\n help=\"watch the play of pre-trained policy only\",\n )\n parser.add_argument(\"--load_buffer_name\", type=str, default=expert_file_name())\n parser.add_argument(\"--show_progress\", action=\"store_true\")\n return parser.parse_known_args()[0]\n\n\ndef test_bcq(args: argparse.Namespace = get_args(), enable_assertions: bool = True) -> None:\n if os.path.exists(args.load_buffer_name) and os.path.isfile(args.load_buffer_name):\n if args.load_buffer_name.endswith(\".hdf5\"):\n buffer = VectorReplayBuffer.load_hdf5(args.load_buffer_name)\n else:\n with open(args.load_buffer_name, \"rb\") as f:\n buffer = pickle.load(f)\n else:\n buffer = gather_data()\n env = gym.make(args.task)\n\n space_info = SpaceInfo.from_env(env)\n\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n args.max_action = space_info.action_info.max_action\n args.state_dim = space_info.observation_info.obs_dim\n args.action_dim = space_info.action_info.action_dim\n\n if args.reward_threshold is None:\n # too low?\n default_reward_threshold = {\"Pendulum-v0\": -1100, \"Pendulum-v1\": -1100}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n\n # test_envs = gym.make(args.task)\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n test_envs.seed(args.seed)\n\n # perturbation network\n net_a = MLP(\n input_dim=args.state_dim + args.action_dim,\n output_dim=args.action_dim,\n hidden_sizes=args.hidden_sizes,\n )\n actor_perturbation = Perturbation(\n preprocess_net=net_a, max_action=args.max_action, phi=args.phi\n ).to(\n args.device,\n )\n actor_optim = AdamOptimizerFactory(lr=args.actor_lr)\n\n net_c = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n concat=True,\n )\n critic = ContinuousCritic(preprocess_net=net_c).to(args.device)\n critic_optim = AdamOptimizerFactory(lr=args.critic_lr)\n\n # vae\n # output_dim = 0, so the last Module in the encoder is ReLU\n vae_encoder = MLP(\n input_dim=args.state_dim + args.action_dim,\n hidden_sizes=args.vae_hidden_sizes,\n )\n if not args.latent_dim:\n args.latent_dim = args.action_dim * 2\n vae_decoder = MLP(\n input_dim=args.state_dim + args.latent_dim,\n output_dim=args.action_dim,\n hidden_sizes=args.vae_hidden_sizes,\n )\n vae = VAE(\n encoder=vae_encoder,\n decoder=vae_decoder,\n hidden_dim=args.vae_hidden_sizes[-1],\n latent_dim=args.latent_dim,\n max_action=args.max_action,\n ).to(args.device)\n vae_optim = AdamOptimizerFactory()\n\n policy = BCQPolicy(\n actor_perturbation=actor_perturbation,\n critic=critic,\n vae=vae,\n action_space=env.action_space,\n )\n algorithm = BCQ(\n policy=policy,\n actor_perturbation_optim=actor_optim,\n critic_optim=critic_optim,\n vae_optim=vae_optim,\n gamma=args.gamma,\n tau=args.tau,\n lmbda=args.lmbda,\n ).to(args.device)\n\n # load a previous policy\n if args.resume_path:\n algorithm.load_state_dict(torch.load(args.resume_path, map_location=args.device))\n print(\"Loaded agent from: \", args.resume_path)\n\n # collector\n # buffer has been gathered\n # training_collector = Collector[CollectStats](policy, training_envs, buffer, exploration_noise=True)\n test_collector = Collector[CollectStats](algorithm, test_envs)\n\n # logger\n t0 = datetime.datetime.now().strftime(\"%m%d_%H%M%S\")\n log_file = f\"seed_{args.seed}_{t0}-{args.task.replace('-', '_')}_bcq\"\n log_path = os.path.join(args.logdir, args.task, \"bcq\", log_file)\n writer = SummaryWriter(log_path)\n writer.add_text(\"args\", str(args))\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n def watch() -> None:\n algorithm.load_state_dict(\n torch.load(os.path.join(log_path, \"policy.pth\"), map_location=torch.device(\"cpu\")),\n )\n collector = Collector[CollectStats](algorithm, env)\n collector.collect(n_episode=1, render=1 / 35)\n\n # train\n result = algorithm.run_training(\n OfflineTrainerParams(\n buffer=buffer,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n save_best_fn=save_best_fn,\n stop_fn=stop_fn,\n logger=logger,\n show_progress=args.show_progress,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_bcq_determinism() -> None:\n main_fn = lambda args: test_bcq(args, enable_assertions=False)\n AlgorithmDeterminismTest(\"offline_bcq\", main_fn, get_args(), is_offline=True).run()\n" }, { "path": "test/offline/test_cql.py", "content": "import argparse\nimport datetime\nimport os\nimport pickle\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom test.offline.gather_pendulum_data import expert_file_name, gather_data\nfrom tianshou.algorithm import CQL, Algorithm\nfrom tianshou.algorithm.modelfree.sac import AutoAlpha, SACPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OfflineTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.continuous import ContinuousActorProbabilistic, ContinuousCritic\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"Pendulum-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=1)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[64, 64])\n parser.add_argument(\"--actor_lr\", type=float, default=1e-3)\n parser.add_argument(\"--critic_lr\", type=float, default=1e-3)\n parser.add_argument(\"--alpha\", type=float, default=0.2)\n parser.add_argument(\"--auto_alpha\", default=True, action=\"store_true\")\n parser.add_argument(\"--alpha_lr\", type=float, default=1e-3)\n parser.add_argument(\"--cql_alpha_lr\", type=float, default=1e-3)\n parser.add_argument(\"--start_timesteps\", type=int, default=10000)\n parser.add_argument(\"--epoch\", type=int, default=5)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=500)\n parser.add_argument(\"--n_step\", type=int, default=3)\n parser.add_argument(\"--batch_size\", type=int, default=64)\n\n parser.add_argument(\"--tau\", type=float, default=0.005)\n parser.add_argument(\"--temperature\", type=float, default=1.0)\n parser.add_argument(\"--cql_weight\", type=float, default=1.0)\n parser.add_argument(\"--with_lagrange\", type=bool, default=True)\n parser.add_argument(\"--lagrange_threshold\", type=float, default=10.0)\n parser.add_argument(\"--gamma\", type=float, default=0.99)\n\n parser.add_argument(\"--eval_freq\", type=int, default=1)\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=1 / 35)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n parser.add_argument(\"--resume_path\", type=str, default=None)\n parser.add_argument(\n \"--watch\",\n default=False,\n action=\"store_true\",\n help=\"watch the play of pre-trained policy only\",\n )\n parser.add_argument(\"--load_buffer_name\", type=str, default=expert_file_name())\n return parser.parse_known_args()[0]\n\n\ndef test_cql(args: argparse.Namespace = get_args(), enable_assertions: bool = True) -> None:\n if os.path.exists(args.load_buffer_name) and os.path.isfile(args.load_buffer_name):\n if args.load_buffer_name.endswith(\".hdf5\"):\n buffer = VectorReplayBuffer.load_hdf5(args.load_buffer_name)\n else:\n with open(args.load_buffer_name, \"rb\") as f:\n buffer = pickle.load(f)\n else:\n buffer = gather_data()\n env = gym.make(args.task)\n assert isinstance(env.action_space, gym.spaces.Box)\n\n space_info = SpaceInfo.from_env(env)\n\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n args.min_action = space_info.action_info.min_action\n args.max_action = space_info.action_info.max_action\n args.state_dim = space_info.observation_info.obs_dim\n args.action_dim = space_info.action_info.action_dim\n\n if args.reward_threshold is None:\n # too low?\n default_reward_threshold = {\"Pendulum-v0\": -1200, \"Pendulum-v1\": -1200}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n\n # test_envs = gym.make(args.task)\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n test_envs.seed(args.seed)\n\n # model\n # actor network\n net_a = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n )\n actor = ContinuousActorProbabilistic(\n preprocess_net=net_a,\n action_shape=args.action_shape,\n unbounded=True,\n conditioned_sigma=True,\n ).to(args.device)\n actor_optim = AdamOptimizerFactory(lr=args.actor_lr)\n\n # critic network\n net_c = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n concat=True,\n )\n critic = ContinuousCritic(preprocess_net=net_c).to(args.device)\n critic_optim = AdamOptimizerFactory(lr=args.critic_lr)\n\n if args.auto_alpha:\n target_entropy = float(-np.prod(args.action_shape))\n log_alpha = 0.0\n alpha_optim = AdamOptimizerFactory(lr=args.alpha_lr)\n args.alpha = AutoAlpha(target_entropy, log_alpha, alpha_optim)\n\n policy = SACPolicy(\n actor=actor,\n # CQL seems to perform better without action scaling\n # TODO: investigate why\n action_scaling=False,\n action_space=env.action_space,\n )\n algorithm = CQL(\n policy=policy,\n policy_optim=actor_optim,\n critic=critic,\n critic_optim=critic_optim,\n cql_alpha_lr=args.cql_alpha_lr,\n cql_weight=args.cql_weight,\n tau=args.tau,\n gamma=args.gamma,\n alpha=args.alpha,\n temperature=args.temperature,\n with_lagrange=args.with_lagrange,\n lagrange_threshold=args.lagrange_threshold,\n min_action=args.min_action,\n max_action=args.max_action,\n )\n\n # load a previous policy\n if args.resume_path:\n algorithm.load_state_dict(torch.load(args.resume_path, map_location=args.device))\n print(\"Loaded agent from: \", args.resume_path)\n\n # collector\n # buffer has been gathered\n # training_collector = Collector[CollectStats](policy, training_envs, buffer, exploration_noise=True)\n test_collector = Collector[CollectStats](algorithm, test_envs)\n # log\n t0 = datetime.datetime.now().strftime(\"%m%d_%H%M%S\")\n log_file = f\"seed_{args.seed}_{t0}-{args.task.replace('-', '_')}_cql\"\n log_path = os.path.join(args.logdir, args.task, \"cql\", log_file)\n writer = SummaryWriter(log_path)\n writer.add_text(\"args\", str(args))\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n # trainer\n result = algorithm.run_training(\n OfflineTrainerParams(\n buffer=buffer,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n save_best_fn=save_best_fn,\n stop_fn=stop_fn,\n logger=logger,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_cql_determinism() -> None:\n main_fn = lambda args: test_cql(args, enable_assertions=False)\n AlgorithmDeterminismTest(\"offline_cql\", main_fn, get_args(), is_offline=True).run()\n" }, { "path": "test/offline/test_discrete_bcq.py", "content": "import argparse\nimport os\nimport pickle\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom test.offline.gather_cartpole_data import expert_file_name, gather_data\nfrom tianshou.algorithm import Algorithm, DiscreteBCQ\nfrom tianshou.algorithm.imitation.discrete_bcq import DiscreteBCQPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import (\n Collector,\n CollectStats,\n PrioritizedVectorReplayBuffer,\n VectorReplayBuffer,\n)\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OfflineTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.discrete import DiscreteActor\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"CartPole-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=1626)\n parser.add_argument(\"--eps_test\", type=float, default=0.001)\n parser.add_argument(\"--lr\", type=float, default=3e-4)\n parser.add_argument(\"--gamma\", type=float, default=0.99)\n parser.add_argument(\"--n_step\", type=int, default=3)\n parser.add_argument(\"--target_update_freq\", type=int, default=320)\n parser.add_argument(\"--unlikely_action_threshold\", type=float, default=0.6)\n parser.add_argument(\"--imitation_logits_penalty\", type=float, default=0.01)\n parser.add_argument(\"--epoch\", type=int, default=5)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=2000)\n parser.add_argument(\"--batch_size\", type=int, default=64)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[64, 64])\n parser.add_argument(\"--num_test_envs\", type=int, default=100)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\"--load_buffer_name\", type=str, default=expert_file_name())\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n parser.add_argument(\"--resume\", action=\"store_true\")\n parser.add_argument(\"--save_interval\", type=int, default=4)\n return parser.parse_known_args()[0]\n\n\ndef test_discrete_bcq(\n args: argparse.Namespace = get_args(),\n enable_assertions: bool = True,\n) -> None:\n # envs\n env = gym.make(args.task)\n assert isinstance(env.action_space, gym.spaces.Discrete)\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n if args.reward_threshold is None:\n default_reward_threshold = {\"CartPole-v1\": 185}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n test_envs.seed(args.seed)\n\n # model\n net = Net(state_shape=args.state_shape, action_shape=args.hidden_sizes[0])\n policy_net = DiscreteActor(\n preprocess_net=net,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n ).to(args.device)\n imitation_net = DiscreteActor(\n preprocess_net=net,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n ).to(args.device)\n optim = AdamOptimizerFactory(lr=args.lr)\n policy = DiscreteBCQPolicy(\n model=policy_net,\n imitator=imitation_net,\n action_space=env.action_space,\n unlikely_action_threshold=args.unlikely_action_threshold,\n eps_inference=args.eps_test,\n )\n algorithm: DiscreteBCQ = DiscreteBCQ(\n policy=policy,\n optim=optim,\n gamma=args.gamma,\n n_step_return_horizon=args.n_step,\n target_update_freq=args.target_update_freq,\n imitation_logits_penalty=args.imitation_logits_penalty,\n )\n\n # buffer\n buffer: VectorReplayBuffer | PrioritizedVectorReplayBuffer\n if os.path.exists(args.load_buffer_name) and os.path.isfile(args.load_buffer_name):\n if args.load_buffer_name.endswith(\".hdf5\"):\n buffer = VectorReplayBuffer.load_hdf5(args.load_buffer_name)\n else:\n with open(args.load_buffer_name, \"rb\") as f:\n buffer = pickle.load(f)\n else:\n buffer = gather_data()\n\n # collector\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n\n # logger\n log_path = os.path.join(args.logdir, args.task, \"discrete_bcq\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer, save_interval=args.save_interval)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n def save_checkpoint_fn(epoch: int, env_step: int, gradient_step: int) -> str:\n # see also: https://pytorch.org/tutorials/beginner/saving_loading_models.html\n ckpt_path = os.path.join(log_path, \"checkpoint.pth\")\n # Example: saving by epoch num\n # ckpt_path = os.path.join(log_path, f\"checkpoint_{epoch}.pth\")\n torch.save(\n algorithm.state_dict(),\n ckpt_path,\n )\n return ckpt_path\n\n if args.resume:\n # load from existing checkpoint\n print(f\"Loading agent under {log_path}\")\n ckpt_path = os.path.join(log_path, \"checkpoint.pth\")\n if os.path.exists(ckpt_path):\n checkpoint = torch.load(ckpt_path, map_location=args.device)\n algorithm.load_state_dict(checkpoint)\n print(\"Successfully restore policy and optim.\")\n else:\n print(\"Fail to restore policy and optim.\")\n\n # train\n result = algorithm.run_training(\n OfflineTrainerParams(\n buffer=buffer,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n resume_from_log=args.resume,\n save_checkpoint_fn=save_checkpoint_fn,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_discrete_bcq_resume(args: argparse.Namespace = get_args()) -> None:\n test_discrete_bcq()\n args.resume = True\n test_discrete_bcq(args)\n\n\ndef test_discrete_bcq_determinism() -> None:\n main_fn = lambda args: test_discrete_bcq(args, enable_assertions=False)\n AlgorithmDeterminismTest(\"offline_discrete_bcq\", main_fn, get_args(), is_offline=True).run()\n" }, { "path": "test/offline/test_discrete_cql.py", "content": "import argparse\nimport os\nimport pickle\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom test.offline.gather_cartpole_data import expert_file_name, gather_data\nfrom tianshou.algorithm import Algorithm, DiscreteCQL\nfrom tianshou.algorithm.modelfree.qrdqn import QRDQNPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import (\n Collector,\n CollectStats,\n PrioritizedVectorReplayBuffer,\n VectorReplayBuffer,\n)\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OfflineTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"CartPole-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=1626)\n parser.add_argument(\"--eps_test\", type=float, default=0.001)\n parser.add_argument(\"--lr\", type=float, default=3e-3)\n parser.add_argument(\"--gamma\", type=float, default=0.99)\n parser.add_argument(\"--num_quantiles\", type=int, default=200)\n parser.add_argument(\"--n_step\", type=int, default=3)\n parser.add_argument(\"--target_update_freq\", type=int, default=500)\n parser.add_argument(\"--min_q_weight\", type=float, default=10.0)\n parser.add_argument(\"--epoch\", type=int, default=5)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=1000)\n parser.add_argument(\"--batch_size\", type=int, default=32)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[64])\n parser.add_argument(\"--num_test_envs\", type=int, default=100)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\"--load_buffer_name\", type=str, default=expert_file_name())\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n return parser.parse_known_args()[0]\n\n\ndef test_discrete_cql(\n args: argparse.Namespace = get_args(),\n enable_assertions: bool = True,\n) -> None:\n # envs\n env = gym.make(args.task)\n assert isinstance(env.action_space, gym.spaces.Discrete)\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n if args.reward_threshold is None:\n default_reward_threshold = {\"CartPole-v1\": 170}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n test_envs.seed(args.seed)\n\n # model\n net = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n softmax=False,\n num_atoms=args.num_quantiles,\n )\n optim = AdamOptimizerFactory(lr=args.lr)\n\n policy = QRDQNPolicy(\n model=net,\n action_space=env.action_space,\n )\n algorithm: DiscreteCQL = DiscreteCQL(\n policy=policy,\n optim=optim,\n gamma=args.gamma,\n num_quantiles=args.num_quantiles,\n n_step_return_horizon=args.n_step,\n target_update_freq=args.target_update_freq,\n min_q_weight=args.min_q_weight,\n ).to(args.device)\n\n # buffer\n buffer: VectorReplayBuffer | PrioritizedVectorReplayBuffer\n if os.path.exists(args.load_buffer_name) and os.path.isfile(args.load_buffer_name):\n if args.load_buffer_name.endswith(\".hdf5\"):\n buffer = VectorReplayBuffer.load_hdf5(args.load_buffer_name)\n else:\n with open(args.load_buffer_name, \"rb\") as f:\n buffer = pickle.load(f)\n else:\n buffer = gather_data()\n\n # collector\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n\n log_path = os.path.join(args.logdir, args.task, \"discrete_cql\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n # train\n result = algorithm.run_training(\n OfflineTrainerParams(\n buffer=buffer,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_discrete_cql_determinism() -> None:\n main_fn = lambda args: test_discrete_cql(args, enable_assertions=False)\n AlgorithmDeterminismTest(\"offline_discrete_cql\", main_fn, get_args(), is_offline=True).run()\n" }, { "path": "test/offline/test_discrete_crr.py", "content": "import argparse\nimport os\nimport pickle\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom test.offline.gather_cartpole_data import expert_file_name, gather_data\nfrom tianshou.algorithm import Algorithm, DiscreteCRR\nfrom tianshou.algorithm.modelfree.reinforce import DiscreteActorPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import (\n Collector,\n CollectStats,\n PrioritizedVectorReplayBuffer,\n VectorReplayBuffer,\n)\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OfflineTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.discrete import DiscreteActor, DiscreteCritic\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"CartPole-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=1626)\n parser.add_argument(\"--lr\", type=float, default=7e-4)\n parser.add_argument(\"--gamma\", type=float, default=0.99)\n parser.add_argument(\"--n_step\", type=int, default=3)\n parser.add_argument(\"--target_update_freq\", type=int, default=320)\n parser.add_argument(\"--epoch\", type=int, default=5)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=1000)\n parser.add_argument(\"--batch_size\", type=int, default=64)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[64, 64])\n parser.add_argument(\"--num_test_envs\", type=int, default=100)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\"--load_buffer_name\", type=str, default=expert_file_name())\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n return parser.parse_known_args()[0]\n\n\ndef test_discrete_crr(\n args: argparse.Namespace = get_args(),\n enable_assertions: bool = True,\n) -> None:\n # envs\n env = gym.make(args.task)\n assert isinstance(env.action_space, gym.spaces.Discrete)\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n if args.reward_threshold is None:\n default_reward_threshold = {\"CartPole-v1\": 180}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n test_envs.seed(args.seed)\n\n # model and algorithm\n net = Net(state_shape=args.state_shape, action_shape=args.hidden_sizes[0])\n actor = DiscreteActor(\n preprocess_net=net,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n softmax_output=False,\n )\n action_dim = space_info.action_info.action_dim\n critic = DiscreteCritic(\n preprocess_net=net,\n hidden_sizes=args.hidden_sizes,\n last_size=action_dim,\n )\n optim = AdamOptimizerFactory(lr=args.lr)\n policy = DiscreteActorPolicy(\n actor=actor,\n action_space=env.action_space,\n )\n algorithm: DiscreteCRR = DiscreteCRR(\n policy=policy,\n critic=critic,\n optim=optim,\n gamma=args.gamma,\n target_update_freq=args.target_update_freq,\n ).to(args.device)\n\n # buffer\n buffer: VectorReplayBuffer | PrioritizedVectorReplayBuffer\n if os.path.exists(args.load_buffer_name) and os.path.isfile(args.load_buffer_name):\n if args.load_buffer_name.endswith(\".hdf5\"):\n buffer = VectorReplayBuffer.load_hdf5(args.load_buffer_name)\n else:\n with open(args.load_buffer_name, \"rb\") as f:\n buffer = pickle.load(f)\n else:\n buffer = gather_data()\n\n # collector\n test_collector = Collector[CollectStats](algorithm, test_envs, exploration_noise=True)\n\n log_path = os.path.join(args.logdir, args.task, \"discrete_crr\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n # train\n result = algorithm.run_training(\n OfflineTrainerParams(\n buffer=buffer,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_discrete_crr_determinism() -> None:\n main_fn = lambda args: test_discrete_crr(args, enable_assertions=False)\n AlgorithmDeterminismTest(\"offline_discrete_crr\", main_fn, get_args(), is_offline=True).run()\n" }, { "path": "test/offline/test_gail.py", "content": "import argparse\nimport os\nimport pickle\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.distributions import Distribution, Independent, Normal\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom test.offline.gather_pendulum_data import expert_file_name, gather_data\nfrom tianshou.algorithm import GAIL, Algorithm\nfrom tianshou.algorithm.modelfree.reinforce import ProbabilisticActorPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.trainer import OnPolicyTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import ActorCritic, Net\nfrom tianshou.utils.net.continuous import ContinuousActorProbabilistic, ContinuousCritic\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"Pendulum-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=1)\n parser.add_argument(\"--buffer_size\", type=int, default=20000)\n parser.add_argument(\"--lr\", type=float, default=1e-3)\n parser.add_argument(\"--disc_lr\", type=float, default=5e-4)\n parser.add_argument(\"--gamma\", type=float, default=0.95)\n parser.add_argument(\"--epoch\", type=int, default=5)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=150000)\n parser.add_argument(\"--collection_step_num_episodes\", type=int, default=16)\n parser.add_argument(\"--update_step_num_repetitions\", type=int, default=2)\n parser.add_argument(\"--disc_update_num\", type=int, default=2)\n parser.add_argument(\"--batch_size\", type=int, default=128)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[64, 64])\n parser.add_argument(\"--num_training_envs\", type=int, default=16)\n parser.add_argument(\"--num_test_envs\", type=int, default=100)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=0.0)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n # ppo special\n parser.add_argument(\"--vf_coef\", type=float, default=0.25)\n parser.add_argument(\"--ent_coef\", type=float, default=0.0)\n parser.add_argument(\"--eps_clip\", type=float, default=0.2)\n parser.add_argument(\"--max_grad_norm\", type=float, default=0.5)\n parser.add_argument(\"--gae_lambda\", type=float, default=0.95)\n parser.add_argument(\"--return_scaling\", type=int, default=1)\n parser.add_argument(\"--dual_clip\", type=float, default=None)\n parser.add_argument(\"--value_clip\", type=int, default=1)\n parser.add_argument(\"--advantage_normalization\", type=int, default=1)\n parser.add_argument(\"--recompute_adv\", type=int, default=0)\n parser.add_argument(\"--resume\", action=\"store_true\")\n parser.add_argument(\"--save_interval\", type=int, default=4)\n parser.add_argument(\"--load_buffer_name\", type=str, default=expert_file_name())\n return parser.parse_known_args()[0]\n\n\ndef test_gail(args: argparse.Namespace = get_args(), enable_assertions: bool = True) -> None:\n if os.path.exists(args.load_buffer_name) and os.path.isfile(args.load_buffer_name):\n if args.load_buffer_name.endswith(\".hdf5\"):\n buffer = VectorReplayBuffer.load_hdf5(args.load_buffer_name)\n else:\n with open(args.load_buffer_name, \"rb\") as f:\n buffer = pickle.load(f)\n else:\n buffer = gather_data()\n env = gym.make(args.task)\n if args.reward_threshold is None:\n default_reward_threshold = {\"Pendulum-v0\": -1100, \"Pendulum-v1\": -1100}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n args.max_action = space_info.action_info.max_action\n training_envs = DummyVectorEnv(\n [lambda: gym.make(args.task) for _ in range(args.num_training_envs)]\n )\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n training_envs.seed(args.seed)\n test_envs.seed(args.seed)\n # model\n net = Net(state_shape=args.state_shape, hidden_sizes=args.hidden_sizes)\n actor = ContinuousActorProbabilistic(\n preprocess_net=net,\n action_shape=args.action_shape,\n max_action=args.max_action,\n ).to(\n args.device,\n )\n critic = ContinuousCritic(\n preprocess_net=Net(state_shape=args.state_shape, hidden_sizes=args.hidden_sizes),\n ).to(args.device)\n actor_critic = ActorCritic(actor, critic)\n # orthogonal initialization\n for m in actor_critic.modules():\n if isinstance(m, torch.nn.Linear):\n torch.nn.init.orthogonal_(m.weight)\n torch.nn.init.zeros_(m.bias)\n optim = AdamOptimizerFactory(lr=args.lr)\n # discriminator\n disc_net = ContinuousCritic(\n preprocess_net=Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n activation=torch.nn.Tanh,\n concat=True,\n ),\n ).to(args.device)\n for m in disc_net.modules():\n if isinstance(m, torch.nn.Linear):\n # orthogonal initialization\n torch.nn.init.orthogonal_(m.weight, gain=np.sqrt(2))\n torch.nn.init.zeros_(m.bias)\n disc_optim = AdamOptimizerFactory(lr=args.disc_lr)\n\n # replace DiagGuassian with Independent(Normal) which is equivalent\n # pass *logits to be consistent with policy.forward\n def dist(loc_scale: tuple[torch.Tensor, torch.Tensor]) -> Distribution:\n loc, scale = loc_scale\n return Independent(Normal(loc, scale), 1)\n\n policy = ProbabilisticActorPolicy(\n actor=actor,\n dist_fn=dist,\n action_space=env.action_space,\n )\n algorithm: GAIL = GAIL(\n policy=policy,\n critic=critic,\n optim=optim,\n expert_buffer=buffer,\n disc_net=disc_net,\n disc_optim=disc_optim,\n disc_update_num=args.disc_update_num,\n gamma=args.gamma,\n max_grad_norm=args.max_grad_norm,\n eps_clip=args.eps_clip,\n vf_coef=args.vf_coef,\n ent_coef=args.ent_coef,\n return_scaling=args.return_scaling,\n advantage_normalization=args.advantage_normalization,\n recompute_advantage=args.recompute_adv,\n dual_clip=args.dual_clip,\n value_clip=args.value_clip,\n gae_lambda=args.gae_lambda,\n )\n # collector\n training_collector = Collector[CollectStats](\n algorithm,\n training_envs,\n VectorReplayBuffer(args.buffer_size, len(training_envs)),\n )\n test_collector = Collector[CollectStats](algorithm, test_envs)\n # log\n log_path = os.path.join(args.logdir, args.task, \"gail\")\n writer = SummaryWriter(log_path)\n logger = TensorboardLogger(writer, save_interval=args.save_interval)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n def save_checkpoint_fn(epoch: int, env_step: int, gradient_step: int) -> str:\n # see also: https://pytorch.org/tutorials/beginner/saving_loading_models.html\n ckpt_path = os.path.join(log_path, \"checkpoint.pth\")\n # Example: saving by epoch num\n # ckpt_path = os.path.join(log_path, f\"checkpoint_{epoch}.pth\")\n torch.save(\n algorithm.state_dict(),\n ckpt_path,\n )\n return ckpt_path\n\n if args.resume:\n # load from existing checkpoint\n print(f\"Loading agent under {log_path}\")\n ckpt_path = os.path.join(log_path, \"checkpoint.pth\")\n if os.path.exists(ckpt_path):\n checkpoint = torch.load(ckpt_path, map_location=args.device)\n algorithm.load_state_dict(checkpoint)\n print(\"Successfully restore policy and optim.\")\n else:\n print(\"Fail to restore policy and optim.\")\n\n # trainer\n result = algorithm.run_training(\n OnPolicyTrainerParams(\n training_collector=training_collector,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n update_step_num_repetitions=args.update_step_num_repetitions,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n collection_step_num_episodes=args.collection_step_num_episodes,\n collection_step_num_env_steps=None,\n stop_fn=stop_fn,\n save_best_fn=save_best_fn,\n logger=logger,\n resume_from_log=args.resume,\n save_checkpoint_fn=save_checkpoint_fn,\n test_in_training=True,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_gail_determinism() -> None:\n main_fn = lambda args: test_gail(args, enable_assertions=False)\n AlgorithmDeterminismTest(\"offline_gail\", main_fn, get_args(), is_offline=True).run()\n" }, { "path": "test/offline/test_td3_bc.py", "content": "import argparse\nimport datetime\nimport os\nimport pickle\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom test.determinism_test import AlgorithmDeterminismTest\nfrom test.offline.gather_pendulum_data import expert_file_name, gather_data\nfrom tianshou.algorithm import TD3BC\nfrom tianshou.algorithm.algorithm_base import Algorithm\nfrom tianshou.algorithm.modelfree.ddpg import ContinuousDeterministicPolicy\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\nfrom tianshou.env import DummyVectorEnv\nfrom tianshou.exploration import GaussianNoise\nfrom tianshou.trainer import OfflineTrainerParams\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.net.common import Net\nfrom tianshou.utils.net.continuous import ContinuousActorDeterministic, ContinuousCritic\nfrom tianshou.utils.space_info import SpaceInfo\n\n\ndef get_args() -> argparse.Namespace:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--task\", type=str, default=\"Pendulum-v1\")\n parser.add_argument(\"--reward_threshold\", type=float, default=None)\n parser.add_argument(\"--seed\", type=int, default=0)\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[64, 64])\n parser.add_argument(\"--actor_lr\", type=float, default=1e-3)\n parser.add_argument(\"--critic_lr\", type=float, default=1e-3)\n parser.add_argument(\"--epoch\", type=int, default=5)\n parser.add_argument(\"--epoch_num_steps\", type=int, default=500)\n parser.add_argument(\"--n_step\", type=int, default=3)\n parser.add_argument(\"--batch_size\", type=int, default=64)\n parser.add_argument(\"--alpha\", type=float, default=2.5)\n parser.add_argument(\"--exploration_noise\", type=float, default=0.1)\n parser.add_argument(\"--policy_noise\", type=float, default=0.2)\n parser.add_argument(\"--noise_clip\", type=float, default=0.5)\n parser.add_argument(\"--update_actor_freq\", type=int, default=2)\n parser.add_argument(\"--tau\", type=float, default=0.005)\n parser.add_argument(\"--gamma\", type=float, default=0.99)\n\n parser.add_argument(\"--eval_freq\", type=int, default=1)\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\n parser.add_argument(\"--render\", type=float, default=1 / 35)\n parser.add_argument(\n \"--device\",\n type=str,\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\n )\n parser.add_argument(\"--resume_path\", type=str, default=None)\n parser.add_argument(\n \"--watch\",\n default=False,\n action=\"store_true\",\n help=\"watch the play of pre-trained policy only\",\n )\n parser.add_argument(\"--load_buffer_name\", type=str, default=expert_file_name())\n return parser.parse_known_args()[0]\n\n\ndef test_td3_bc(args: argparse.Namespace = get_args(), enable_assertions: bool = True) -> None:\n if os.path.exists(args.load_buffer_name) and os.path.isfile(args.load_buffer_name):\n if args.load_buffer_name.endswith(\".hdf5\"):\n buffer = VectorReplayBuffer.load_hdf5(args.load_buffer_name)\n else:\n with open(args.load_buffer_name, \"rb\") as f:\n buffer = pickle.load(f)\n else:\n buffer = gather_data()\n env = gym.make(args.task)\n space_info = SpaceInfo.from_env(env)\n args.state_shape = space_info.observation_info.obs_shape\n args.action_shape = space_info.action_info.action_shape\n args.max_action = space_info.action_info.max_action\n if args.reward_threshold is None:\n # too low?\n default_reward_threshold = {\"Pendulum-v0\": -1200, \"Pendulum-v1\": -1200}\n args.reward_threshold = default_reward_threshold.get(\n args.task,\n env.spec.reward_threshold if env.spec else None,\n )\n\n args.state_dim = space_info.action_info.action_dim\n args.action_dim = space_info.observation_info.obs_dim\n test_envs = DummyVectorEnv([lambda: gym.make(args.task) for _ in range(args.num_test_envs)])\n\n # seed\n np.random.seed(args.seed)\n torch.manual_seed(args.seed)\n test_envs.seed(args.seed)\n\n # actor network\n net_a = Net(\n state_shape=args.state_shape,\n hidden_sizes=args.hidden_sizes,\n )\n actor = ContinuousActorDeterministic(\n preprocess_net=net_a,\n action_shape=args.action_shape,\n max_action=args.max_action,\n ).to(args.device)\n actor_optim = AdamOptimizerFactory(lr=args.actor_lr)\n\n # critic networks\n net_c1 = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n concat=True,\n )\n net_c2 = Net(\n state_shape=args.state_shape,\n action_shape=args.action_shape,\n hidden_sizes=args.hidden_sizes,\n concat=True,\n )\n critic1 = ContinuousCritic(preprocess_net=net_c1).to(args.device)\n critic1_optim = AdamOptimizerFactory(lr=args.critic_lr)\n critic2 = ContinuousCritic(preprocess_net=net_c2).to(args.device)\n critic2_optim = AdamOptimizerFactory(lr=args.critic_lr)\n\n # policy and algorithm\n policy = ContinuousDeterministicPolicy(\n actor=actor,\n action_space=env.action_space,\n exploration_noise=GaussianNoise(sigma=args.exploration_noise),\n )\n algorithm: TD3BC = TD3BC(\n policy=policy,\n policy_optim=actor_optim,\n critic=critic1,\n critic_optim=critic1_optim,\n critic2=critic2,\n critic2_optim=critic2_optim,\n tau=args.tau,\n gamma=args.gamma,\n policy_noise=args.policy_noise,\n update_actor_freq=args.update_actor_freq,\n noise_clip=args.noise_clip,\n alpha=args.alpha,\n n_step_return_horizon=args.n_step,\n )\n\n # load a previous policy\n if args.resume_path:\n algorithm.load_state_dict(torch.load(args.resume_path, map_location=args.device))\n print(\"Loaded agent from: \", args.resume_path)\n\n # collector\n # buffer has been gathered\n # training_collector = Collector[CollectStats](policy, training_envs, buffer, exploration_noise=True)\n test_collector = Collector[CollectStats](algorithm, test_envs)\n\n # logger\n t0 = datetime.datetime.now().strftime(\"%m%d_%H%M%S\")\n log_file = f\"seed_{args.seed}_{t0}-{args.task.replace('-', '_')}_td3_bc\"\n log_path = os.path.join(args.logdir, args.task, \"td3_bc\", log_file)\n writer = SummaryWriter(log_path)\n writer.add_text(\"args\", str(args))\n logger = TensorboardLogger(writer)\n\n def save_best_fn(policy: Algorithm) -> None:\n torch.save(policy.state_dict(), os.path.join(log_path, \"policy.pth\"))\n\n def stop_fn(mean_rewards: float) -> bool:\n return mean_rewards >= args.reward_threshold\n\n # train\n result = algorithm.run_training(\n OfflineTrainerParams(\n buffer=buffer,\n test_collector=test_collector,\n max_epochs=args.epoch,\n epoch_num_steps=args.epoch_num_steps,\n test_step_num_episodes=args.num_test_envs,\n batch_size=args.batch_size,\n save_best_fn=save_best_fn,\n stop_fn=stop_fn,\n logger=logger,\n )\n )\n\n if enable_assertions:\n assert stop_fn(result.best_reward)\n\n\ndef test_td3_bc_determinism() -> None:\n main_fn = lambda args: test_td3_bc(args, enable_assertions=False)\n AlgorithmDeterminismTest(\"offline_td3_bc\", main_fn, get_args(), is_offline=True).run()\n" }, { "path": "test/pettingzoo/pistonball.py", "content": "import argparse\r\nimport os\r\nimport warnings\r\n\r\nimport gymnasium as gym\r\nimport numpy as np\r\nimport torch\r\nfrom pettingzoo.butterfly import pistonball_v6\r\nfrom torch.utils.tensorboard import SummaryWriter\r\n\r\nfrom tianshou.algorithm import DQN, Algorithm, MultiAgentOffPolicyAlgorithm\r\nfrom tianshou.algorithm.algorithm_base import OffPolicyAlgorithm\r\nfrom tianshou.algorithm.modelfree.dqn import DiscreteQLearningPolicy\r\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\r\nfrom tianshou.data import Collector, CollectStats, InfoStats, VectorReplayBuffer\r\nfrom tianshou.env import DummyVectorEnv\r\nfrom tianshou.env.pettingzoo_env import PettingZooEnv\r\nfrom tianshou.trainer import OffPolicyTrainerParams\r\nfrom tianshou.utils import TensorboardLogger\r\nfrom tianshou.utils.net.common import Net\r\n\r\n\r\ndef get_parser() -> argparse.ArgumentParser:\r\n parser = argparse.ArgumentParser()\r\n parser.add_argument(\"--seed\", type=int, default=1626)\r\n parser.add_argument(\"--eps_test\", type=float, default=0.05)\r\n parser.add_argument(\"--eps_train\", type=float, default=0.1)\r\n parser.add_argument(\"--buffer_size\", type=int, default=2000)\r\n parser.add_argument(\"--lr\", type=float, default=1e-3)\r\n parser.add_argument(\r\n \"--gamma\",\r\n type=float,\r\n default=0.9,\r\n help=\"a smaller gamma favors earlier win\",\r\n )\r\n parser.add_argument(\r\n \"--n_pistons\",\r\n type=int,\r\n default=3,\r\n help=\"Number of pistons(agents) in the env\",\r\n )\r\n parser.add_argument(\"--n_step\", type=int, default=100)\r\n parser.add_argument(\"--target_update_freq\", type=int, default=320)\r\n parser.add_argument(\"--epoch\", type=int, default=3)\r\n parser.add_argument(\"--epoch_num_steps\", type=int, default=500)\r\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=10)\r\n parser.add_argument(\"--update_per_step\", type=float, default=0.1)\r\n parser.add_argument(\"--batch_size\", type=int, default=100)\r\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[64, 64])\r\n parser.add_argument(\"--num_training_envs\", type=int, default=10)\r\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\r\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\r\n parser.add_argument(\"--render\", type=float, default=0.0)\r\n\r\n parser.add_argument(\r\n \"--watch\",\r\n default=False,\r\n action=\"store_true\",\r\n help=\"no training, watch the play of pre-trained models\",\r\n )\r\n parser.add_argument(\r\n \"--device\",\r\n type=str,\r\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\r\n )\r\n return parser\r\n\r\n\r\ndef get_args() -> argparse.Namespace:\r\n parser = get_parser()\r\n return parser.parse_known_args()[0]\r\n\r\n\r\ndef get_env(args: argparse.Namespace = get_args()) -> PettingZooEnv:\r\n return PettingZooEnv(pistonball_v6.env(continuous=False, n_pistons=args.n_pistons))\r\n\r\n\r\ndef get_agents(\r\n args: argparse.Namespace = get_args(),\r\n agents: list[OffPolicyAlgorithm] | None = None,\r\n optims: list[torch.optim.Optimizer] | None = None,\r\n) -> tuple[Algorithm, list[torch.optim.Optimizer] | None, list]:\r\n env = get_env()\r\n observation_space = (\r\n env.observation_space[\"observation\"]\r\n if isinstance(env.observation_space, gym.spaces.Dict)\r\n else env.observation_space\r\n )\r\n args.state_shape = observation_space.shape or int(observation_space.n)\r\n args.action_shape = env.action_space.shape or int(env.action_space.n)\r\n\r\n if agents is not None:\r\n algorithms = agents\r\n else:\r\n algorithms = []\r\n optims = []\r\n for _ in range(args.n_pistons):\r\n # model\r\n net = Net(\r\n state_shape=args.state_shape,\r\n action_shape=args.action_shape,\r\n hidden_sizes=args.hidden_sizes,\r\n ).to(args.device)\r\n optim = AdamOptimizerFactory(lr=args.lr)\r\n policy = DiscreteQLearningPolicy(\r\n model=net,\r\n action_space=env.action_space,\r\n eps_training=args.eps_train,\r\n eps_inference=args.eps_test,\r\n )\r\n agent: DQN = DQN(\r\n policy=policy,\r\n optim=optim,\r\n gamma=args.gamma,\r\n n_step_return_horizon=args.n_step,\r\n target_update_freq=args.target_update_freq,\r\n )\r\n algorithms.append(agent)\r\n optims.append(optim)\r\n\r\n ma_algorithm = MultiAgentOffPolicyAlgorithm(algorithms=algorithms, env=env)\r\n return ma_algorithm, optims, env.agents\r\n\r\n\r\ndef train_agent(\r\n args: argparse.Namespace = get_args(),\r\n agents: list[OffPolicyAlgorithm] | None = None,\r\n optims: list[torch.optim.Optimizer] | None = None,\r\n) -> tuple[InfoStats, Algorithm]:\r\n training_envs = DummyVectorEnv([get_env for _ in range(args.num_training_envs)])\r\n test_envs = DummyVectorEnv([get_env for _ in range(args.num_test_envs)])\r\n # seed\r\n np.random.seed(args.seed)\r\n torch.manual_seed(args.seed)\r\n training_envs.seed(args.seed)\r\n test_envs.seed(args.seed)\r\n\r\n marl_algorithm, optim, agents = get_agents(args, agents=agents, optims=optims)\r\n\r\n # collector\r\n training_collector = Collector[CollectStats](\r\n marl_algorithm,\r\n training_envs,\r\n VectorReplayBuffer(args.buffer_size, len(training_envs)),\r\n exploration_noise=True,\r\n )\r\n test_collector = Collector[CollectStats](marl_algorithm, test_envs, exploration_noise=True)\r\n training_collector.reset()\r\n training_collector.collect(n_step=args.batch_size * args.num_training_envs)\r\n # log\r\n log_path = os.path.join(args.logdir, \"pistonball\", \"dqn\")\r\n writer = SummaryWriter(log_path)\r\n writer.add_text(\"args\", str(args))\r\n logger = TensorboardLogger(writer)\r\n\r\n def save_best_fn(policy: Algorithm) -> None:\r\n pass\r\n\r\n def stop_fn(mean_rewards: float) -> bool:\r\n return False\r\n\r\n def reward_metric(rews: np.ndarray) -> np.ndarray:\r\n return rews[:, 0]\r\n\r\n # trainer\r\n result = marl_algorithm.run_training(\r\n OffPolicyTrainerParams(\r\n training_collector=training_collector,\r\n test_collector=test_collector,\r\n max_epochs=args.epoch,\r\n epoch_num_steps=args.epoch_num_steps,\r\n collection_step_num_env_steps=args.collection_step_num_env_steps,\r\n test_step_num_episodes=args.num_test_envs,\r\n batch_size=args.batch_size,\r\n stop_fn=stop_fn,\r\n save_best_fn=save_best_fn,\r\n update_step_num_gradient_steps_per_sample=args.update_per_step,\r\n logger=logger,\r\n test_in_training=False,\r\n multi_agent_return_reduction=reward_metric,\r\n )\r\n )\r\n return result, marl_algorithm\r\n\r\n\r\ndef watch(args: argparse.Namespace = get_args(), policy: Algorithm | None = None) -> None:\r\n env = DummyVectorEnv([get_env])\r\n if not policy:\r\n warnings.warn(\r\n \"watching random agents, as loading pre-trained policies is currently not supported\",\r\n )\r\n policy, _, _ = get_agents(args)\r\n collector = Collector[CollectStats](policy, env, exploration_noise=True)\r\n result = collector.collect(n_episode=1, render=args.render)\r\n result.pprint_asdict()\r\n" }, { "path": "test/pettingzoo/pistonball_continuous.py", "content": "import argparse\r\nimport os\r\nimport warnings\r\nfrom typing import Any\r\n\r\nimport gymnasium as gym\r\nimport numpy as np\r\nimport torch\r\nfrom pettingzoo.butterfly import pistonball_v6\r\nfrom torch import nn\r\nfrom torch.distributions import Distribution, Independent, Normal\r\nfrom torch.utils.tensorboard import SummaryWriter\r\n\r\nfrom tianshou.algorithm import PPO, Algorithm\r\nfrom tianshou.algorithm.algorithm_base import OnPolicyAlgorithm\r\nfrom tianshou.algorithm.modelfree.reinforce import ProbabilisticActorPolicy\r\nfrom tianshou.algorithm.multiagent.marl import MultiAgentOnPolicyAlgorithm\r\nfrom tianshou.algorithm.optim import AdamOptimizerFactory\r\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\r\nfrom tianshou.data.stats import InfoStats\r\nfrom tianshou.env import DummyVectorEnv\r\nfrom tianshou.env.pettingzoo_env import PettingZooEnv\r\nfrom tianshou.trainer import OnPolicyTrainerParams\r\nfrom tianshou.utils import TensorboardLogger\r\nfrom tianshou.utils.net.common import ModuleWithVectorOutput\r\nfrom tianshou.utils.net.continuous import ContinuousActorProbabilistic, ContinuousCritic\r\n\r\n\r\nclass DQNet(ModuleWithVectorOutput):\r\n \"\"\"Reference: Human-level control through deep reinforcement learning.\"\"\"\r\n\r\n def __init__(\r\n self,\r\n c: int,\r\n h: int,\r\n w: int,\r\n device: str | int | torch.device = \"cpu\",\r\n ) -> None:\r\n net = nn.Sequential(\r\n nn.Conv2d(c, 32, kernel_size=8, stride=4),\r\n nn.ReLU(inplace=True),\r\n nn.Conv2d(32, 64, kernel_size=4, stride=2),\r\n nn.ReLU(inplace=True),\r\n nn.Conv2d(64, 64, kernel_size=3, stride=1),\r\n nn.ReLU(inplace=True),\r\n nn.Flatten(),\r\n )\r\n with torch.no_grad():\r\n output_dim = np.prod(net(torch.zeros(1, c, h, w)).shape[1:])\r\n super().__init__(int(output_dim))\r\n self.device = device\r\n self.c = c\r\n self.h = h\r\n self.w = w\r\n self.net = net\r\n\r\n def forward(\r\n self,\r\n x: np.ndarray | torch.Tensor,\r\n state: Any | None = None,\r\n info: dict[str, Any] | None = None,\r\n ) -> tuple[torch.Tensor, Any]:\r\n r\"\"\"Mapping: x -> Q(x, \\*).\"\"\"\r\n if info is None:\r\n info = {}\r\n x = torch.as_tensor(x, device=self.device, dtype=torch.float32)\r\n return self.net(x.reshape(-1, self.c, self.w, self.h)), state\r\n\r\n\r\ndef get_parser() -> argparse.ArgumentParser:\r\n parser = argparse.ArgumentParser()\r\n parser.add_argument(\"--seed\", type=int, default=1626)\r\n parser.add_argument(\"--eps_test\", type=float, default=0.05)\r\n parser.add_argument(\"--eps_train\", type=float, default=0.1)\r\n parser.add_argument(\"--buffer_size\", type=int, default=2000)\r\n parser.add_argument(\"--lr\", type=float, default=1e-3)\r\n parser.add_argument(\r\n \"--gamma\",\r\n type=float,\r\n default=0.9,\r\n help=\"a smaller gamma favors earlier win\",\r\n )\r\n parser.add_argument(\r\n \"--n_pistons\",\r\n type=int,\r\n default=3,\r\n help=\"Number of pistons(agents) in the env\",\r\n )\r\n parser.add_argument(\"--n_step\", type=int, default=100)\r\n parser.add_argument(\"--target_update_freq\", type=int, default=320)\r\n parser.add_argument(\"--epoch\", type=int, default=5)\r\n parser.add_argument(\"--epoch_num_steps\", type=int, default=500)\r\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=10)\r\n parser.add_argument(\"--collection_step_num_episodes\", type=int, default=16)\r\n parser.add_argument(\"--update_step_num_repetitions\", type=int, default=2)\r\n parser.add_argument(\"--update_per_step\", type=float, default=0.1)\r\n parser.add_argument(\"--batch_size\", type=int, default=32)\r\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[64, 64])\r\n parser.add_argument(\"--num_training_envs\", type=int, default=10)\r\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\r\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\r\n\r\n parser.add_argument(\r\n \"--watch\",\r\n default=False,\r\n action=\"store_true\",\r\n help=\"no training, watch the play of pre-trained models\",\r\n )\r\n parser.add_argument(\r\n \"--device\",\r\n type=str,\r\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\r\n )\r\n # ppo special\r\n parser.add_argument(\"--vf_coef\", type=float, default=0.25)\r\n parser.add_argument(\"--ent_coef\", type=float, default=0.0)\r\n parser.add_argument(\"--eps_clip\", type=float, default=0.2)\r\n parser.add_argument(\"--max_grad_norm\", type=float, default=0.5)\r\n parser.add_argument(\"--gae_lambda\", type=float, default=0.95)\r\n parser.add_argument(\"--return_scaling\", type=int, default=1)\r\n parser.add_argument(\"--dual_clip\", type=float, default=None)\r\n parser.add_argument(\"--value_clip\", type=int, default=1)\r\n parser.add_argument(\"--advantage_normalization\", type=int, default=1)\r\n parser.add_argument(\"--recompute_adv\", type=int, default=0)\r\n parser.add_argument(\"--resume\", action=\"store_true\")\r\n parser.add_argument(\"--save_interval\", type=int, default=4)\r\n parser.add_argument(\"--render\", type=float, default=0.0)\r\n\r\n return parser\r\n\r\n\r\ndef get_args() -> argparse.Namespace:\r\n parser = get_parser()\r\n return parser.parse_known_args()[0]\r\n\r\n\r\ndef get_env(args: argparse.Namespace = get_args()) -> PettingZooEnv:\r\n return PettingZooEnv(pistonball_v6.env(continuous=True, n_pistons=args.n_pistons))\r\n\r\n\r\ndef get_agents(\r\n args: argparse.Namespace = get_args(),\r\n agents: list[OnPolicyAlgorithm] | None = None,\r\n optims: list[torch.optim.Optimizer] | None = None,\r\n) -> tuple[Algorithm, list[torch.optim.Optimizer] | None, list]:\r\n env = get_env()\r\n observation_space = (\r\n env.observation_space[\"observation\"]\r\n if isinstance(env.observation_space, gym.spaces.Dict)\r\n else env.observation_space\r\n )\r\n args.state_shape = observation_space.shape or observation_space.n\r\n args.action_shape = env.action_space.shape or env.action_space.n\r\n args.max_action = env.action_space.high[0]\r\n\r\n if agents is not None:\r\n algorithms = agents\r\n else:\r\n algorithms = []\r\n optims = []\r\n for _ in range(args.n_pistons):\r\n # model\r\n net = DQNet(\r\n observation_space.shape[2],\r\n observation_space.shape[1],\r\n observation_space.shape[0],\r\n device=args.device,\r\n ).to(args.device)\r\n\r\n actor = ContinuousActorProbabilistic(\r\n preprocess_net=net,\r\n action_shape=args.action_shape,\r\n max_action=args.max_action,\r\n ).to(args.device)\r\n net2 = DQNet(\r\n observation_space.shape[2],\r\n observation_space.shape[1],\r\n observation_space.shape[0],\r\n device=args.device,\r\n ).to(args.device)\r\n critic = ContinuousCritic(preprocess_net=net2).to(args.device)\r\n for m in set(actor.modules()).union(critic.modules()):\r\n if isinstance(m, torch.nn.Linear):\r\n torch.nn.init.orthogonal_(m.weight)\r\n torch.nn.init.zeros_(m.bias)\r\n optim = AdamOptimizerFactory(lr=args.lr)\r\n\r\n def dist(loc_scale: tuple[torch.Tensor, torch.Tensor]) -> Distribution:\r\n loc, scale = loc_scale\r\n return Independent(Normal(loc, scale), 1)\r\n\r\n policy = ProbabilisticActorPolicy(\r\n actor=actor,\r\n dist_fn=dist,\r\n action_space=env.action_space,\r\n action_scaling=True,\r\n action_bound_method=\"clip\",\r\n )\r\n algorithm: PPO = PPO(\r\n policy=policy,\r\n critic=critic,\r\n optim=optim,\r\n gamma=args.gamma,\r\n max_grad_norm=args.max_grad_norm,\r\n eps_clip=args.eps_clip,\r\n vf_coef=args.vf_coef,\r\n ent_coef=args.ent_coef,\r\n return_scaling=args.return_scaling,\r\n advantage_normalization=args.advantage_normalization,\r\n recompute_advantage=args.recompute_adv,\r\n # dual_clip=args.dual_clip,\r\n # dual clip cause monotonically increasing log_std :)\r\n value_clip=args.value_clip,\r\n gae_lambda=args.gae_lambda,\r\n )\r\n\r\n algorithms.append(algorithm)\r\n optims.append(optim)\r\n\r\n ma_algorithm = MultiAgentOnPolicyAlgorithm(\r\n algorithms=algorithms,\r\n env=env,\r\n )\r\n return ma_algorithm, optims, env.agents\r\n\r\n\r\ndef train_agent(\r\n args: argparse.Namespace = get_args(),\r\n agents: list[OnPolicyAlgorithm] | None = None,\r\n optims: list[torch.optim.Optimizer] | None = None,\r\n) -> tuple[InfoStats, Algorithm]:\r\n training_envs = DummyVectorEnv([get_env for _ in range(args.num_training_envs)])\r\n test_envs = DummyVectorEnv([get_env for _ in range(args.num_test_envs)])\r\n # seed\r\n np.random.seed(args.seed)\r\n torch.manual_seed(args.seed)\r\n training_envs.seed(args.seed)\r\n test_envs.seed(args.seed)\r\n\r\n marl_algorithm, optim, agents = get_agents(args, agents=agents, optims=optims)\r\n\r\n # collector\r\n training_collector = Collector[CollectStats](\r\n marl_algorithm,\r\n training_envs,\r\n VectorReplayBuffer(args.buffer_size, len(training_envs)),\r\n exploration_noise=False, # True\r\n )\r\n test_collector = Collector[CollectStats](marl_algorithm, test_envs)\r\n # training_collector.collect(n_step=args.batch_size * args.num_training_envs, reset_before_collect=True)\r\n # log\r\n log_path = os.path.join(args.logdir, \"pistonball\", \"dqn\")\r\n writer = SummaryWriter(log_path)\r\n writer.add_text(\"args\", str(args))\r\n logger = TensorboardLogger(writer)\r\n\r\n def save_best_fn(policy: Algorithm) -> None:\r\n pass\r\n\r\n def stop_fn(mean_rewards: float) -> bool:\r\n return False\r\n\r\n def reward_metric(rews: np.ndarray) -> np.ndarray:\r\n return rews[:, 0]\r\n\r\n # train\r\n result = marl_algorithm.run_training(\r\n OnPolicyTrainerParams(\r\n training_collector=training_collector,\r\n test_collector=test_collector,\r\n max_epochs=args.epoch,\r\n epoch_num_steps=args.epoch_num_steps,\r\n update_step_num_repetitions=args.update_step_num_repetitions,\r\n test_step_num_episodes=args.num_test_envs,\r\n batch_size=args.batch_size,\r\n collection_step_num_episodes=args.collection_step_num_episodes,\r\n collection_step_num_env_steps=None,\r\n stop_fn=stop_fn,\r\n save_best_fn=save_best_fn,\r\n logger=logger,\r\n resume_from_log=args.resume,\r\n test_in_training=True,\r\n )\r\n )\r\n\r\n return result, marl_algorithm\r\n\r\n\r\ndef watch(args: argparse.Namespace = get_args(), policy: Algorithm | None = None) -> None:\r\n env = DummyVectorEnv([get_env])\r\n if not policy:\r\n warnings.warn(\r\n \"watching random agents, as loading pre-trained policies is currently not supported\",\r\n )\r\n policy, _, _ = get_agents(args)\r\n collector = Collector[CollectStats](policy, env)\r\n collector_result = collector.collect(n_episode=1, render=args.render)\r\n collector_result.pprint_asdict()\r\n" }, { "path": "test/pettingzoo/test_pistonball.py", "content": "import argparse\n\nimport pytest\nfrom pistonball import get_args, train_agent, watch\n\n\n@pytest.mark.skip(reason=\"Performance bound was never tested, no point in running this for now\")\ndef test_piston_ball(args: argparse.Namespace = get_args()) -> None:\n if args.watch:\n watch(args)\n return\n\n train_agent(args)\n # assert result.best_reward >= args.win_rate\n" }, { "path": "test/pettingzoo/test_pistonball_continuous.py", "content": "import argparse\n\nimport pytest\nfrom pistonball_continuous import get_args, train_agent, watch\n\n\n@pytest.mark.skip(reason=\"runtime too long and unstable result\")\ndef test_piston_ball_continuous(args: argparse.Namespace = get_args()) -> None:\n if args.watch:\n watch(args)\n return\n\n result, agent = train_agent(args)\n # assert result.best_reward >= 30.0\n" }, { "path": "test/pettingzoo/test_tic_tac_toe.py", "content": "import argparse\r\n\r\nfrom tic_tac_toe import get_args, train_agent, watch\r\n\r\n\r\ndef test_tic_tac_toe(args: argparse.Namespace = get_args()) -> None:\r\n if args.watch:\r\n watch(args)\r\n return\r\n\r\n result, agent = train_agent(args)\r\n assert result.best_reward >= args.win_rate\r\n" }, { "path": "test/pettingzoo/tic_tac_toe.py", "content": "import argparse\r\nimport os\r\nfrom copy import deepcopy\r\nfrom functools import partial\r\n\r\nimport gymnasium\r\nimport numpy as np\r\nimport torch\r\nfrom pettingzoo.classic import tictactoe_v3\r\nfrom torch.utils.tensorboard import SummaryWriter\r\n\r\nfrom tianshou.algorithm import (\r\n DQN,\r\n Algorithm,\r\n MARLRandomDiscreteMaskedOffPolicyAlgorithm,\r\n MultiAgentOffPolicyAlgorithm,\r\n)\r\nfrom tianshou.algorithm.algorithm_base import OffPolicyAlgorithm\r\nfrom tianshou.algorithm.modelfree.dqn import DiscreteQLearningPolicy\r\nfrom tianshou.algorithm.optim import AdamOptimizerFactory, OptimizerFactory\r\nfrom tianshou.data import Collector, CollectStats, VectorReplayBuffer\r\nfrom tianshou.data.stats import InfoStats\r\nfrom tianshou.env import DummyVectorEnv\r\nfrom tianshou.env.pettingzoo_env import PettingZooEnv\r\nfrom tianshou.trainer import OffPolicyTrainerParams\r\nfrom tianshou.utils import TensorboardLogger\r\nfrom tianshou.utils.net.common import Net\r\n\r\n\r\ndef get_env(render_mode: str | None = None) -> PettingZooEnv:\r\n return PettingZooEnv(tictactoe_v3.env(render_mode=render_mode))\r\n\r\n\r\ndef get_parser() -> argparse.ArgumentParser:\r\n parser = argparse.ArgumentParser()\r\n parser.add_argument(\"--seed\", type=int, default=1626)\r\n parser.add_argument(\"--eps_test\", type=float, default=0.05)\r\n parser.add_argument(\"--eps_train\", type=float, default=0.1)\r\n parser.add_argument(\"--buffer_size\", type=int, default=20000)\r\n parser.add_argument(\"--lr\", type=float, default=1e-4)\r\n parser.add_argument(\r\n \"--gamma\",\r\n type=float,\r\n default=0.9,\r\n help=\"a smaller gamma favors earlier win\",\r\n )\r\n parser.add_argument(\"--n_step\", type=int, default=3)\r\n parser.add_argument(\"--target_update_freq\", type=int, default=320)\r\n parser.add_argument(\"--epoch\", type=int, default=50)\r\n parser.add_argument(\"--epoch_num_steps\", type=int, default=1000)\r\n parser.add_argument(\"--collection_step_num_env_steps\", type=int, default=10)\r\n parser.add_argument(\"--update_per_step\", type=float, default=0.1)\r\n parser.add_argument(\"--batch_size\", type=int, default=64)\r\n parser.add_argument(\"--hidden_sizes\", type=int, nargs=\"*\", default=[128, 128, 128, 128])\r\n parser.add_argument(\"--num_training_envs\", type=int, default=10)\r\n parser.add_argument(\"--num_test_envs\", type=int, default=10)\r\n parser.add_argument(\"--logdir\", type=str, default=\"log\")\r\n parser.add_argument(\"--render\", type=float, default=0.1)\r\n parser.add_argument(\r\n \"--win_rate\",\r\n type=float,\r\n default=0.6,\r\n help=\"the expected winning rate: Optimal policy can get 0.7\",\r\n )\r\n parser.add_argument(\r\n \"--watch\",\r\n default=False,\r\n action=\"store_true\",\r\n help=\"no training, watch the play of pre-trained models\",\r\n )\r\n parser.add_argument(\r\n \"--agent_id\",\r\n type=int,\r\n default=2,\r\n help=\"the learned agent plays as the agent_id-th player. Choices are 1 and 2.\",\r\n )\r\n parser.add_argument(\r\n \"--resume_path\",\r\n type=str,\r\n default=\"\",\r\n help=\"the path of agent pth file for resuming from a pre-trained agent\",\r\n )\r\n parser.add_argument(\r\n \"--opponent_path\",\r\n type=str,\r\n default=\"\",\r\n help=\"the path of opponent agent pth file for resuming from a pre-trained agent\",\r\n )\r\n parser.add_argument(\r\n \"--device\",\r\n type=str,\r\n default=\"cuda\" if torch.cuda.is_available() else \"cpu\",\r\n )\r\n return parser\r\n\r\n\r\ndef get_args() -> argparse.Namespace:\r\n parser = get_parser()\r\n return parser.parse_known_args()[0]\r\n\r\n\r\ndef get_agents(\r\n args: argparse.Namespace = get_args(),\r\n agent_learn: OffPolicyAlgorithm | None = None,\r\n agent_opponent: OffPolicyAlgorithm | None = None,\r\n optim: OptimizerFactory | None = None,\r\n) -> tuple[MultiAgentOffPolicyAlgorithm, torch.optim.Optimizer | None, list]:\r\n env = get_env()\r\n observation_space = (\r\n env.observation_space.spaces[\"observation\"]\r\n if isinstance(env.observation_space, gymnasium.spaces.Dict)\r\n else env.observation_space\r\n )\r\n args.state_shape = observation_space.shape or int(observation_space.n)\r\n args.action_shape = env.action_space.shape or int(env.action_space.n)\r\n if agent_learn is None:\r\n # model\r\n net = Net(\r\n state_shape=args.state_shape,\r\n action_shape=args.action_shape,\r\n hidden_sizes=args.hidden_sizes,\r\n ).to(args.device)\r\n if optim is None:\r\n optim = AdamOptimizerFactory(lr=args.lr)\r\n algorithm = DiscreteQLearningPolicy(\r\n model=net,\r\n action_space=env.action_space,\r\n eps_training=args.eps_train,\r\n eps_inference=args.eps_test,\r\n )\r\n agent_learn = DQN(\r\n policy=algorithm,\r\n optim=optim,\r\n n_step_return_horizon=args.n_step,\r\n gamma=args.gamma,\r\n target_update_freq=args.target_update_freq,\r\n )\r\n if args.resume_path:\r\n agent_learn.load_state_dict(torch.load(args.resume_path))\r\n\r\n if agent_opponent is None:\r\n if args.opponent_path:\r\n agent_opponent = deepcopy(agent_learn)\r\n agent_opponent.load_state_dict(torch.load(args.opponent_path))\r\n else:\r\n agent_opponent = MARLRandomDiscreteMaskedOffPolicyAlgorithm(\r\n action_space=env.action_space\r\n )\r\n\r\n if args.agent_id == 1:\r\n agents = [agent_learn, agent_opponent]\r\n else:\r\n agents = [agent_opponent, agent_learn]\r\n ma_algorithm = MultiAgentOffPolicyAlgorithm(algorithms=agents, env=env)\r\n return ma_algorithm, optim, env.agents\r\n\r\n\r\ndef train_agent(\r\n args: argparse.Namespace = get_args(),\r\n agent_learn: OffPolicyAlgorithm | None = None,\r\n agent_opponent: OffPolicyAlgorithm | None = None,\r\n optim: OptimizerFactory | None = None,\r\n) -> tuple[InfoStats, OffPolicyAlgorithm]:\r\n training_envs = DummyVectorEnv([get_env for _ in range(args.num_training_envs)])\r\n test_envs = DummyVectorEnv([get_env for _ in range(args.num_test_envs)])\r\n # seed\r\n np.random.seed(args.seed)\r\n torch.manual_seed(args.seed)\r\n training_envs.seed(args.seed)\r\n test_envs.seed(args.seed)\r\n\r\n marl_algorithm, optim, agents = get_agents(\r\n args,\r\n agent_learn=agent_learn,\r\n agent_opponent=agent_opponent,\r\n optim=optim,\r\n )\r\n\r\n # collector\r\n training_collector = Collector[CollectStats](\r\n marl_algorithm,\r\n training_envs,\r\n VectorReplayBuffer(args.buffer_size, len(training_envs)),\r\n exploration_noise=True,\r\n )\r\n test_collector = Collector[CollectStats](marl_algorithm, test_envs, exploration_noise=True)\r\n training_collector.reset()\r\n training_collector.collect(n_step=args.batch_size * args.num_training_envs)\r\n # log\r\n log_path = os.path.join(args.logdir, \"tic_tac_toe\", \"dqn\")\r\n writer = SummaryWriter(log_path)\r\n writer.add_text(\"args\", str(args))\r\n logger = TensorboardLogger(writer)\r\n\r\n player_agent_id = agents[args.agent_id - 1]\r\n\r\n def save_best_fn(policy: Algorithm) -> None:\r\n if hasattr(args, \"model_save_path\"):\r\n model_save_path = args.model_save_path\r\n else:\r\n model_save_path = os.path.join(args.logdir, \"tic_tac_toe\", \"dqn\", \"policy.pth\")\r\n torch.save(policy.get_algorithm(player_agent_id).state_dict(), model_save_path)\r\n\r\n def stop_fn(mean_rewards: float) -> bool:\r\n return mean_rewards >= args.win_rate\r\n\r\n def reward_metric(rews: np.ndarray) -> np.ndarray:\r\n return rews[:, args.agent_id - 1]\r\n\r\n # trainer\r\n result = marl_algorithm.run_training(\r\n OffPolicyTrainerParams(\r\n training_collector=training_collector,\r\n test_collector=test_collector,\r\n max_epochs=args.epoch,\r\n epoch_num_steps=args.epoch_num_steps,\r\n collection_step_num_env_steps=args.collection_step_num_env_steps,\r\n test_step_num_episodes=args.num_test_envs,\r\n batch_size=args.batch_size,\r\n stop_fn=stop_fn,\r\n save_best_fn=save_best_fn,\r\n update_step_num_gradient_steps_per_sample=args.update_per_step,\r\n logger=logger,\r\n test_in_training=False,\r\n multi_agent_return_reduction=reward_metric,\r\n )\r\n )\r\n\r\n return result, marl_algorithm.get_algorithm(player_agent_id)\r\n\r\n\r\ndef watch(\r\n args: argparse.Namespace = get_args(),\r\n agent_learn: OffPolicyAlgorithm | None = None,\r\n agent_opponent: OffPolicyAlgorithm | None = None,\r\n) -> None:\r\n env = DummyVectorEnv([partial(get_env, render_mode=\"human\")])\r\n policy, optim, agents = get_agents(args, agent_learn=agent_learn, agent_opponent=agent_opponent)\r\n collector = Collector[CollectStats](policy, env, exploration_noise=True)\r\n result = collector.collect(n_episode=1, render=args.render, reset_before_collect=True)\r\n result.pprint_asdict()\r\n" }, { "path": "tianshou/__init__.py", "content": "# isort: skip_file\n# NOTE: Import order is important to avoid circular import errors!\nfrom tianshou import data, env, exploration, algorithm, trainer, utils\n\n__version__ = \"2.0.0\"\n\n\ndef _register_log_config_callback() -> None:\n from sensai.util import logging\n\n def configure() -> None:\n logging.getLogger(\"numba\").setLevel(logging.INFO)\n\n logging.set_configure_callback(configure)\n\n\n_register_log_config_callback()\n\n\n__all__ = [\n \"algorithm\",\n \"data\",\n \"env\",\n \"exploration\",\n \"trainer\",\n \"utils\",\n]\n" }, { "path": "tianshou/algorithm/__init__.py", "content": "\"\"\"Algorithm package.\"\"\"\n# isort:skip_file\n\nfrom tianshou.algorithm.algorithm_base import Algorithm, TrainingStats\nfrom tianshou.algorithm.modelfree.reinforce import Reinforce\nfrom tianshou.algorithm.modelfree.dqn import DQN\nfrom tianshou.algorithm.modelfree.ddpg import DDPG\n\nfrom tianshou.algorithm.random import MARLRandomDiscreteMaskedOffPolicyAlgorithm\nfrom tianshou.algorithm.modelfree.bdqn import BDQN\nfrom tianshou.algorithm.modelfree.c51 import C51\nfrom tianshou.algorithm.modelfree.rainbow import RainbowDQN\nfrom tianshou.algorithm.modelfree.qrdqn import QRDQN\nfrom tianshou.algorithm.modelfree.iqn import IQN\nfrom tianshou.algorithm.modelfree.fqf import FQF\nfrom tianshou.algorithm.modelfree.a2c import A2C\nfrom tianshou.algorithm.modelfree.npg import NPG\nfrom tianshou.algorithm.modelfree.ppo import PPO\nfrom tianshou.algorithm.modelfree.trpo import TRPO\nfrom tianshou.algorithm.modelfree.td3 import TD3\nfrom tianshou.algorithm.modelfree.sac import SAC\nfrom tianshou.algorithm.modelfree.redq import REDQ\nfrom tianshou.algorithm.modelfree.discrete_sac import DiscreteSAC\nfrom tianshou.algorithm.imitation.imitation_base import OffPolicyImitationLearning\nfrom tianshou.algorithm.imitation.bcq import BCQ\nfrom tianshou.algorithm.imitation.cql import CQL\nfrom tianshou.algorithm.imitation.td3_bc import TD3BC\nfrom tianshou.algorithm.imitation.discrete_bcq import DiscreteBCQ\nfrom tianshou.algorithm.imitation.discrete_cql import DiscreteCQL\nfrom tianshou.algorithm.imitation.discrete_crr import DiscreteCRR\nfrom tianshou.algorithm.imitation.gail import GAIL\nfrom tianshou.algorithm.modelbased.psrl import PSRL\nfrom tianshou.algorithm.modelbased.icm import ICMOffPolicyWrapper\nfrom tianshou.algorithm.modelbased.icm import ICMOnPolicyWrapper\nfrom tianshou.algorithm.multiagent.marl import MultiAgentOffPolicyAlgorithm\n" }, { "path": "tianshou/algorithm/algorithm_base.py", "content": "import logging\nimport time\nfrom abc import ABC, abstractmethod\nfrom collections.abc import Callable, Mapping\nfrom dataclasses import dataclass, field\nfrom typing import TYPE_CHECKING, Any, Generic, Literal, TypeVar, cast\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom gymnasium.spaces import Box, Discrete, MultiBinary, MultiDiscrete\nfrom numba import njit\nfrom numpy.typing import ArrayLike\nfrom overrides import override\nfrom sensai.util.hash import pickle_hash\nfrom sensai.util.helper import mark_used\nfrom torch import nn\nfrom torch.nn.modules.module import (\n _IncompatibleKeys, # we have to do this since we override load_state_dict\n)\nfrom torch.optim.lr_scheduler import LRScheduler\n\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import ReplayBuffer, SequenceSummaryStats, to_numpy, to_torch_as\nfrom tianshou.data.batch import Batch, BatchProtocol, TArr\nfrom tianshou.data.buffer.buffer_base import TBuffer\nfrom tianshou.data.types import (\n ActBatchProtocol,\n ActStateBatchProtocol,\n BatchWithReturnsProtocol,\n ObsBatchProtocol,\n RolloutBatchProtocol,\n)\nfrom tianshou.utils.determinism import TraceLogger\nfrom tianshou.utils.lagged_network import (\n EvalModeModuleWrapper,\n LaggedNetworkCollection,\n)\nfrom tianshou.utils.net.common import RandomActor\nfrom tianshou.utils.print import DataclassPPrintMixin\nfrom tianshou.utils.torch_utils import policy_within_training_step, torch_train_mode\n\nif TYPE_CHECKING:\n from tianshou.data.stats import InfoStats\n from tianshou.trainer import (\n OfflineTrainer,\n OfflineTrainerParams,\n OffPolicyTrainer,\n OffPolicyTrainerParams,\n OnPolicyTrainer,\n OnPolicyTrainerParams,\n Trainer,\n TrainerParams,\n )\n\n mark_used(TrainerParams)\n\nlogger = logging.getLogger(__name__)\n\nTArrOrActBatch = TypeVar(\"TArrOrActBatch\", bound=\"np.ndarray | ActBatchProtocol\")\n\n\n@dataclass(kw_only=True)\nclass TrainingStats(DataclassPPrintMixin):\n _non_loss_fields = (\"train_time\", \"smoothed_loss\")\n\n train_time: float = 0.0\n \"\"\"The time for learning models.\"\"\"\n\n # TODO: modified in the trainer but not used anywhere else. Should be refactored.\n smoothed_loss: dict = field(default_factory=dict)\n \"\"\"The smoothed loss statistics of the policy learn step.\"\"\"\n\n # Mainly so that we can override this in the TrainingStatsWrapper\n def _get_self_dict(self) -> dict[str, Any]:\n return self.__dict__\n\n def get_loss_stats_dict(self) -> dict[str, float]:\n \"\"\"Return loss statistics as a dict for logging.\n\n Returns a dict with all fields except train_time and smoothed_loss. Moreover, fields with value None excluded,\n and instances of SequenceSummaryStats are replaced by their mean.\n \"\"\"\n result = {}\n for k, v in self._get_self_dict().items():\n if k.startswith(\"_\"):\n logger.debug(f\"Skipping {k=} as it starts with an underscore.\")\n continue\n if k in self._non_loss_fields or v is None:\n continue\n if isinstance(v, SequenceSummaryStats):\n result[k] = v.mean\n else:\n result[k] = v\n\n return result\n\n\nclass TrainingStatsWrapper(TrainingStats):\n _setattr_frozen = False\n _training_stats_public_fields = TrainingStats.__dataclass_fields__.keys()\n\n def __init__(self, wrapped_stats: TrainingStats) -> None:\n \"\"\"In this particular case, super().__init__() should be called LAST in the subclass init.\"\"\"\n self._wrapped_stats = wrapped_stats\n\n # HACK: special sauce for the existing attributes of the base TrainingStats class\n # for some reason, delattr doesn't work here, so we need to delegate their handling\n # to the wrapped stats object by always keeping the value there and in self in sync\n # see also __setattr__\n for k in self._training_stats_public_fields:\n super().__setattr__(k, getattr(self._wrapped_stats, k))\n\n self._setattr_frozen = True\n\n @override\n def _get_self_dict(self) -> dict[str, Any]:\n return {**self._wrapped_stats._get_self_dict(), **self.__dict__}\n\n @property\n def wrapped_stats(self) -> TrainingStats:\n return self._wrapped_stats\n\n def __getattr__(self, name: str) -> Any:\n return getattr(self._wrapped_stats, name)\n\n def __setattr__(self, name: str, value: Any) -> None:\n \"\"\"Setattr logic for wrapper of a dataclass with default values.\n\n 1. If name exists directly in self, set it there.\n 2. If it exists in self._wrapped_stats, set it there instead.\n 3. Special case: if name is in the base TrainingStats class, keep it in sync between self and the _wrapped_stats.\n 4. If name doesn't exist in either and attribute setting is frozen, raise an AttributeError.\n \"\"\"\n # HACK: special sauce for the existing attributes of the base TrainingStats class, see init\n # Need to keep them in sync with the wrapped stats object\n if name in self._training_stats_public_fields:\n setattr(self._wrapped_stats, name, value)\n super().__setattr__(name, value)\n return\n\n if not self._setattr_frozen:\n super().__setattr__(name, value)\n return\n\n if not hasattr(self, name):\n raise AttributeError(\n f\"Setting new attributes on StatsWrappers outside of init is not allowed. \"\n f\"Tried to set {name=}, {value=} on {self.__class__.__name__}. \\n\"\n f\"NOTE: you may get this error if you call super().__init__() in your subclass init too early! \"\n f\"The call to super().__init__() should be the last call in your subclass init.\",\n )\n if hasattr(self._wrapped_stats, name):\n setattr(self._wrapped_stats, name, value)\n else:\n super().__setattr__(name, value)\n\n\nclass Policy(nn.Module, ABC):\n \"\"\"Represents a policy, which provides the fundamental mapping from observations to actions.\"\"\"\n\n def __init__(\n self,\n action_space: gym.Space,\n observation_space: gym.Space | None = None,\n action_scaling: bool = False,\n action_bound_method: Literal[\"clip\", \"tanh\"] | None = \"clip\",\n ):\n \"\"\"\n :param action_space: the environment's action_space.\n :param observation_space: the environment's observation space.\n :param action_scaling: flag indicating whether, for continuous action spaces, actions\n should be scaled from the standard neural network output range [-1, 1] to the\n environment's action space range [action_space.low, action_space.high].\n This applies to continuous action spaces only (gym.spaces.Box) and has no effect\n for discrete spaces.\n When enabled, policy outputs are expected to be in the normalized range [-1, 1]\n (after bounding), and are then linearly transformed to the actual required range.\n This improves neural network training stability, allows the same algorithm to work\n across environments with different action ranges, and standardizes exploration\n strategies.\n Should be disabled if the actor model already produces outputs in the correct range.\n :param action_bound_method: the method used for bounding actions in continuous action spaces\n to the range [-1, 1] before scaling them to the environment's action space (provided\n that `action_scaling` is enabled).\n This applies to continuous action spaces only (`gym.spaces.Box`) and should be set to None\n for discrete spaces.\n When set to \"clip\", actions exceeding the [-1, 1] range are simply clipped to this\n range. When set to \"tanh\", a hyperbolic tangent function is applied, which smoothly\n constrains outputs to [-1, 1] while preserving gradients.\n The choice of bounding method affects both training dynamics and exploration behavior.\n Clipping provides hard boundaries but may create plateau regions in the gradient\n landscape, while tanh provides smoother transitions but can compress sensitivity\n near the boundaries.\n Should be set to None if the actor model inherently produces bounded outputs.\n Typically used together with `action_scaling=True`.\n \"\"\"\n allowed_action_bound_methods = (\"clip\", \"tanh\")\n if (\n action_bound_method is not None\n and action_bound_method not in allowed_action_bound_methods\n ):\n raise ValueError(\n f\"Got invalid {action_bound_method=}. \"\n f\"Valid values are: {allowed_action_bound_methods}.\",\n )\n if action_scaling and not isinstance(action_space, Box):\n raise ValueError(\n f\"action_scaling can only be True when action_space is Box but got: {action_space}\",\n )\n super().__init__()\n self.observation_space = observation_space\n self.action_space = action_space\n if isinstance(action_space, Discrete | MultiDiscrete | MultiBinary):\n action_type = \"discrete\"\n elif isinstance(action_space, Box):\n action_type = \"continuous\"\n else:\n raise ValueError(f\"Unsupported action space: {action_space}.\")\n self._action_type = cast(Literal[\"discrete\", \"continuous\"], action_type)\n self.agent_id = 0\n self.action_scaling = action_scaling\n self.action_bound_method = action_bound_method\n self.is_within_training_step = False\n \"\"\"\n flag indicating whether we are currently within a training step,\n which encompasses data collection for training (in online RL algorithms)\n and the policy update (gradient steps).\n\n It can be used, for example, to control whether a flag controlling deterministic evaluation should\n indeed be applied, because within a training step, we typically always want to apply stochastic evaluation\n (even if such a flag is enabled), as well as stochastic action computation for q-targets (e.g. in SAC\n based algorithms).\n\n This flag should normally remain False and should be set to True only by the algorithm which performs\n training steps. This is done automatically by the Trainer classes. If a policy is used outside of a Trainer,\n the user should ensure that this flag is set correctly.\n \"\"\"\n self._compile()\n\n @property\n def action_type(self) -> Literal[\"discrete\", \"continuous\"]:\n return self._action_type\n\n @staticmethod\n def _action_to_numpy(act: TArr) -> np.ndarray:\n act = to_numpy(act) # NOTE: to_numpy could confusingly also return a Batch\n if not isinstance(act, np.ndarray):\n raise ValueError(\n f\"act should have been be a numpy.ndarray, but got {type(act)}.\",\n )\n return act\n\n def map_action(\n self,\n act: TArr,\n ) -> np.ndarray:\n \"\"\"Map raw network output to action range in gym's env.action_space.\n\n This function is called in :meth:`~tianshou.data.Collector.collect` and only\n affects action sending to env. Remapped action will not be stored in buffer\n and thus can be viewed as a part of env (a black box action transformation).\n\n Action mapping includes 2 standard procedures: bounding and scaling. Bounding\n procedure expects original action range is (-inf, inf) and maps it to [-1, 1],\n while scaling procedure expects original action range is (-1, 1) and maps it\n to [action_space.low, action_space.high]. Bounding procedure is applied first.\n\n :param act: a data batch or numpy.ndarray which is the action taken by\n policy.forward.\n\n :return: action in the same form of input \"act\" but remap to the target action\n space.\n \"\"\"\n act = self._action_to_numpy(act)\n if isinstance(self.action_space, gym.spaces.Box):\n if self.action_bound_method == \"clip\":\n act = np.clip(act, -1.0, 1.0)\n elif self.action_bound_method == \"tanh\":\n act = np.tanh(act)\n if self.action_scaling:\n assert np.min(act) >= -1.0 and np.max(act) <= 1.0, (\n f\"action scaling only accepts raw action range = [-1, 1], but got: {act}\"\n )\n low, high = self.action_space.low, self.action_space.high\n act = low + (high - low) * (act + 1.0) / 2.0\n return act\n\n def map_action_inverse(\n self,\n act: TArr,\n ) -> np.ndarray:\n \"\"\"Inverse operation to :meth:`map_action`.\n\n This function is called in :meth:`~tianshou.data.Collector.collect` for\n random initial steps. It scales [action_space.low, action_space.high] to\n the value ranges of policy.forward.\n\n :param act: a data batch, list or numpy.ndarray which is the action taken\n by gym.spaces.Box.sample().\n\n :return: action remapped.\n \"\"\"\n act = self._action_to_numpy(act)\n if isinstance(self.action_space, gym.spaces.Box):\n if self.action_scaling:\n low, high = self.action_space.low, self.action_space.high\n scale = high - low\n eps = np.finfo(np.float32).eps.item()\n scale[scale < eps] += eps\n act = (act - low) * 2.0 / scale - 1.0\n if self.action_bound_method == \"tanh\":\n act = (np.log(1.0 + act) - np.log(1.0 - act)) / 2.0\n\n return act\n\n def compute_action(\n self,\n obs: ArrayLike,\n info: dict[str, Any] | None = None,\n state: dict | BatchProtocol | np.ndarray | None = None,\n ) -> np.ndarray | int:\n \"\"\"Get action as int (for discrete env's) or array (for continuous ones) from an env's observation and info.\n\n :param obs: observation from the gym's env.\n :param info: information given by the gym's env.\n :param state: the hidden state of RNN policy, used for recurrent policy.\n :return: action as int (for discrete env's) or array (for continuous ones).\n \"\"\"\n obs = np.array(obs) # convert array-like to array (e.g. LazyFrames)\n obs = obs[None, :] # add batch dimension\n obs_batch = cast(ObsBatchProtocol, Batch(obs=obs, info=info))\n act = self.forward(obs_batch, state=state).act.squeeze()\n if isinstance(act, torch.Tensor):\n act = act.detach().cpu().numpy()\n act = self.map_action(act)\n if isinstance(self.action_space, Discrete):\n # could be an array of shape (), easier to just convert to int\n act = int(act) # type: ignore\n return act\n\n @staticmethod\n def _compile() -> None:\n f64 = np.array([0, 1], dtype=np.float64)\n f32 = np.array([0, 1], dtype=np.float32)\n b = np.array([False, True], dtype=np.bool_)\n i64 = np.array([[0, 1]], dtype=np.int64)\n _gae(f64, f64, f64, b, 0.1, 0.1)\n _gae(f32, f32, f64, b, 0.1, 0.1)\n _nstep_return(f64, b, f32.reshape(-1, 1), i64, 0.1, 1)\n\n _TArrOrActBatch = TypeVar(\"_TArrOrActBatch\", bound=\"np.ndarray | ActBatchProtocol\")\n\n def add_exploration_noise(\n self,\n act: _TArrOrActBatch,\n batch: ObsBatchProtocol,\n ) -> _TArrOrActBatch:\n \"\"\"(Optionally) adds noise to an actions computed by the policy's forward method for\n exploration purposes.\n\n NOTE: The base implementation does not add any noise, but subclasses can override\n this method to add appropriate mechanisms for adding noise.\n\n :param act: a data batch or numpy.ndarray containing actions computed by the policy's\n forward method.\n :param batch: the corresponding input batch that was passed to forward; provided for\n advanced usage.\n :return: actions in the same format as the input `act` but with added exploration\n noise (if implemented - otherwise returns `act` unchanged).\n \"\"\"\n return act\n\n\nclass LaggedNetworkAlgorithmMixin(ABC):\n \"\"\"\n Base class for an algorithm mixin which adds support for lagged networks (target networks) whose weights\n are updated periodically.\n \"\"\"\n\n def __init__(self) -> None:\n self._lagged_networks = LaggedNetworkCollection()\n\n def _add_lagged_network(self, src: torch.nn.Module) -> EvalModeModuleWrapper:\n \"\"\"\n Adds a lagged network to the collection, returning the target network, which\n is forced to eval mode. The target network is a copy of the source network,\n which, however, supports only the forward method (hence the type torch.nn.Module);\n attribute access is not supported.\n\n :param src: the source network whose parameters are to be copied to the target network\n :return: the target network, which supports only the forward method and is forced to eval mode\n \"\"\"\n return self._lagged_networks.add_lagged_network(src)\n\n @abstractmethod\n def _update_lagged_network_weights(self) -> None:\n pass\n\n\nclass LaggedNetworkFullUpdateAlgorithmMixin(LaggedNetworkAlgorithmMixin):\n \"\"\"\n Algorithm mixin which adds support for lagged networks (target networks) where weights\n are updated by fully copying the weights of the source network to the target network.\n \"\"\"\n\n def _update_lagged_network_weights(self) -> None:\n self._lagged_networks.full_parameter_update()\n\n\nclass LaggedNetworkPolyakUpdateAlgorithmMixin(LaggedNetworkAlgorithmMixin):\n \"\"\"\n Algorithm mixin which adds support for lagged networks (target networks) where weights\n are updated via Polyak averaging (soft update using a convex combination of the parameters\n of the source and target networks with weight `tau` and `1-tau` respectively).\n \"\"\"\n\n def __init__(self, tau: float) -> None:\n \"\"\"\n :param tau: the fraction with which to use the source network's parameters, the inverse `1-tau` being\n the fraction with which to retain the target network's parameters.\n \"\"\"\n super().__init__()\n self.tau = tau\n\n def _update_lagged_network_weights(self) -> None:\n self._lagged_networks.polyak_parameter_update(self.tau)\n\n\nTPolicy = TypeVar(\"TPolicy\", bound=Policy)\nTTrainerParams = TypeVar(\"TTrainerParams\", bound=\"TrainerParams\")\n\n\nclass Algorithm(torch.nn.Module, Generic[TPolicy, TTrainerParams], ABC):\n \"\"\"\n The base class for reinforcement learning algorithms in Tianshou.\n\n An algorithm critically defines how to update the parameters of neural networks\n based on a batch data, optionally applying pre-processing and post-processing to the data.\n The actual update step is highly algorithm-specific and thus is defined in subclasses.\n \"\"\"\n\n _STATE_DICT_KEY_OPTIMIZERS = \"_optimizers\"\n\n def __init__(\n self,\n *,\n policy: TPolicy,\n ) -> None:\n \"\"\":param policy: the policy\"\"\"\n super().__init__()\n self.policy: TPolicy = policy\n self.lr_schedulers: list[LRScheduler] = []\n self._optimizers: list[Algorithm.Optimizer] = []\n \"\"\"\n list of optimizers associated with the algorithm (created via `_create_optimizer`),\n whose states will be returned when calling `state_dict` and which will be restored\n when calling `load_state_dict` accordingly\n \"\"\"\n\n class Optimizer:\n \"\"\"Wrapper for a torch optimizer that optionally performs gradient clipping.\"\"\"\n\n def __init__(\n self,\n optim: torch.optim.Optimizer,\n module: torch.nn.Module,\n max_grad_norm: float | None = None,\n ) -> None:\n \"\"\"\n :param optim: the optimizer\n :param module: the module whose parameters are being affected by `optim`\n :param max_grad_norm: the maximum L2 norm threshold for gradient clipping.\n When not None, gradients will be rescaled using to ensure their L2 norm does not\n exceed this value. This prevents exploding gradients and stabilizes training by\n limiting the magnitude of parameter updates.\n Set to None to disable gradient clipping.\n \"\"\"\n super().__init__()\n self._optim = optim\n self._module = module\n self._max_grad_norm = max_grad_norm\n\n def step(\n self,\n loss: torch.Tensor,\n retain_graph: bool | None = None,\n create_graph: bool = False,\n ) -> None:\n \"\"\"Performs an optimizer step, optionally applying gradient clipping (if configured at construction).\n\n :param loss: the loss to backpropagate\n :param retain_graph: passed on to `backward`\n :param create_graph: passed on to `backward`\n \"\"\"\n self._optim.zero_grad()\n loss.backward(retain_graph=retain_graph, create_graph=create_graph)\n if self._max_grad_norm is not None:\n nn.utils.clip_grad_norm_(self._module.parameters(), max_norm=self._max_grad_norm)\n self._optim.step()\n\n def state_dict(self) -> dict:\n \"\"\"Returns the `state_dict` of the wrapped optimizer.\"\"\"\n return self._optim.state_dict()\n\n def load_state_dict(self, state_dict: dict) -> None:\n \"\"\"Loads the given `state_dict` into the wrapped optimizer.\"\"\"\n self._optim.load_state_dict(state_dict)\n\n def _create_optimizer(\n self,\n module: torch.nn.Module,\n factory: OptimizerFactory,\n max_grad_norm: float | None = None,\n ) -> Optimizer:\n optimizer, lr_scheduler = factory.create_instances(module)\n if lr_scheduler is not None:\n self.lr_schedulers.append(lr_scheduler)\n optim = self.Optimizer(optimizer, module, max_grad_norm=max_grad_norm)\n self._optimizers.append(optim)\n return optim\n\n def state_dict(self, *args, destination=None, prefix=\"\", keep_vars=False): # type: ignore\n d = super().state_dict(*args, destination=destination, prefix=prefix, keep_vars=keep_vars)\n\n # add optimizer states\n opt_key = prefix + self._STATE_DICT_KEY_OPTIMIZERS\n assert opt_key not in d\n d[opt_key] = [o.state_dict() for o in self._optimizers]\n\n return d\n\n def load_state_dict(\n self, state_dict: Mapping[str, Any], strict: bool = True, assign: bool = False\n ) -> _IncompatibleKeys:\n # don't override type in annotation since it's is declared as Mapping in nn.Module\n state_dict = cast(dict[str, Any], state_dict)\n # restore optimizer states\n optimizers_state_dict = state_dict.pop(self._STATE_DICT_KEY_OPTIMIZERS)\n for optim, optim_state in zip(self._optimizers, optimizers_state_dict, strict=True):\n optim.load_state_dict(optim_state)\n\n return super().load_state_dict(state_dict, strict=strict, assign=assign)\n\n def _preprocess_batch(\n self,\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n ) -> RolloutBatchProtocol:\n \"\"\"Pre-process the data from the provided replay buffer.\n\n Meant to be overridden by subclasses. Typical usage is to add new keys to the\n batch, e.g., to add the value function of the next state. Used in :meth:`update`,\n which is usually called repeatedly during training.\n\n For modifying the replay buffer only once at the beginning\n (e.g., for offline learning) see :meth:`process_buffer`.\n \"\"\"\n return batch\n\n def _postprocess_batch(\n self,\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n ) -> None:\n \"\"\"Post-process the data from the provided replay buffer.\n\n This will only have an effect if the buffer has the\n method `update_weight` and the batch has the attribute `weight`.\n\n Typical usage is to update the sampling weight in prioritized\n experience replay. Used in :meth:`update`.\n \"\"\"\n if hasattr(buffer, \"update_weight\"):\n if hasattr(batch, \"weight\"):\n buffer.update_weight(indices, batch.weight)\n else:\n logger.warning(\n \"batch has no attribute 'weight', but buffer has an \"\n \"update_weight method. This is probably a mistake.\"\n \"Prioritized replay is disabled for this batch.\",\n )\n\n def _update(\n self,\n sample_size: int | None,\n buffer: ReplayBuffer | None,\n update_with_batch_fn: Callable[[RolloutBatchProtocol], TrainingStats],\n ) -> TrainingStats:\n \"\"\"Orchestrates an update step.\n\n An update involves three algorithm-specific sub-steps:\n * pre-processing of the batch,\n * performing the actual network update with the batch, and\n * post-processing of the batch.\n\n The return value is that of the network update call, augmented with the\n training time within update.\n\n :param sample_size: 0 means it will extract all the data from the buffer,\n otherwise it will sample a batch with given sample_size. None also\n means it will extract all the data from the buffer, but it will be shuffled\n first.\n :param buffer: the corresponding replay buffer.\n :param update_with_batch_fn: the function to call for the actual update step,\n which is algorithm-specific and thus provided by the subclass.\n\n :return: A dataclass object containing data to be logged (e.g., loss)\n \"\"\"\n if not self.policy.is_within_training_step:\n raise RuntimeError(\n f\"update() was called outside of a training step as signalled by {self.policy.is_within_training_step=} \"\n f\"If you want to update the policy without a Trainer, you will have to manage the above-mentioned \"\n f\"flag yourself. You can to this e.g., by using the contextmanager {policy_within_training_step.__name__}.\",\n )\n\n if buffer is None:\n return TrainingStats()\n start_time = time.time()\n batch, indices = buffer.sample(sample_size)\n TraceLogger.log(logger, lambda: f\"Updating with batch: indices={pickle_hash(indices)}\")\n batch = self._preprocess_batch(batch, buffer, indices)\n with torch_train_mode(self):\n training_stat = update_with_batch_fn(batch)\n self._postprocess_batch(batch, buffer, indices)\n for lr_scheduler in self.lr_schedulers:\n lr_scheduler.step()\n training_stat.train_time = time.time() - start_time\n return training_stat\n\n @staticmethod\n def value_mask(buffer: ReplayBuffer, indices: np.ndarray) -> np.ndarray:\n \"\"\"Value mask determines whether the obs_next of buffer[indices] is valid.\n\n For instance, usually \"obs_next\" after \"done\" flag is considered to be invalid,\n and its q/advantage value can provide meaningless (even misleading)\n information, and should be set to 0 by hand. But if \"done\" flag is generated\n because timelimit of game length (info[\"TimeLimit.truncated\"] is set to True in\n gym's settings), \"obs_next\" will instead be valid. Value mask is typically used\n for assisting in calculating the correct q/advantage value.\n\n :param buffer: the corresponding replay buffer.\n :param numpy.ndarray indices: indices of replay buffer whose \"obs_next\" will be\n judged.\n\n :return: A bool type numpy.ndarray in the same shape with indices. \"True\" means\n \"obs_next\" of that buffer[indices] is valid.\n \"\"\"\n return ~buffer.terminated[indices]\n\n @staticmethod\n def compute_episodic_return(\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n v_s_: np.ndarray | torch.Tensor | None = None,\n v_s: np.ndarray | torch.Tensor | None = None,\n gamma: float = 0.99,\n gae_lambda: float = 0.95,\n ) -> tuple[np.ndarray, np.ndarray]:\n r\"\"\"Compute returns over given batch.\n\n Use Implementation of Generalized Advantage Estimator (arXiv:1506.02438)\n to calculate q/advantage value of given batch. Returns are calculated as\n advantage + value, which is exactly equivalent to using :math:`TD(\\lambda)`\n for estimating returns.\n\n Setting `v_s_` and `v_s` to None (or all zeros) and `gae_lambda` to 1.0 calculates the\n discounted return-to-go/ Monte-Carlo return.\n\n :param batch: a data batch which contains several episodes of data in\n sequential order. Mind that the end of each finished episode of batch\n should be marked by done flag, unfinished (or collecting) episodes will be\n recognized by buffer.unfinished_index().\n :param buffer: the corresponding replay buffer.\n :param indices: tells the batch's location in buffer, batch is equal\n to buffer[indices].\n :param v_s_: the value function of all next states :math:`V(s')`.\n If None, it will be set to an array of 0.\n :param v_s: the value function of all current states :math:`V(s)`. If None,\n it is set based upon `v_s_` rolled by 1.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param gae_lambda: the lambda parameter in [0, 1] for generalized advantage estimation (GAE).\n Controls the bias-variance tradeoff in advantage estimates, acting as a\n weighting factor for combining different n-step advantage estimators. Higher values\n (closer to 1) reduce bias but increase variance by giving more weight to longer\n trajectories, while lower values (closer to 0) reduce variance but increase bias\n by relying more on the immediate TD error and value function estimates. At λ=0,\n GAE becomes equivalent to the one-step TD error (high bias, low variance); at λ=1,\n it becomes equivalent to Monte Carlo advantage estimation (low bias, high variance).\n Intermediate values create a weighted average of n-step returns, with exponentially\n decaying weights for longer-horizon returns. Typically set between 0.9 and 0.99 for\n most policy gradient methods.\n\n :return: two numpy arrays (returns, advantage) with each shape (bsz, ).\n \"\"\"\n rew = batch.rew\n if v_s_ is None:\n assert np.isclose(gae_lambda, 1.0)\n v_s_ = np.zeros_like(rew)\n else:\n v_s_ = to_numpy(v_s_.flatten())\n v_s_ = v_s_ * Algorithm.value_mask(buffer, indices)\n v_s = np.roll(v_s_, 1) if v_s is None else to_numpy(v_s.flatten())\n\n end_flag = np.logical_or(batch.terminated, batch.truncated)\n end_flag[np.isin(indices, buffer.unfinished_index())] = True\n advantage = _gae(v_s, v_s_, rew, end_flag, gamma, gae_lambda)\n returns = advantage + v_s\n # normalization varies from each policy, so we don't do it here\n return returns, advantage\n\n @staticmethod\n def compute_nstep_return(\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n target_q_fn: Callable[[ReplayBuffer, np.ndarray], torch.Tensor],\n gamma: float = 0.99,\n n_step: int = 1,\n ) -> BatchWithReturnsProtocol:\n r\"\"\"\n Computes the n-step return for Q-learning targets, adds it to the batch and returns the resulting batch.\n\n .. math::\n G_t = \\sum_{i = t}^{t + n - 1} \\gamma^{i - t}(1 - d_i)r_i +\n \\gamma^n (1 - d_{t + n}) Q_{\\mathrm{target}}(s_{t + n})\n\n where :math:`\\gamma` is the discount factor, :math:`\\gamma \\in [0, 1]`,\n :math:`d_t` is the done flag of step :math:`t`.\n\n :param batch: a data batch, which is equal to buffer[indices].\n :param buffer: the data buffer.\n :param indices: tell batch's location in buffer\n :param target_q_fn: a function which computes the target Q value\n of \"obs_next\" given data buffer and wanted indices (`n_step` steps ahead).\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param n_step: the number of estimation step, should be an int greater\n than 0.\n :return: a Batch. The result will be stored in `batch.returns` as a\n torch.Tensor with the same shape as target_q_fn's return tensor.\n \"\"\"\n if len(indices) != len(batch):\n raise ValueError(f\"Batch size {len(batch)} and indices size {len(indices)} mismatch.\")\n\n # naming convention\n # I = number of indices\n # B = size of the replay buffer\n # N = n_step\n # A = the output dimension of target_q_fn for a single index. Presumably\n # this is the number of actions in the discrete case, or something like that.\n # 1 = 1 extra dimension\n # TODO: it's very weird that this is not always one!\n # We set the n-step-return for a single index to be the same shape as the target_q_fn.\n # I don't understand how a non-scalar value would make sense there, but such cases are covered by tests\n\n # support in following naming convention\n I = len(indices)\n N = n_step\n\n _indices_to_stack = [indices]\n for _ in range(N - 1):\n next_indices = buffer.next(_indices_to_stack[-1])\n _indices_to_stack.append(next_indices)\n stacked_indices_NI = np.stack(_indices_to_stack)\n \"\"\"The stacked indices represent a 2d array of shape `IxN` of the type\n [\n [i_1, i_2,...],\n [i_(next(1)), i_(next(2)), ...],\n [i_(next(next(1)), ...\n ...\n ]\n where `next` is the subsequent transition in the buffer.\n \"\"\"\n indices_after_n_steps_I = stacked_indices_NI[-1]\n \"\"\"Indicates indexes of transitions in buffer that occur N steps after the user provided 'indices';\n they are truncated at the end of each episode\"\"\"\n\n with torch.no_grad():\n target_q_torch_IA = target_q_fn(buffer, indices_after_n_steps_I)\n target_q_IA = to_numpy(target_q_torch_IA.reshape(I, -1))\n \"\"\"Represents the Q-values (one for each action) of the transition after N steps.\"\"\"\n\n target_q_IA *= Algorithm.value_mask(buffer, indices_after_n_steps_I).reshape(-1, 1)\n end_flag_B = buffer.done.copy()\n end_flag_B[buffer.unfinished_index()] = True\n n_step_return_IA = _nstep_return(\n buffer.rew,\n end_flag_B,\n target_q_IA,\n stacked_indices_NI,\n gamma,\n n_step,\n )\n \"\"\"The n-step return plus the last Q-values, see method's docstring\"\"\"\n\n batch.returns = to_torch_as(n_step_return_IA, target_q_torch_IA)\n\n # TODO: this is simply converting to a certain type. Why is this necessary, and why is it happening here?\n if hasattr(batch, \"weight\"):\n batch.weight = to_torch_as(batch.weight, target_q_torch_IA)\n\n return cast(BatchWithReturnsProtocol, batch)\n\n @abstractmethod\n def create_trainer(self, params: TTrainerParams) -> \"Trainer\":\n pass\n\n def run_training(self, params: TTrainerParams) -> \"InfoStats\":\n trainer = self.create_trainer(params)\n return trainer.run()\n\n\nclass OnPolicyAlgorithm(\n Algorithm[TPolicy, \"OnPolicyTrainerParams\"],\n Generic[TPolicy],\n ABC,\n):\n \"\"\"Base class for on-policy RL algorithms.\"\"\"\n\n def create_trainer(self, params: \"OnPolicyTrainerParams\") -> \"OnPolicyTrainer\":\n from tianshou.trainer import OnPolicyTrainer\n\n return OnPolicyTrainer(self, params)\n\n @abstractmethod\n def _update_with_batch(\n self, batch: RolloutBatchProtocol, batch_size: int | None, repeat: int\n ) -> TrainingStats:\n \"\"\"Performs an update step based on the given batch of data, updating the network\n parameters.\n\n :param batch: the batch of data\n :param batch_size: the minibatch size for gradient updates\n :param repeat: the number of times to repeat the update over the whole batch\n :return: a dataclas object containing statistics on the learning process, including\n the data needed to be logged (e.g. loss values).\n \"\"\"\n\n def update(\n self,\n buffer: ReplayBuffer,\n batch_size: int | None,\n repeat: int,\n ) -> TrainingStats:\n update_with_batch_fn = lambda batch: self._update_with_batch(\n batch=batch, batch_size=batch_size, repeat=repeat\n )\n return super()._update(\n sample_size=0, buffer=buffer, update_with_batch_fn=update_with_batch_fn\n )\n\n\nclass OffPolicyAlgorithm(\n Algorithm[TPolicy, \"OffPolicyTrainerParams\"],\n Generic[TPolicy],\n ABC,\n):\n \"\"\"Base class for off-policy RL algorithms.\"\"\"\n\n def create_trainer(self, params: \"OffPolicyTrainerParams\") -> \"OffPolicyTrainer\":\n from tianshou.trainer import OffPolicyTrainer\n\n return OffPolicyTrainer(self, params)\n\n @abstractmethod\n def _update_with_batch(\n self,\n batch: RolloutBatchProtocol,\n ) -> TrainingStats:\n \"\"\"Performs an update step based on the given batch of data, updating the network\n parameters.\n\n :param batch: the batch of data\n :return: a dataclas object containing statistics on the learning process, including\n the data needed to be logged (e.g. loss values).\n \"\"\"\n\n def update(\n self,\n buffer: ReplayBuffer,\n sample_size: int | None,\n ) -> TrainingStats:\n update_with_batch_fn = lambda batch: self._update_with_batch(batch)\n return super()._update(\n sample_size=sample_size,\n buffer=buffer,\n update_with_batch_fn=update_with_batch_fn,\n )\n\n\nclass OfflineAlgorithm(\n Algorithm[TPolicy, \"OfflineTrainerParams\"],\n Generic[TPolicy],\n ABC,\n):\n \"\"\"Base class for offline RL algorithms.\"\"\"\n\n def process_buffer(self, buffer: TBuffer) -> TBuffer:\n \"\"\"Pre-process the replay buffer to prepare for offline learning, e.g. to add new keys.\"\"\"\n return buffer\n\n def run_training(self, params: \"OfflineTrainerParams\") -> \"InfoStats\":\n # NOTE: This override is required for correct typing when converting\n # an algorithm to an offline algorithm using diamond inheritance\n # (e.g. DiscreteCQL) in order to make it match first in the MRO\n return super().run_training(params)\n\n def create_trainer(self, params: \"OfflineTrainerParams\") -> \"OfflineTrainer\":\n from tianshou.trainer import OfflineTrainer\n\n return OfflineTrainer(self, params)\n\n @abstractmethod\n def _update_with_batch(\n self,\n batch: RolloutBatchProtocol,\n ) -> TrainingStats:\n \"\"\"Performs an update step based on the given batch of data, updating the network\n parameters.\n\n :param batch: the batch of data\n :return: a dataclas object containing statistics on the learning process, including\n the data needed to be logged (e.g. loss values).\n \"\"\"\n\n def update(\n self,\n buffer: ReplayBuffer,\n sample_size: int | None,\n ) -> TrainingStats:\n update_with_batch_fn = lambda batch: self._update_with_batch(batch)\n return super()._update(\n sample_size=sample_size,\n buffer=buffer,\n update_with_batch_fn=update_with_batch_fn,\n )\n\n\nclass OnPolicyWrapperAlgorithm(\n OnPolicyAlgorithm[TPolicy],\n Generic[TPolicy],\n ABC,\n):\n \"\"\"\n Base class for an on-policy algorithm that is a wrapper around another algorithm.\n\n It applies the wrapped algorithm's pre-processing and post-processing methods\n and chains the update method of the wrapped algorithm with the wrapper's own update method.\n \"\"\"\n\n def __init__(\n self,\n wrapped_algorithm: OnPolicyAlgorithm[TPolicy],\n ):\n super().__init__(policy=wrapped_algorithm.policy)\n self.wrapped_algorithm = wrapped_algorithm\n\n def _preprocess_batch(\n self,\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n ) -> RolloutBatchProtocol:\n \"\"\"Performs the pre-processing as defined by the wrapped algorithm.\"\"\"\n return self.wrapped_algorithm._preprocess_batch(batch, buffer, indices)\n\n def _postprocess_batch(\n self,\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n ) -> None:\n \"\"\"Performs the batch post-processing as defined by the wrapped algorithm.\"\"\"\n self.wrapped_algorithm._postprocess_batch(batch, buffer, indices)\n\n def _update_with_batch(\n self, batch: RolloutBatchProtocol, batch_size: int | None, repeat: int\n ) -> TrainingStats:\n \"\"\"Performs the update as defined by the wrapped algorithm, followed by the wrapper's update.\"\"\"\n original_stats = self.wrapped_algorithm._update_with_batch(\n batch, batch_size=batch_size, repeat=repeat\n )\n return self._wrapper_update_with_batch(batch, batch_size, repeat, original_stats)\n\n @abstractmethod\n def _wrapper_update_with_batch(\n self,\n batch: RolloutBatchProtocol,\n batch_size: int | None,\n repeat: int,\n original_stats: TrainingStats,\n ) -> TrainingStats:\n pass\n\n\nclass OffPolicyWrapperAlgorithm(\n OffPolicyAlgorithm[TPolicy],\n Generic[TPolicy],\n ABC,\n):\n \"\"\"\n Base class for an off-policy algorithm that is a wrapper around another algorithm.\n\n It applies the wrapped algorithm's pre-processing and post-processing methods\n and chains the update method of the wrapped algorithm with the wrapper's own update method.\n \"\"\"\n\n def __init__(\n self,\n wrapped_algorithm: OffPolicyAlgorithm[TPolicy],\n ):\n super().__init__(policy=wrapped_algorithm.policy)\n self.wrapped_algorithm = wrapped_algorithm\n\n def _preprocess_batch(\n self,\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n ) -> RolloutBatchProtocol:\n \"\"\"Performs the pre-processing as defined by the wrapped algorithm.\"\"\"\n return self.wrapped_algorithm._preprocess_batch(batch, buffer, indices)\n\n def _postprocess_batch(\n self,\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n ) -> None:\n \"\"\"Performs the batch post-processing as defined by the wrapped algorithm.\"\"\"\n self.wrapped_algorithm._postprocess_batch(batch, buffer, indices)\n\n def _update_with_batch(\n self,\n batch: RolloutBatchProtocol,\n ) -> TrainingStats:\n \"\"\"Performs the update as defined by the wrapped algorithm, followed by the wrapper's update .\"\"\"\n original_stats = self.wrapped_algorithm._update_with_batch(batch)\n return self._wrapper_update_with_batch(batch, original_stats)\n\n @abstractmethod\n def _wrapper_update_with_batch(\n self, batch: RolloutBatchProtocol, original_stats: TrainingStats\n ) -> TrainingStats:\n pass\n\n\nclass RandomActionPolicy(Policy):\n def __init__(\n self,\n action_space: gym.Space,\n ) -> None:\n super().__init__(action_space=action_space)\n if not isinstance(action_space, gym.spaces.Discrete | gym.spaces.Box):\n raise NotImplementedError(\n f\"RandomActionPolicy currently only supports Discrete and Box action spaces, but got {action_space}.\",\n )\n self.actor = RandomActor(action_space)\n\n def forward(\n self,\n batch: ObsBatchProtocol,\n state: dict | BatchProtocol | np.ndarray | None = None,\n **kwargs: Any,\n ) -> ActStateBatchProtocol:\n act, next_state = self.actor.compute_action_batch(batch.obs), state\n return cast(ActStateBatchProtocol, Batch(act=act, state=next_state))\n\n\n@njit\ndef _gae(\n v_s: np.ndarray,\n v_s_: np.ndarray,\n rew: np.ndarray,\n end_flag: np.ndarray,\n gamma: float,\n gae_lambda: float,\n) -> np.ndarray:\n r\"\"\"Computes advantages with GAE.\n\n The return is given by the output of this + v_s. Note that the advantages plus v_s\n is exactly the same as the TD-lambda target, which is computed by the recursive\n formula:\n\n .. math::\n G_t^\\lambda = r_t + \\gamma ( \\lambda G_{t+1}^\\lambda + (1 - \\lambda) V_{t+1} )\n\n The GAE is computed recursively as:\n\n .. math::\n \\delta_t = r_t + \\gamma V_{t+1} - V_t \\n\n A_t^\\lambda= \\delta_t + \\gamma \\lambda A_{t+1}^\\lambda\n\n And the following equality holds:\n\n .. math::\n G_t^\\lambda = A_t^\\lambda+ V_t\n\n :param v_s: values in an episode, i.e. $V_t$\n :param v_s_: next values in an episode, i.e. v_s shifted by 1, equivalent to\n $V_{t+1}$\n :param rew: rewards in an episode, i.e. $r_t$\n :param end_flag: boolean array indicating whether the episode is done\n :param gamma: the discount factor in [0, 1] for future rewards.\n :param gae_lambda: the lambda parameter in [0, 1] for generalized advantage estimation (GAE).\n Controls the bias-variance tradeoff in advantage estimates, acting as a\n weighting factor for combining different n-step advantage estimators. Higher values\n (closer to 1) reduce bias but increase variance by giving more weight to longer\n trajectories, while lower values (closer to 0) reduce variance but increase bias\n by relying more on the immediate TD error and value function estimates. At λ=0,\n GAE becomes equivalent to the one-step TD error (high bias, low variance); at λ=1,\n it becomes equivalent to Monte Carlo advantage estimation (low bias, high variance).\n Intermediate values create a weighted average of n-step returns, with exponentially\n decaying weights for longer-horizon returns. Typically set between 0.9 and 0.99 for\n most policy gradient methods.\n :return:\n \"\"\"\n returns = np.zeros(rew.shape)\n delta = rew + v_s_ * gamma - v_s\n discount = (1.0 - end_flag) * (gamma * gae_lambda)\n gae = 0.0\n for i in range(len(rew) - 1, -1, -1):\n gae = delta[i] + discount[i] * gae\n returns[i] = gae\n return returns\n\n\n@njit\ndef episode_mc_return_to_go(rewards: np.ndarray, gamma: float = 0.99) -> np.ndarray:\n \"\"\"Calculates discounted monte-carlo returns to go from rewards of a single episode.\n\n :param rewards: rewards of a single episode. Assumed to be a 1-dim array from reset till the end of the episode.\n :param gamma: discount factor\n :return: a numpy array of shape (len(rewards), ).\n \"\"\"\n len_episode = len(rewards)\n ret2go = np.zeros(len_episode)\n ret2go[-1] = rewards[-1]\n\n for j in range(len_episode - 2, -1, -1):\n ret2go[j] = rewards[j] + gamma * ret2go[j + 1]\n return ret2go\n\n\n@njit\ndef _nstep_return(\n rew_B: np.ndarray,\n end_flag_B: np.ndarray,\n target_q_IA: np.ndarray,\n stacked_indices_NI: np.ndarray,\n gamma: float,\n n_step: int,\n) -> np.ndarray:\n \"\"\"Computes n-step returns starting at the transitions at the selected indices in the buffer.\n Importantly, this is not a pure MC n-step return but it also uses the Q-values of the\n obs-action pair after the n-step transition to compute the return.\n\n Thus, it computes `n_step_return + gamma^(n) * Q(s_{t+n}, a_{t+n})` where\n `n_step_return = r_t + gamma * r_{t+1} + ... + gamma^(n-1) * r_{t+n-1}`.\n See the docstring of `compute_nstep_return` for more details.\n\n The target_q_B should be the array of `Q(s_{t+n}, a_{t+n})` corresponding to\n the batch of rewards that started at t=0.\n\n Notation:\n I = number of indices\n B = size of the replay buffer\n N = n_step\n A = the output dimension of target_q_fn for a single index. Presumably,\n this is the number of actions in the discrete case, or something like that.\n See comments in the method `compute_nstep_return` for more details.\n 1 = 1 extra dimension\n\n :param rew_B: rewards of the entire replay buffer\n :param end_flag_B: end flags (where done=True) of the entire replay buffer\n :param target_q_IA: Q-values of the transitions after n steps. Passed as a 2d array of shape (I, A)\n :param stacked_indices_NI: indices of the transitions in the buffer of the structure\n [\n [i_1, i_2,...],\n [i_(next(1)), i_(next(2)), ...],\n [i_(next(next(1)), ...\n ...\n ]\n where `next` is the subsequent transition in the buffer.\n \"\"\"\n N = n_step\n I, A = target_q_IA.shape\n gamma_buffer_N = np.ones(N + 1)\n for i in range(1, N + 1):\n gamma_buffer_N[i] = gamma_buffer_N[i - 1] * gamma\n target_q_IA = target_q_IA.reshape(I, -1)\n \"\"\"Make sure tarqet_q_I has an empty extra dimension, usually already passed with the\n right shape, hence the input param name\"\"\"\n n_step_mc_returns_IA = np.zeros(target_q_IA.shape)\n \"\"\"Will hold the n_step MC return part of the final n_step + Q-value return.\n \"\"\"\n gammas_IN = np.full(I, N)\n for n in range(N - 1, -1, -1):\n now = stacked_indices_NI[n]\n gammas_IN[end_flag_B[now] > 0] = n + 1\n n_step_mc_returns_IA[end_flag_B[now] > 0] = 0.0\n n_step_mc_returns_IA = rew_B[now].reshape(I, 1) + gamma * n_step_mc_returns_IA\n\n n_step_return_with_Q_IA = (\n target_q_IA * gamma_buffer_N[gammas_IN].reshape(I, 1) + n_step_mc_returns_IA\n )\n return n_step_return_with_Q_IA.reshape((I, A))\n" }, { "path": "tianshou/algorithm/imitation/__init__.py", "content": "" }, { "path": "tianshou/algorithm/imitation/bcq.py", "content": "import copy\nfrom dataclasses import dataclass\nfrom typing import Any, Literal, TypeVar, cast\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nimport torch.nn.functional as F\n\nfrom tianshou.algorithm.algorithm_base import (\n LaggedNetworkPolyakUpdateAlgorithmMixin,\n OfflineAlgorithm,\n Policy,\n TrainingStats,\n)\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import Batch, to_torch\nfrom tianshou.data.batch import BatchProtocol\nfrom tianshou.data.types import ActBatchProtocol, ObsBatchProtocol, RolloutBatchProtocol\nfrom tianshou.utils.net.continuous import VAE\n\n\n@dataclass(kw_only=True)\nclass BCQTrainingStats(TrainingStats):\n actor_loss: float\n critic1_loss: float\n critic2_loss: float\n vae_loss: float\n\n\nTBCQTrainingStats = TypeVar(\"TBCQTrainingStats\", bound=BCQTrainingStats)\n\n\nclass BCQPolicy(Policy):\n def __init__(\n self,\n *,\n actor_perturbation: torch.nn.Module,\n action_space: gym.Space,\n critic: torch.nn.Module,\n vae: VAE,\n forward_sampled_times: int = 100,\n observation_space: gym.Space | None = None,\n action_scaling: bool = False,\n action_bound_method: Literal[\"clip\", \"tanh\"] | None = \"clip\",\n ) -> None:\n \"\"\"\n :param actor_perturbation: the actor perturbation. `(s, a -> perturbed a)`\n :param critic: the first critic network.\n :param vae: the VAE network, generating actions similar to those in batch.\n :param forward_sampled_times: the number of sampled actions in forward function.\n The policy samples many actions and takes the action with the max value.\n :param observation_space: the environment's observation space\n :param action_scaling: flag indicating whether, for continuous action spaces, actions\n should be scaled from the standard neural network output range [-1, 1] to the\n environment's action space range [action_space.low, action_space.high].\n This applies to continuous action spaces only (gym.spaces.Box) and has no effect\n for discrete spaces.\n When enabled, policy outputs are expected to be in the normalized range [-1, 1]\n (after bounding), and are then linearly transformed to the actual required range.\n This improves neural network training stability, allows the same algorithm to work\n across environments with different action ranges, and standardizes exploration\n strategies.\n Should be disabled if the actor model already produces outputs in the correct range.\n :param action_bound_method: the method used for bounding actions in continuous action spaces\n to the range [-1, 1] before scaling them to the environment's action space (provided\n that `action_scaling` is enabled).\n This applies to continuous action spaces only (`gym.spaces.Box`) and should be set to None\n for discrete spaces.\n When set to \"clip\", actions exceeding the [-1, 1] range are simply clipped to this\n range. When set to \"tanh\", a hyperbolic tangent function is applied, which smoothly\n constrains outputs to [-1, 1] while preserving gradients.\n The choice of bounding method affects both training dynamics and exploration behavior.\n Clipping provides hard boundaries but may create plateau regions in the gradient\n landscape, while tanh provides smoother transitions but can compress sensitivity\n near the boundaries.\n Should be set to None if the actor model inherently produces bounded outputs.\n Typically used together with `action_scaling=True`.\n \"\"\"\n super().__init__(\n action_space=action_space,\n observation_space=observation_space,\n action_scaling=action_scaling,\n action_bound_method=action_bound_method,\n )\n self.actor_perturbation = actor_perturbation\n self.critic = critic\n self.vae = vae\n self.forward_sampled_times = forward_sampled_times\n\n def forward(\n self,\n batch: ObsBatchProtocol,\n state: dict | BatchProtocol | np.ndarray | None = None,\n **kwargs: Any,\n ) -> ActBatchProtocol:\n \"\"\"Compute action over the given batch data.\"\"\"\n # There is \"obs\" in the Batch\n # obs_group: several groups. Each group has a state.\n device = next(self.parameters()).device\n obs_group: torch.Tensor = to_torch(batch.obs, device=device)\n act_group = []\n for obs_orig in obs_group:\n # now obs is (state_dim)\n obs = (obs_orig.reshape(1, -1)).repeat(self.forward_sampled_times, 1)\n # now obs is (forward_sampled_times, state_dim)\n\n # decode(obs) generates action and actor perturbs it\n act = self.actor_perturbation(obs, self.vae.decode(obs))\n # now action is (forward_sampled_times, action_dim)\n q1 = self.critic(obs, act)\n # q1 is (forward_sampled_times, 1)\n max_indice = q1.argmax(0)\n act_group.append(act[max_indice].cpu().data.numpy().flatten())\n act_group = np.array(act_group)\n return cast(ActBatchProtocol, Batch(act=act_group))\n\n\nclass BCQ(\n OfflineAlgorithm[BCQPolicy],\n LaggedNetworkPolyakUpdateAlgorithmMixin,\n):\n \"\"\"Implementation of Batch-Constrained Deep Q-learning (BCQ) algorithm. arXiv:1812.02900.\"\"\"\n\n def __init__(\n self,\n *,\n policy: BCQPolicy,\n actor_perturbation_optim: OptimizerFactory,\n critic_optim: OptimizerFactory,\n vae_optim: OptimizerFactory,\n critic2: torch.nn.Module | None = None,\n critic2_optim: OptimizerFactory | None = None,\n gamma: float = 0.99,\n tau: float = 0.005,\n lmbda: float = 0.75,\n num_sampled_action: int = 10,\n ) -> None:\n \"\"\"\n :param policy: the policy\n :param actor_perturbation_optim: the optimizer factory for the policy's actor perturbation network.\n :param critic_optim: the optimizer factory for the policy's critic network.\n :param critic2: the second critic network; if None, clone the critic from the policy\n :param critic2_optim: the optimizer factory for the second critic network; if None, use optimizer factory of first critic\n :param vae_optim: the optimizer factory for the VAE network.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param tau: the soft update coefficient for target networks, controlling the rate at which\n target networks track the learned networks.\n When the parameters of the target network are updated with the current (source) network's\n parameters, a weighted average is used: target = tau * source + (1 - tau) * target.\n Smaller values (closer to 0) create more stable but slower learning as target networks\n change more gradually. Higher values (closer to 1) allow faster learning but may reduce\n stability.\n Typically set to a small value (0.001 to 0.01) for most reinforcement learning tasks.\n :param lmbda: param for Clipped Double Q-learning.\n :param num_sampled_action: the number of sampled actions in calculating target Q.\n The algorithm samples several actions using VAE, and perturbs each action to get the target Q.\n \"\"\"\n # actor is Perturbation!\n super().__init__(\n policy=policy,\n )\n LaggedNetworkPolyakUpdateAlgorithmMixin.__init__(self, tau=tau)\n self.actor_perturbation_target = self._add_lagged_network(self.policy.actor_perturbation)\n self.actor_perturbation_optim = self._create_optimizer(\n self.policy.actor_perturbation, actor_perturbation_optim\n )\n\n self.critic_target = self._add_lagged_network(self.policy.critic)\n self.critic_optim = self._create_optimizer(self.policy.critic, critic_optim)\n\n self.critic2 = critic2 or copy.deepcopy(self.policy.critic)\n self.critic2_target = self._add_lagged_network(self.critic2)\n self.critic2_optim = self._create_optimizer(self.critic2, critic2_optim or critic_optim)\n\n self.vae_optim = self._create_optimizer(self.policy.vae, vae_optim)\n\n self.gamma = gamma\n self.lmbda = lmbda\n self.num_sampled_action = num_sampled_action\n\n def _update_with_batch(\n self,\n batch: RolloutBatchProtocol,\n ) -> BCQTrainingStats:\n # batch: obs, act, rew, done, obs_next. (numpy array)\n # (batch_size, state_dim)\n # TODO: This does not use policy.forward but computes things directly, which seems odd\n\n device = next(self.parameters()).device\n batch: Batch = to_torch(batch, dtype=torch.float, device=device)\n obs, act = batch.obs, batch.act\n batch_size = obs.shape[0]\n\n # mean, std: (state.shape[0], latent_dim)\n recon, mean, std = self.policy.vae(obs, act)\n recon_loss = F.mse_loss(act, recon)\n # (....) is D_KL( N(mu, sigma) || N(0,1) )\n KL_loss = (-torch.log(std) + (std.pow(2) + mean.pow(2) - 1) / 2).mean()\n vae_loss = recon_loss + KL_loss / 2\n\n self.vae_optim.step(vae_loss)\n\n # critic training:\n with torch.no_grad():\n # repeat num_sampled_action times\n obs_next = batch.obs_next.repeat_interleave(self.num_sampled_action, dim=0)\n # now obs_next: (num_sampled_action * batch_size, state_dim)\n\n # perturbed action generated by VAE\n act_next = self.policy.vae.decode(obs_next)\n # now obs_next: (num_sampled_action * batch_size, action_dim)\n target_Q1 = self.critic_target(obs_next, act_next)\n target_Q2 = self.critic2_target(obs_next, act_next)\n\n # Clipped Double Q-learning\n target_Q = self.lmbda * torch.min(target_Q1, target_Q2) + (1 - self.lmbda) * torch.max(\n target_Q1,\n target_Q2,\n )\n # now target_Q: (num_sampled_action * batch_size, 1)\n\n # the max value of Q\n target_Q = target_Q.reshape(batch_size, -1).max(dim=1)[0].reshape(-1, 1)\n # now target_Q: (batch_size, 1)\n\n target_Q = (\n batch.rew.reshape(-1, 1)\n + torch.logical_not(batch.done).reshape(-1, 1) * self.gamma * target_Q\n )\n target_Q = target_Q.float()\n\n current_Q1 = self.policy.critic(obs, act)\n current_Q2 = self.critic2(obs, act)\n\n critic1_loss = F.mse_loss(current_Q1, target_Q)\n critic2_loss = F.mse_loss(current_Q2, target_Q)\n self.critic_optim.step(critic1_loss)\n self.critic2_optim.step(critic2_loss)\n\n sampled_act = self.policy.vae.decode(obs)\n perturbed_act = self.policy.actor_perturbation(obs, sampled_act)\n\n # max\n actor_loss = -self.policy.critic(obs, perturbed_act).mean()\n\n self.actor_perturbation_optim.step(actor_loss)\n\n # update target networks\n self._update_lagged_network_weights()\n\n return BCQTrainingStats(\n actor_loss=actor_loss.item(),\n critic1_loss=critic1_loss.item(),\n critic2_loss=critic2_loss.item(),\n vae_loss=vae_loss.item(),\n )\n" }, { "path": "tianshou/algorithm/imitation/cql.py", "content": "from copy import deepcopy\nfrom dataclasses import dataclass\nfrom typing import cast\n\nimport numpy as np\nimport torch\nimport torch.nn.functional as F\nfrom overrides import override\n\nfrom tianshou.algorithm.algorithm_base import (\n LaggedNetworkPolyakUpdateAlgorithmMixin,\n OfflineAlgorithm,\n)\nfrom tianshou.algorithm.modelfree.sac import Alpha, SACPolicy, SACTrainingStats\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import Batch, ReplayBuffer, to_torch\nfrom tianshou.data.buffer.buffer_base import TBuffer\nfrom tianshou.data.types import RolloutBatchProtocol\nfrom tianshou.utils.conversion import to_optional_float\nfrom tianshou.utils.torch_utils import torch_device\n\n\n@dataclass(kw_only=True)\nclass CQLTrainingStats(SACTrainingStats):\n \"\"\"A data structure for storing loss statistics of the CQL learn step.\"\"\"\n\n cql_alpha: float | None = None\n cql_alpha_loss: float | None = None\n\n\n# TODO: Perhaps SACPolicy should get a more generic name\nclass CQL(OfflineAlgorithm[SACPolicy], LaggedNetworkPolyakUpdateAlgorithmMixin):\n \"\"\"Implementation of the conservative Q-learning (CQL) algorithm. arXiv:2006.04779.\"\"\"\n\n def __init__(\n self,\n *,\n policy: SACPolicy,\n policy_optim: OptimizerFactory,\n critic: torch.nn.Module,\n critic_optim: OptimizerFactory,\n critic2: torch.nn.Module | None = None,\n critic2_optim: OptimizerFactory | None = None,\n cql_alpha_lr: float = 1e-4,\n cql_weight: float = 1.0,\n tau: float = 0.005,\n gamma: float = 0.99,\n alpha: float | Alpha = 0.2,\n temperature: float = 1.0,\n with_lagrange: bool = True,\n lagrange_threshold: float = 10.0,\n min_action: float = -1.0,\n max_action: float = 1.0,\n num_repeat_actions: int = 10,\n alpha_min: float = 0.0,\n alpha_max: float = 1e6,\n max_grad_norm: float = 1.0,\n calibrated: bool = True,\n ) -> None:\n \"\"\"\n :param actor: the actor network following the rules (s -> a)\n :param policy_optim: the optimizer factory for the policy/its actor network.\n :param critic: the first critic network.\n :param critic_optim: the optimizer factory for the first critic network.\n :param action_space: the environment's action space.\n :param critic2: the second critic network. (s, a -> Q(s, a)).\n If None, use the same network as critic (via deepcopy).\n :param critic2_optim: the optimizer factory for the second critic network.\n If None, clone the first critic's optimizer factory.\n :param cql_alpha_lr: the learning rate for the Lagrange multiplier optimization.\n Controls how quickly the CQL regularization coefficient (alpha) adapts during training.\n Higher values allow faster adaptation but may cause instability in the training process.\n Lower values provide more stable but slower adaptation of the regularization strength.\n Only relevant when with_lagrange=True.\n :param cql_weight: the coefficient that scales the conservative regularization term in the Q-function loss.\n Controls the strength of the conservative Q-learning component relative to standard TD learning.\n Higher values enforce more conservative value estimates by penalizing overestimation more strongly.\n Lower values allow the algorithm to behave more like standard Q-learning.\n Increasing this weight typically improves performance in purely offline settings where\n overestimation bias can lead to poor policy extraction.\n :param tau: the soft update coefficient for target networks, controlling the rate at which\n target networks track the learned networks.\n When the parameters of the target network are updated with the current (source) network's\n parameters, a weighted average is used: target = tau * source + (1 - tau) * target.\n Smaller values (closer to 0) create more stable but slower learning as target networks\n change more gradually. Higher values (closer to 1) allow faster learning but may reduce\n stability.\n Typically set to a small value (0.001 to 0.01) for most reinforcement learning tasks.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param alpha: the entropy regularization coefficient alpha or an object\n which can be used to automatically tune it (e.g. an instance of `AutoAlpha`).\n :param temperature: the temperature parameter used in the LogSumExp calculation of the CQL loss.\n Controls the sharpness of the softmax distribution when computing the expected Q-values.\n Lower values make the LogSumExp operation more selective, focusing on the highest Q-values.\n Higher values make the operation closer to an average, giving more weight to all Q-values.\n The temperature affects how conservatively the algorithm penalizes out-of-distribution actions.\n :param with_lagrange: a flag indicating whether to automatically tune the CQL regularization strength.\n If True, uses Lagrangian dual gradient descent to dynamically adjust the CQL alpha parameter.\n This formulation maintains the CQL regularization loss near the lagrange_threshold value.\n Adaptive tuning helps balance conservative learning against excessive pessimism.\n If False, the conservative loss is scaled by a fixed cql_weight throughout training.\n The original CQL paper recommends setting this to True for most offline RL tasks.\n :param lagrange_threshold: the target value for the CQL regularization loss when using Lagrangian optimization.\n When with_lagrange=True, the algorithm dynamically adjusts the CQL alpha parameter to maintain\n the regularization loss close to this threshold.\n Lower values result in more conservative behavior by enforcing stronger penalties on\n out-of-distribution actions.\n Higher values allow more optimistic Q-value estimates similar to standard Q-learning.\n This threshold effectively controls the level of conservatism in CQL's value estimation.\n :param min_action: the lower bound for each dimension of the action space.\n Used when sampling random actions for the CQL regularization term.\n Should match the environment's action space minimum values.\n These random actions help penalize Q-values for out-of-distribution actions.\n Typically set to -1.0 for normalized continuous action spaces.\n :param max_action: the upper bound for each dimension of the action space.\n Used when sampling random actions for the CQL regularization term.\n Should match the environment's action space maximum values.\n These random actions help penalize Q-values for out-of-distribution actions.\n Typically set to 1.0 for normalized continuous action spaces.\n :param num_repeat_actions: the number of action samples generated per state when computing\n the CQL regularization term.\n Controls how many random and policy actions are sampled for each state in the batch when\n estimating expected Q-values.\n Higher values provide more accurate approximation of the expected Q-values but increase\n computational cost.\n Lower values reduce computation but may provide less stable or less accurate regularization.\n The original CQL paper typically uses values around 10.\n :param alpha_min: the minimum value allowed for the adaptive CQL regularization coefficient.\n When using Lagrangian optimization (with_lagrange=True), constrains the automatically tuned\n cql_alpha parameter to be at least this value.\n Prevents the regularization strength from becoming too small during training.\n Setting a positive value ensures the algorithm maintains at least some degree of conservatism.\n Only relevant when with_lagrange=True.\n :param alpha_max: the maximum value allowed for the adaptive CQL regularization coefficient.\n When using Lagrangian optimization (with_lagrange=True), constrains the automatically tuned\n cql_alpha parameter to be at most this value.\n Prevents the regularization strength from becoming too large during training.\n Setting an appropriate upper limit helps avoid overly conservative behavior that might hinder\n learning useful value functions.\n Only relevant when with_lagrange=True.\n :param max_grad_norm: the maximum L2 norm threshold for gradient clipping when updating critic networks.\n Gradients with norm exceeding this value will be rescaled to have norm equal to this value.\n Helps stabilize training by preventing excessively large parameter updates from outlier samples.\n Higher values allow larger updates but may lead to training instability.\n Lower values enforce more conservative updates but may slow down learning.\n Setting to a large value effectively disables gradient clipping.\n :param calibrated: a flag indicating whether to use the calibrated version of CQL (CalQL).\n If True, calibrates Q-values by taking the maximum of computed Q-values and Monte Carlo returns.\n This modification helps address the excessive pessimism problem in standard CQL.\n Particularly useful for offline pre-training followed by online fine-tuning scenarios.\n Experimental results suggest this approach often achieves better performance than vanilla CQL.\n Based on techniques from the CalQL paper (arXiv:2303.05479).\n \"\"\"\n super().__init__(\n policy=policy,\n )\n LaggedNetworkPolyakUpdateAlgorithmMixin.__init__(self, tau=tau)\n\n device = torch_device(policy)\n\n self.policy_optim = self._create_optimizer(self.policy, policy_optim)\n self.critic = critic\n self.critic_optim = self._create_optimizer(\n self.critic, critic_optim, max_grad_norm=max_grad_norm\n )\n self.critic2 = critic2 or deepcopy(critic)\n self.critic2_optim = self._create_optimizer(\n self.critic2, critic2_optim or critic_optim, max_grad_norm=max_grad_norm\n )\n self.critic_old = self._add_lagged_network(self.critic)\n self.critic2_old = self._add_lagged_network(self.critic2)\n\n self.gamma = gamma\n self.alpha = Alpha.from_float_or_instance(alpha)\n\n self.temperature = temperature\n self.with_lagrange = with_lagrange\n self.lagrange_threshold = lagrange_threshold\n\n self.cql_weight = cql_weight\n\n self.cql_log_alpha = torch.tensor([0.0], requires_grad=True)\n # TODO: Use an OptimizerFactory?\n self.cql_alpha_optim = torch.optim.Adam([self.cql_log_alpha], lr=cql_alpha_lr)\n self.cql_log_alpha = self.cql_log_alpha.to(device)\n\n self.min_action = min_action\n self.max_action = max_action\n\n self.num_repeat_actions = num_repeat_actions\n\n self.alpha_min = alpha_min\n self.alpha_max = alpha_max\n\n self.calibrated = calibrated\n\n def _policy_pred(self, obs: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:\n batch = Batch(obs=obs, info=[None] * len(obs))\n obs_result = self.policy(batch)\n return obs_result.act, obs_result.log_prob\n\n def _calc_policy_loss(self, obs: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:\n act_pred, log_pi = self._policy_pred(obs)\n q1 = self.critic(obs, act_pred)\n q2 = self.critic2(obs, act_pred)\n min_Q = torch.min(q1, q2)\n # self.alpha: float | torch.Tensor\n actor_loss = (self.alpha.value * log_pi - min_Q).mean()\n # actor_loss.shape: (), log_pi.shape: (batch_size, 1)\n return actor_loss, log_pi\n\n def _calc_pi_values(\n self,\n obs_pi: torch.Tensor,\n obs_to_pred: torch.Tensor,\n ) -> tuple[torch.Tensor, torch.Tensor]:\n act_pred, log_pi = self._policy_pred(obs_pi)\n\n q1 = self.critic(obs_to_pred, act_pred)\n q2 = self.critic2(obs_to_pred, act_pred)\n\n return q1 - log_pi.detach(), q2 - log_pi.detach()\n\n def _calc_random_values(\n self,\n obs: torch.Tensor,\n act: torch.Tensor,\n ) -> tuple[torch.Tensor, torch.Tensor]:\n random_value1 = self.critic(obs, act)\n random_log_prob1 = np.log(0.5 ** act.shape[-1])\n\n random_value2 = self.critic2(obs, act)\n random_log_prob2 = np.log(0.5 ** act.shape[-1])\n\n return random_value1 - random_log_prob1, random_value2 - random_log_prob2\n\n @override\n def process_buffer(self, buffer: TBuffer) -> TBuffer:\n \"\"\"If `self.calibrated = True`, adds `calibration_returns` to buffer._meta.\n\n :param buffer:\n :return:\n \"\"\"\n if self.calibrated:\n # otherwise _meta hack cannot work\n assert isinstance(buffer, ReplayBuffer)\n batch, indices = buffer.sample(0)\n returns, _ = self.compute_episodic_return(\n batch=batch,\n buffer=buffer,\n indices=indices,\n gamma=self.gamma,\n gae_lambda=1.0,\n )\n # TODO: don't access _meta directly\n buffer._meta = cast(\n RolloutBatchProtocol,\n Batch(**buffer._meta.__dict__, calibration_returns=returns),\n )\n return buffer\n\n def _update_with_batch(self, batch: RolloutBatchProtocol) -> CQLTrainingStats:\n device = torch_device(self.policy)\n batch: Batch = to_torch(batch, dtype=torch.float, device=device)\n obs, act, rew, obs_next = batch.obs, batch.act, batch.rew, batch.obs_next\n batch_size = obs.shape[0]\n\n # compute actor loss and update actor\n actor_loss, log_pi = self._calc_policy_loss(obs)\n self.policy_optim.step(actor_loss)\n\n entropy = -log_pi.detach()\n alpha_loss = self.alpha.update(entropy)\n\n # compute target_Q\n with torch.no_grad():\n act_next, new_log_pi = self._policy_pred(obs_next)\n\n target_Q1 = self.critic_old(obs_next, act_next)\n target_Q2 = self.critic2_old(obs_next, act_next)\n\n target_Q = torch.min(target_Q1, target_Q2) - self.alpha.value * new_log_pi\n\n target_Q = rew + torch.logical_not(batch.done) * self.gamma * target_Q.flatten()\n target_Q = target_Q.float()\n # shape: (batch_size)\n\n # compute critic loss\n current_Q1 = self.critic(obs, act).flatten()\n current_Q2 = self.critic2(obs, act).flatten()\n # shape: (batch_size)\n\n critic1_loss = F.mse_loss(current_Q1, target_Q)\n critic2_loss = F.mse_loss(current_Q2, target_Q)\n\n # CQL\n random_actions = (\n torch.FloatTensor(batch_size * self.num_repeat_actions, act.shape[-1])\n .uniform_(-self.min_action, self.max_action)\n .to(device)\n )\n\n obs_len = len(obs.shape)\n repeat_size = [1, self.num_repeat_actions] + [1] * (obs_len - 1)\n view_size = [batch_size * self.num_repeat_actions, *list(obs.shape[1:])]\n tmp_obs = obs.unsqueeze(1).repeat(*repeat_size).view(*view_size)\n tmp_obs_next = obs_next.unsqueeze(1).repeat(*repeat_size).view(*view_size)\n # tmp_obs & tmp_obs_next: (batch_size * num_repeat, state_dim)\n\n current_pi_value1, current_pi_value2 = self._calc_pi_values(tmp_obs, tmp_obs)\n next_pi_value1, next_pi_value2 = self._calc_pi_values(tmp_obs_next, tmp_obs)\n\n random_value1, random_value2 = self._calc_random_values(tmp_obs, random_actions)\n\n for value in [\n current_pi_value1,\n current_pi_value2,\n next_pi_value1,\n next_pi_value2,\n random_value1,\n random_value2,\n ]:\n value.reshape(batch_size, self.num_repeat_actions, 1)\n\n if self.calibrated:\n returns = (\n batch.calibration_returns.unsqueeze(1)\n .repeat(\n (1, self.num_repeat_actions),\n )\n .view(-1, 1)\n )\n random_value1 = torch.max(random_value1, returns)\n random_value2 = torch.max(random_value2, returns)\n\n current_pi_value1 = torch.max(current_pi_value1, returns)\n current_pi_value2 = torch.max(current_pi_value2, returns)\n\n next_pi_value1 = torch.max(next_pi_value1, returns)\n next_pi_value2 = torch.max(next_pi_value2, returns)\n\n # cat q values\n cat_q1 = torch.cat([random_value1, current_pi_value1, next_pi_value1], 1)\n cat_q2 = torch.cat([random_value2, current_pi_value2, next_pi_value2], 1)\n # shape: (batch_size, 3 * num_repeat, 1)\n\n cql1_scaled_loss = (\n torch.logsumexp(cat_q1 / self.temperature, dim=1).mean()\n * self.cql_weight\n * self.temperature\n - current_Q1.mean() * self.cql_weight\n )\n cql2_scaled_loss = (\n torch.logsumexp(cat_q2 / self.temperature, dim=1).mean()\n * self.cql_weight\n * self.temperature\n - current_Q2.mean() * self.cql_weight\n )\n # shape: (1)\n\n cql_alpha_loss = None\n cql_alpha = None\n if self.with_lagrange:\n cql_alpha = torch.clamp(\n self.cql_log_alpha.exp(),\n self.alpha_min,\n self.alpha_max,\n )\n cql1_scaled_loss = cql_alpha * (cql1_scaled_loss - self.lagrange_threshold)\n cql2_scaled_loss = cql_alpha * (cql2_scaled_loss - self.lagrange_threshold)\n\n self.cql_alpha_optim.zero_grad()\n cql_alpha_loss = -(cql1_scaled_loss + cql2_scaled_loss) * 0.5\n cql_alpha_loss.backward(retain_graph=True)\n self.cql_alpha_optim.step()\n\n critic1_loss = critic1_loss + cql1_scaled_loss\n critic2_loss = critic2_loss + cql2_scaled_loss\n\n # update critics\n self.critic_optim.step(critic1_loss, retain_graph=True)\n self.critic2_optim.step(critic2_loss)\n\n self._update_lagged_network_weights()\n\n return CQLTrainingStats(\n actor_loss=to_optional_float(actor_loss),\n critic1_loss=to_optional_float(critic1_loss),\n critic2_loss=to_optional_float(critic2_loss),\n alpha=to_optional_float(self.alpha.value),\n alpha_loss=to_optional_float(alpha_loss),\n cql_alpha_loss=to_optional_float(cql_alpha_loss),\n cql_alpha=to_optional_float(cql_alpha),\n )\n" }, { "path": "tianshou/algorithm/imitation/discrete_bcq.py", "content": "import math\nfrom dataclasses import dataclass\nfrom typing import Any, cast\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nimport torch.nn.functional as F\nfrom torch import nn\n\nfrom tianshou.algorithm.algorithm_base import (\n LaggedNetworkFullUpdateAlgorithmMixin,\n OfflineAlgorithm,\n)\nfrom tianshou.algorithm.modelfree.dqn import DiscreteQLearningPolicy\nfrom tianshou.algorithm.modelfree.reinforce import SimpleLossTrainingStats\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import Batch, ReplayBuffer, to_torch\nfrom tianshou.data.types import (\n BatchWithReturnsProtocol,\n ImitationBatchProtocol,\n ObsBatchProtocol,\n RolloutBatchProtocol,\n)\n\nfloat_info = torch.finfo(torch.float32)\nINF = float_info.max\n\n\n@dataclass(kw_only=True)\nclass DiscreteBCQTrainingStats(SimpleLossTrainingStats):\n q_loss: float\n i_loss: float\n reg_loss: float\n\n\nclass DiscreteBCQPolicy(DiscreteQLearningPolicy):\n def __init__(\n self,\n *,\n model: torch.nn.Module,\n imitator: torch.nn.Module,\n target_update_freq: int = 8000,\n unlikely_action_threshold: float = 0.3,\n action_space: gym.spaces.Discrete,\n observation_space: gym.Space | None = None,\n eps_inference: float = 0.0,\n ) -> None:\n \"\"\"\n :param model: a model following the rules (s_B -> action_values_BA)\n :param imitator: a model following the rules (s -> imitation_logits)\n :param target_update_freq: the number of training iterations between each complete update of\n the target network.\n Controls how frequently the target Q-network parameters are updated with the current\n Q-network values.\n A value of 0 disables the target network entirely, using only a single network for both\n action selection and bootstrap targets.\n Higher values provide more stable learning targets but slow down the propagation of new\n value estimates. Lower positive values allow faster learning but may lead to instability\n due to rapidly changing targets.\n Typically set between 100-10000 for DQN variants, with exact values depending on environment\n complexity.\n :param unlikely_action_threshold: the threshold (tau) for unlikely\n actions, as shown in Equ. (17) in the paper.\n :param target_update_freq: the number of training iterations between each complete update of\n the target network.\n Controls how frequently the target Q-network parameters are updated with the current\n Q-network values.\n A value of 0 disables the target network entirely, using only a single network for both\n action selection and bootstrap targets.\n Higher values provide more stable learning targets but slow down the propagation of new\n value estimates. Lower positive values allow faster learning but may lead to instability\n due to rapidly changing targets.\n Typically set between 100-10000 for DQN variants, with exact values depending on environment\n complexity.\n :param action_space: the environment's action space.\n :param observation_space: the environment's observation space.\n :param eps_inference: the epsilon value for epsilon-greedy exploration during inference,\n i.e. non-training cases (such as evaluation during test steps).\n The epsilon value is the probability of choosing a random action instead of the action\n chosen by the policy.\n A value of 0.0 means no exploration (fully greedy) and a value of 1.0 means full\n exploration (fully random).\n \"\"\"\n super().__init__(\n model=model,\n action_space=action_space,\n observation_space=observation_space,\n eps_training=0.0, # no training data collection (offline)\n eps_inference=eps_inference,\n )\n self.imitator = imitator\n assert target_update_freq > 0, (\n f\"BCQ needs target_update_freq>0 but got: {target_update_freq}.\"\n )\n assert 0.0 <= unlikely_action_threshold < 1.0, (\n f\"unlikely_action_threshold should be in [0, 1) but got: {unlikely_action_threshold}\"\n )\n if unlikely_action_threshold > 0:\n self._log_tau = math.log(unlikely_action_threshold)\n else:\n self._log_tau = -np.inf\n\n def forward(\n self,\n batch: ObsBatchProtocol,\n state: Any | None = None,\n model: nn.Module | None = None,\n ) -> ImitationBatchProtocol:\n if model is None:\n model = self.model\n q_value, state = model(batch.obs, state=state, info=batch.info)\n imitation_logits, _ = self.imitator(batch.obs, state=state, info=batch.info)\n\n # mask actions for argmax\n ratio = imitation_logits - imitation_logits.max(dim=-1, keepdim=True).values\n mask = (ratio < self._log_tau).float()\n act = (q_value - INF * mask).argmax(dim=-1)\n\n result = Batch(\n act=act,\n state=state,\n q_value=q_value,\n imitation_logits=imitation_logits,\n logits=imitation_logits,\n )\n return cast(ImitationBatchProtocol, result)\n\n\nclass DiscreteBCQ(\n OfflineAlgorithm[DiscreteBCQPolicy],\n LaggedNetworkFullUpdateAlgorithmMixin,\n):\n \"\"\"Implementation of the discrete batch-constrained deep Q-learning (BCQ) algorithm. arXiv:1910.01708.\"\"\"\n\n def __init__(\n self,\n *,\n policy: DiscreteBCQPolicy,\n optim: OptimizerFactory,\n gamma: float = 0.99,\n n_step_return_horizon: int = 1,\n target_update_freq: int = 8000,\n imitation_logits_penalty: float = 1e-2,\n ) -> None:\n \"\"\"\n :param policy: the policy\n :param optim: the optimizer factory for the policy's model.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param n_step_return_horizon: the number of future steps (> 0) to consider when computing temporal\n difference (TD) targets. Controls the balance between TD learning and Monte Carlo methods:\n higher values reduce bias (by relying less on potentially inaccurate value estimates)\n but increase variance (by incorporating more environmental stochasticity and reducing\n the averaging effect). A value of 1 corresponds to standard TD learning with immediate\n bootstrapping, while very large values approach Monte Carlo-like estimation that uses\n complete episode returns.\n :param target_update_freq: the number of training iterations between each complete update of\n the target network.\n Controls how frequently the target Q-network parameters are updated with the current\n Q-network values.\n A value of 0 disables the target network entirely, using only a single network for both\n action selection and bootstrap targets.\n Higher values provide more stable learning targets but slow down the propagation of new\n value estimates. Lower positive values allow faster learning but may lead to instability\n due to rapidly changing targets.\n Typically set between 100-10000 for DQN variants, with exact values depending on environment\n complexity.\n :param imitation_logits_penalty: regularization weight for imitation\n logits.\n :param n_step_return_horizon: the number of future steps (> 0) to consider when computing temporal\n difference (TD) targets. Controls the balance between TD learning and Monte Carlo methods:\n higher values reduce bias (by relying less on potentially inaccurate value estimates)\n but increase variance (by incorporating more environmental stochasticity and reducing\n the averaging effect). A value of 1 corresponds to standard TD learning with immediate\n bootstrapping, while very large values approach Monte Carlo-like estimation that uses\n complete episode returns.\n :param target_update_freq: the number of training iterations between each complete update of\n the target network.\n Controls how frequently the target Q-network parameters are updated with the current\n Q-network values.\n A value of 0 disables the target network entirely, using only a single network for both\n action selection and bootstrap targets.\n Higher values provide more stable learning targets but slow down the propagation of new\n value estimates. Lower positive values allow faster learning but may lead to instability\n due to rapidly changing targets.\n Typically set between 100-10000 for DQN variants, with exact values depending on environment\n complexity.\n \"\"\"\n super().__init__(\n policy=policy,\n )\n LaggedNetworkFullUpdateAlgorithmMixin.__init__(self)\n self.optim = self._create_optimizer(self.policy, optim)\n assert 0.0 <= gamma <= 1.0, f\"discount factor should be in [0, 1] but got: {gamma}\"\n self.gamma = gamma\n assert n_step_return_horizon > 0, (\n f\"n_step_return_horizon should be greater than 0 but got: {n_step_return_horizon}\"\n )\n self.n_step = n_step_return_horizon\n self._target = target_update_freq > 0\n self.freq = target_update_freq\n self._iter = 0\n if self._target:\n self.model_old = self._add_lagged_network(self.policy.model)\n self._weight_reg = imitation_logits_penalty\n\n def _preprocess_batch(\n self,\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n ) -> BatchWithReturnsProtocol:\n return self.compute_nstep_return(\n batch=batch,\n buffer=buffer,\n indices=indices,\n target_q_fn=self._target_q,\n gamma=self.gamma,\n n_step=self.n_step,\n )\n\n def _target_q(self, buffer: ReplayBuffer, indices: np.ndarray) -> torch.Tensor:\n batch = buffer[indices] # batch.obs_next: s_{t+n}\n next_obs_batch = Batch(obs=batch.obs_next, info=[None] * len(batch))\n # target_Q = Q_old(s_, argmax(Q_new(s_, *)))\n act = self.policy(next_obs_batch).act\n target_q, _ = self.model_old(batch.obs_next)\n return target_q[np.arange(len(act)), act]\n\n def _update_with_batch( # type: ignore[override]\n self,\n batch: BatchWithReturnsProtocol,\n ) -> DiscreteBCQTrainingStats:\n if self._iter % self.freq == 0:\n self._update_lagged_network_weights()\n self._iter += 1\n\n target_q = batch.returns.flatten()\n result = self.policy(batch)\n imitation_logits = result.imitation_logits\n current_q = result.q_value[np.arange(len(target_q)), batch.act]\n act = to_torch(batch.act, dtype=torch.long, device=target_q.device)\n q_loss = F.smooth_l1_loss(current_q, target_q)\n i_loss = F.nll_loss(F.log_softmax(imitation_logits, dim=-1), act)\n reg_loss = imitation_logits.pow(2).mean()\n loss = q_loss + i_loss + self._weight_reg * reg_loss\n\n self.optim.step(loss)\n\n return DiscreteBCQTrainingStats(\n loss=loss.item(),\n q_loss=q_loss.item(),\n i_loss=i_loss.item(),\n reg_loss=reg_loss.item(),\n )\n" }, { "path": "tianshou/algorithm/imitation/discrete_cql.py", "content": "from dataclasses import dataclass\n\nimport numpy as np\nimport torch\nimport torch.nn.functional as F\n\nfrom tianshou.algorithm import QRDQN\nfrom tianshou.algorithm.algorithm_base import OfflineAlgorithm\nfrom tianshou.algorithm.modelfree.qrdqn import QRDQNPolicy\nfrom tianshou.algorithm.modelfree.reinforce import SimpleLossTrainingStats\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import to_torch\nfrom tianshou.data.types import RolloutBatchProtocol\n\n\n@dataclass(kw_only=True)\nclass DiscreteCQLTrainingStats(SimpleLossTrainingStats):\n cql_loss: float\n qr_loss: float\n\n\n# NOTE: This uses diamond inheritance to convert from off-policy to offline\nclass DiscreteCQL(OfflineAlgorithm[QRDQNPolicy], QRDQN[QRDQNPolicy]): # type: ignore[misc]\n \"\"\"Implementation of discrete Conservative Q-Learning algorithm. arXiv:2006.04779.\"\"\"\n\n def __init__(\n self,\n *,\n policy: QRDQNPolicy,\n optim: OptimizerFactory,\n min_q_weight: float = 10.0,\n gamma: float = 0.99,\n num_quantiles: int = 200,\n n_step_return_horizon: int = 1,\n target_update_freq: int = 0,\n ) -> None:\n \"\"\"\n :param policy: the policy\n :param optim: the optimizer factory for the policy's model.\n :param min_q_weight: the weight for the cql loss.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param num_quantiles: the number of quantile midpoints in the inverse\n cumulative distribution function of the value.\n :param n_step_return_horizon: the number of future steps (> 0) to consider when computing temporal\n difference (TD) targets. Controls the balance between TD learning and Monte Carlo methods:\n higher values reduce bias (by relying less on potentially inaccurate value estimates)\n but increase variance (by incorporating more environmental stochasticity and reducing\n the averaging effect). A value of 1 corresponds to standard TD learning with immediate\n bootstrapping, while very large values approach Monte Carlo-like estimation that uses\n complete episode returns.\n :param target_update_freq: the number of training iterations between each complete update of\n the target network.\n Controls how frequently the target Q-network parameters are updated with the current\n Q-network values.\n A value of 0 disables the target network entirely, using only a single network for both\n action selection and bootstrap targets.\n Higher values provide more stable learning targets but slow down the propagation of new\n value estimates. Lower positive values allow faster learning but may lead to instability\n due to rapidly changing targets.\n Typically set between 100-10000 for DQN variants, with exact values depending on environment\n complexity.\n \"\"\"\n QRDQN.__init__(\n self,\n policy=policy,\n optim=optim,\n gamma=gamma,\n num_quantiles=num_quantiles,\n n_step_return_horizon=n_step_return_horizon,\n target_update_freq=target_update_freq,\n )\n self.min_q_weight = min_q_weight\n\n def _update_with_batch(\n self,\n batch: RolloutBatchProtocol,\n ) -> DiscreteCQLTrainingStats:\n self._periodically_update_lagged_network_weights()\n weight = batch.pop(\"weight\", 1.0)\n all_dist = self.policy(batch).logits\n act = to_torch(batch.act, dtype=torch.long, device=all_dist.device)\n curr_dist = all_dist[np.arange(len(act)), act, :].unsqueeze(2)\n target_dist = batch.returns.unsqueeze(1)\n # calculate each element's difference between curr_dist and target_dist\n dist_diff = F.smooth_l1_loss(target_dist, curr_dist, reduction=\"none\")\n huber_loss = (\n (dist_diff * (self.tau_hat - (target_dist - curr_dist).detach().le(0.0).float()).abs())\n .sum(-1)\n .mean(1)\n )\n qr_loss = (huber_loss * weight).mean()\n # ref: https://github.com/ku2482/fqf-iqn-qrdqn.pytorch/\n # blob/master/fqf_iqn_qrdqn/agent/qrdqn_agent.py L130\n batch.weight = dist_diff.detach().abs().sum(-1).mean(1) # prio-buffer\n # add CQL loss\n q = self.policy.compute_q_value(all_dist, None)\n dataset_expec = q.gather(1, act.unsqueeze(1)).mean()\n negative_sampling = q.logsumexp(1).mean()\n min_q_loss = negative_sampling - dataset_expec\n loss = qr_loss + min_q_loss * self.min_q_weight\n self.optim.step(loss)\n\n return DiscreteCQLTrainingStats(\n loss=loss.item(),\n qr_loss=qr_loss.item(),\n cql_loss=min_q_loss.item(),\n )\n" }, { "path": "tianshou/algorithm/imitation/discrete_crr.py", "content": "from dataclasses import dataclass\nfrom typing import Literal\n\nimport numpy as np\nimport torch\nimport torch.nn.functional as F\nfrom torch.distributions import Categorical\nfrom torch.nn import ModuleList\n\nfrom tianshou.algorithm.algorithm_base import (\n LaggedNetworkFullUpdateAlgorithmMixin,\n OfflineAlgorithm,\n)\nfrom tianshou.algorithm.modelfree.reinforce import (\n DiscountedReturnComputation,\n DiscreteActorPolicy,\n SimpleLossTrainingStats,\n)\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import ReplayBuffer, to_torch, to_torch_as\nfrom tianshou.data.types import BatchWithReturnsProtocol, RolloutBatchProtocol\nfrom tianshou.utils.lagged_network import EvalModeModuleWrapper\nfrom tianshou.utils.net.discrete import DiscreteCritic\n\n\n@dataclass\nclass DiscreteCRRTrainingStats(SimpleLossTrainingStats):\n actor_loss: float\n critic_loss: float\n cql_loss: float\n\n\nclass DiscreteCRR(\n OfflineAlgorithm[DiscreteActorPolicy],\n LaggedNetworkFullUpdateAlgorithmMixin,\n):\n r\"\"\"Implementation of discrete Critic Regularized Regression. arXiv:2006.15134.\"\"\"\n\n def __init__(\n self,\n *,\n policy: DiscreteActorPolicy,\n critic: torch.nn.Module | DiscreteCritic,\n optim: OptimizerFactory,\n gamma: float = 0.99,\n policy_improvement_mode: Literal[\"exp\", \"binary\", \"all\"] = \"exp\",\n ratio_upper_bound: float = 20.0,\n beta: float = 1.0,\n min_q_weight: float = 10.0,\n target_update_freq: int = 0,\n return_standardization: bool = False,\n ) -> None:\n r\"\"\"\n :param policy: the policy\n :param critic: the action-value critic (i.e., Q function)\n network. (s -> Q(s, \\*))\n :param optim: the optimizer factory for the policy's actor network and the critic networks.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param str policy_improvement_mode: type of the weight function f. Possible\n values: \"binary\"/\"exp\"/\"all\".\n :param ratio_upper_bound: when policy_improvement_mode is \"exp\", the value\n of the exp function is upper-bounded by this parameter.\n :param beta: when policy_improvement_mode is \"exp\", this is the denominator\n of the exp function.\n :param min_q_weight: weight for CQL loss/regularizer. Default to 10.\n :param target_update_freq: the number of training iterations between each complete update of\n the target network.\n Controls how frequently the target Q-network parameters are updated with the current\n Q-network values.\n A value of 0 disables the target network entirely, using only a single network for both\n action selection and bootstrap targets.\n Higher values provide more stable learning targets but slow down the propagation of new\n value estimates. Lower positive values allow faster learning but may lead to instability\n due to rapidly changing targets.\n Typically set between 100-10000 for DQN variants, with exact values depending on environment\n complexity.\n :param return_standardization: whether to standardize episode returns\n by subtracting the running mean and dividing by the running standard deviation.\n Note that this is known to be detrimental to performance in many cases!\n \"\"\"\n super().__init__(\n policy=policy,\n )\n LaggedNetworkFullUpdateAlgorithmMixin.__init__(self)\n self.discounted_return_computation = DiscountedReturnComputation(\n gamma=gamma,\n return_standardization=return_standardization,\n )\n self.critic = critic\n self.optim = self._create_optimizer(ModuleList([self.policy, self.critic]), optim)\n self._target = target_update_freq > 0\n self._freq = target_update_freq\n self._iter = 0\n self.actor_old: torch.nn.Module | EvalModeModuleWrapper\n self.critic_old: torch.nn.Module | EvalModeModuleWrapper\n if self._target:\n self.actor_old = self._add_lagged_network(self.policy.actor)\n self.critic_old = self._add_lagged_network(self.critic)\n else:\n self.actor_old = self.policy.actor\n self.critic_old = self.critic\n self._policy_improvement_mode = policy_improvement_mode\n self._ratio_upper_bound = ratio_upper_bound\n self._beta = beta\n self._min_q_weight = min_q_weight\n\n def _preprocess_batch(\n self,\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n ) -> BatchWithReturnsProtocol:\n return self.discounted_return_computation.add_discounted_returns(\n batch,\n buffer,\n indices,\n )\n\n def _update_with_batch( # type: ignore[override]\n self,\n batch: BatchWithReturnsProtocol,\n ) -> DiscreteCRRTrainingStats:\n if self._target and self._iter % self._freq == 0:\n self._update_lagged_network_weights()\n q_t = self.critic(batch.obs)\n act = to_torch(batch.act, dtype=torch.long, device=q_t.device)\n qa_t = q_t.gather(1, act.unsqueeze(1))\n # Critic loss\n with torch.no_grad():\n target_a_t, _ = self.actor_old(batch.obs_next)\n target_m = Categorical(logits=target_a_t)\n q_t_target = self.critic_old(batch.obs_next)\n rew = to_torch_as(batch.rew, q_t_target)\n expected_target_q = (q_t_target * target_m.probs).sum(-1, keepdim=True)\n expected_target_q[batch.done > 0] = 0.0\n target = rew.unsqueeze(1) + self.discounted_return_computation.gamma * expected_target_q\n critic_loss = 0.5 * F.mse_loss(qa_t, target)\n # Actor loss\n act_target, _ = self.policy.actor(batch.obs)\n dist = Categorical(logits=act_target)\n expected_policy_q = (q_t * dist.probs).sum(-1, keepdim=True)\n advantage = qa_t - expected_policy_q\n if self._policy_improvement_mode == \"binary\":\n actor_loss_coef = (advantage > 0).float()\n elif self._policy_improvement_mode == \"exp\":\n actor_loss_coef = (advantage / self._beta).exp().clamp(0, self._ratio_upper_bound)\n else:\n actor_loss_coef = 1.0 # effectively behavior cloning\n actor_loss = (-dist.log_prob(act) * actor_loss_coef).mean()\n # CQL loss/regularizer\n min_q_loss = (q_t.logsumexp(1) - qa_t).mean()\n loss = actor_loss + critic_loss + self._min_q_weight * min_q_loss\n self.optim.step(loss)\n self._iter += 1\n\n return DiscreteCRRTrainingStats(\n loss=loss.item(),\n actor_loss=actor_loss.item(),\n critic_loss=critic_loss.item(),\n cql_loss=min_q_loss.item(),\n )\n" }, { "path": "tianshou/algorithm/imitation/gail.py", "content": "from dataclasses import dataclass\n\nimport numpy as np\nimport torch\nimport torch.nn.functional as F\n\nfrom tianshou.algorithm.modelfree.a2c import A2CTrainingStats\nfrom tianshou.algorithm.modelfree.ppo import PPO\nfrom tianshou.algorithm.modelfree.reinforce import ProbabilisticActorPolicy\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import (\n ReplayBuffer,\n SequenceSummaryStats,\n to_numpy,\n to_torch,\n)\nfrom tianshou.data.types import LogpOldProtocol, RolloutBatchProtocol\nfrom tianshou.utils.net.common import ModuleWithVectorOutput\nfrom tianshou.utils.net.continuous import ContinuousCritic\nfrom tianshou.utils.net.discrete import DiscreteCritic\nfrom tianshou.utils.torch_utils import torch_device\n\n\n@dataclass(kw_only=True)\nclass GailTrainingStats(A2CTrainingStats):\n disc_loss: SequenceSummaryStats\n acc_pi: SequenceSummaryStats\n acc_exp: SequenceSummaryStats\n\n\nclass GAIL(PPO):\n \"\"\"Implementation of Generative Adversarial Imitation Learning. arXiv:1606.03476.\"\"\"\n\n def __init__(\n self,\n *,\n policy: ProbabilisticActorPolicy,\n critic: torch.nn.Module | ContinuousCritic | DiscreteCritic,\n optim: OptimizerFactory,\n expert_buffer: ReplayBuffer,\n disc_net: torch.nn.Module,\n disc_optim: OptimizerFactory,\n disc_update_num: int = 4,\n eps_clip: float = 0.2,\n dual_clip: float | None = None,\n value_clip: bool = False,\n advantage_normalization: bool = True,\n recompute_advantage: bool = False,\n vf_coef: float = 0.5,\n ent_coef: float = 0.01,\n max_grad_norm: float | None = None,\n gae_lambda: float = 0.95,\n max_batchsize: int = 256,\n gamma: float = 0.99,\n return_scaling: bool = False,\n ) -> None:\n \"\"\"\n :param policy: the policy (which must use an actor with known output dimension, i.e.\n any Tianshou `Actor` implementation or other subclass of `ModuleWithVectorOutput`).\n :param critic: the critic network. (s -> V(s))\n :param optim: the optimizer factory for the actor and critic networks.\n :param expert_buffer: the replay buffer containing expert experience.\n :param disc_net: the discriminator neural network that distinguishes between expert and policy behaviors.\n Takes concatenated state-action pairs [obs, act] as input and outputs an unbounded logit value.\n The raw output is transformed in the algorithm using sigmoid functions: o(output) for expert\n probability and -log(1-o(-output)) for policy rewards.\n Positive output values indicate the discriminator believes the behavior is from an expert.\n Negative output values indicate the discriminator believes the behavior is from the policy.\n The network architecture should end with a linear layer of output size 1 without any\n activation function, as sigmoid operations are applied separately.\n :param disc_optim: the optimizer factory for the discriminator network.\n :param disc_update_num: the number of discriminator update steps performed for each policy update step.\n Controls the learning dynamics between the policy and the discriminator.\n Higher values strengthen the discriminator relative to the policy, potentially improving\n the quality of the reward signal but slowing down training.\n Lower values allow faster policy updates but may result in a weaker discriminator that fails\n to properly distinguish between expert and policy behaviors.\n Typical values range from 1 to 10, with the original GAIL paper using multiple discriminator\n updates per policy update.\n :param eps_clip: determines the range of allowed change in the policy during a policy update:\n The ratio of action probabilities indicated by the new and old policy is\n constrained to stay in the interval [1 - eps_clip, 1 + eps_clip].\n Small values thus force the new policy to stay close to the old policy.\n Typical values range between 0.1 and 0.3, the value of 0.2 is recommended\n in the original PPO paper.\n The optimal value depends on the environment; more stochastic environments may\n need larger values.\n :param dual_clip: a clipping parameter (denoted as c in the literature) that prevents\n excessive pessimism in policy updates for negative-advantage actions.\n Excessive pessimism occurs when the policy update too strongly reduces the probability\n of selecting actions that led to negative advantages, potentially eliminating useful\n actions based on limited negative experiences.\n When enabled (c > 1), the objective for negative advantages becomes:\n max(min(r(θ)A, clip(r(θ), 1-ε, 1+ε)A), c*A), where min(r(θ)A, clip(r(θ), 1-ε, 1+ε)A)\n is the original single-clipping objective determined by `eps_clip`.\n This creates a floor on negative policy gradients, maintaining some probability\n of exploring actions despite initial negative outcomes.\n Larger values (e.g., 2.0 to 5.0) maintain more exploration, while values closer\n to 1.0 provide less protection against pessimistic updates.\n Set to None to disable dual clipping.\n :param value_clip: flag indicating whether to enable clipping for value function updates.\n When enabled, restricts how much the value function estimate can change from its\n previous prediction, using the same clipping range as the policy updates (eps_clip).\n This stabilizes training by preventing large fluctuations in value estimates,\n particularly useful in environments with high reward variance.\n The clipped value loss uses a pessimistic approach, taking the maximum of the\n original and clipped value errors:\n max((returns - value)², (returns - v_clipped)²)\n Setting to True often improves training stability but may slow convergence.\n Implementation follows the approach mentioned in arXiv:1811.02553v3 Sec. 4.1.\n :param advantage_normalization: whether to do per mini-batch advantage\n normalization.\n :param recompute_advantage: whether to recompute advantage every update\n repeat according to https://arxiv.org/pdf/2006.05990.pdf Sec. 3.5.\n :param vf_coef: coefficient that weights the value loss relative to the actor loss in\n the overall loss function.\n Higher values prioritize accurate value function estimation over policy improvement.\n Controls the trade-off between policy optimization and value function fitting.\n Typically set between 0.5 and 1.0 for most actor-critic implementations.\n :param ent_coef: coefficient that weights the entropy bonus relative to the actor loss.\n Controls the exploration-exploitation trade-off by encouraging policy entropy.\n Higher values promote more exploration by encouraging a more uniform action distribution.\n Lower values focus more on exploitation of the current policy's knowledge.\n Typically set between 0.01 and 0.05 for most actor-critic implementations.\n :param max_grad_norm: the maximum L2 norm threshold for gradient clipping.\n When not None, gradients will be rescaled using to ensure their L2 norm does not\n exceed this value. This prevents exploding gradients and stabilizes training by\n limiting the size of parameter updates.\n Set to None to disable gradient clipping.\n :param gae_lambda: the lambda parameter in [0, 1] for generalized advantage estimation (GAE).\n Controls the bias-variance tradeoff in advantage estimates, acting as a\n weighting factor for combining different n-step advantage estimators. Higher values\n (closer to 1) reduce bias but increase variance by giving more weight to longer\n trajectories, while lower values (closer to 0) reduce variance but increase bias\n by relying more on the immediate TD error and value function estimates. At λ=0,\n GAE becomes equivalent to the one-step TD error (high bias, low variance); at λ=1,\n it becomes equivalent to Monte Carlo advantage estimation (low bias, high variance).\n Intermediate values create a weighted average of n-step returns, with exponentially\n decaying weights for longer-horizon returns. Typically set between 0.9 and 0.99 for\n most policy gradient methods.\n :param max_batchsize: the maximum number of samples to process at once when computing\n generalized advantage estimation (GAE) and value function predictions.\n Controls memory usage by breaking large batches into smaller chunks processed sequentially.\n Higher values may increase speed but require more GPU/CPU memory; lower values\n reduce memory requirements but may increase computation time. Should be adjusted\n based on available hardware resources and total batch size of your training data.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param return_scaling: flag indicating whether to enable scaling of estimated returns by\n dividing them by their running standard deviation without centering the mean.\n This reduces the magnitude variation of advantages across different episodes while\n preserving their signs and relative ordering.\n The use of running statistics (rather than batch-specific scaling) means that early\n training experiences may be scaled differently than later ones as the statistics evolve.\n When enabled, this improves training stability in environments with highly variable\n reward scales and makes the algorithm less sensitive to learning rate settings.\n However, it may reduce the algorithm's ability to distinguish between episodes with\n different absolute return magnitudes.\n Best used in environments where the relative ordering of actions is more important\n than the absolute scale of returns.\n \"\"\"\n super().__init__(\n policy=policy,\n critic=critic,\n optim=optim,\n eps_clip=eps_clip,\n dual_clip=dual_clip,\n value_clip=value_clip,\n advantage_normalization=advantage_normalization,\n recompute_advantage=recompute_advantage,\n vf_coef=vf_coef,\n ent_coef=ent_coef,\n max_grad_norm=max_grad_norm,\n gae_lambda=gae_lambda,\n max_batchsize=max_batchsize,\n gamma=gamma,\n return_scaling=return_scaling,\n )\n self.disc_net = disc_net\n self.disc_optim = self._create_optimizer(self.disc_net, disc_optim)\n self.disc_update_num = disc_update_num\n self.expert_buffer = expert_buffer\n actor = self.policy.actor\n if not isinstance(actor, ModuleWithVectorOutput):\n raise TypeError(\"GAIL requires the policy to use an actor with known output dimension.\")\n self.action_dim = actor.get_output_dim()\n\n def _preprocess_batch(\n self,\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n ) -> LogpOldProtocol:\n \"\"\"Pre-process the data from the provided replay buffer.\n\n Used in :meth:`update`. Check out :ref:`process_fn` for more information.\n \"\"\"\n # update reward\n with torch.no_grad():\n batch.rew = to_numpy(-F.logsigmoid(-self.disc(batch)).flatten())\n return super()._preprocess_batch(batch, buffer, indices)\n\n def disc(self, batch: RolloutBatchProtocol) -> torch.Tensor:\n device = torch_device(self.disc_net)\n obs = to_torch(batch.obs, device=device)\n act = to_torch(batch.act, device=device)\n return self.disc_net(torch.cat([obs, act], dim=1))\n\n def _update_with_batch( # type: ignore[override]\n self,\n batch: LogpOldProtocol,\n batch_size: int | None,\n repeat: int,\n ) -> GailTrainingStats:\n # update discriminator\n losses = []\n acc_pis = []\n acc_exps = []\n bsz = len(batch) // self.disc_update_num\n for b in batch.split(bsz, merge_last=True):\n logits_pi = self.disc(b)\n exp_b = self.expert_buffer.sample(bsz)[0]\n logits_exp = self.disc(exp_b)\n loss_pi = -F.logsigmoid(-logits_pi).mean()\n loss_exp = -F.logsigmoid(logits_exp).mean()\n loss_disc = loss_pi + loss_exp\n self.disc_optim.step(loss_disc)\n losses.append(loss_disc.item())\n acc_pis.append((logits_pi < 0).float().mean().item())\n acc_exps.append((logits_exp > 0).float().mean().item())\n # update policy\n ppo_loss_stat = super()._update_with_batch(batch, batch_size, repeat)\n\n disc_losses_summary = SequenceSummaryStats.from_sequence(losses)\n acc_pi_summary = SequenceSummaryStats.from_sequence(acc_pis)\n acc_exps_summary = SequenceSummaryStats.from_sequence(acc_exps)\n\n return GailTrainingStats(\n **ppo_loss_stat.__dict__,\n disc_loss=disc_losses_summary,\n acc_pi=acc_pi_summary,\n acc_exp=acc_exps_summary,\n )\n" }, { "path": "tianshou/algorithm/imitation/imitation_base.py", "content": "from dataclasses import dataclass\nfrom typing import Any, Literal, cast\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nimport torch.nn.functional as F\n\nfrom tianshou.algorithm import Algorithm\nfrom tianshou.algorithm.algorithm_base import (\n OfflineAlgorithm,\n OffPolicyAlgorithm,\n Policy,\n TrainingStats,\n)\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import Batch, to_torch\nfrom tianshou.data.batch import BatchProtocol\nfrom tianshou.data.types import (\n ModelOutputBatchProtocol,\n ObsBatchProtocol,\n RolloutBatchProtocol,\n)\n\n# Dimension Naming Convention\n# B - Batch Size\n# A - Action\n# D - Dist input (usually 2, loc and scale)\n# H - Dimension of hidden, can be None\n\n\n@dataclass(kw_only=True)\nclass ImitationTrainingStats(TrainingStats):\n loss: float = 0.0\n\n\nclass ImitationPolicy(Policy):\n def __init__(\n self,\n *,\n actor: torch.nn.Module,\n action_space: gym.Space,\n observation_space: gym.Space | None = None,\n action_scaling: bool = False,\n action_bound_method: Literal[\"clip\", \"tanh\"] | None = \"clip\",\n ):\n \"\"\"\n :param actor: a model following the rules (s -> a)\n :param action_space: the environment's action_space.\n :param observation_space: the environment's observation space\n :param action_scaling: flag indicating whether, for continuous action spaces, actions\n should be scaled from the standard neural network output range [-1, 1] to the\n environment's action space range [action_space.low, action_space.high].\n This applies to continuous action spaces only (gym.spaces.Box) and has no effect\n for discrete spaces.\n When enabled, policy outputs are expected to be in the normalized range [-1, 1]\n (after bounding), and are then linearly transformed to the actual required range.\n This improves neural network training stability, allows the same algorithm to work\n across environments with different action ranges, and standardizes exploration\n strategies.\n Should be disabled if the actor model already produces outputs in the correct range.\n :param action_bound_method: the method used for bounding actions in continuous action spaces\n to the range [-1, 1] before scaling them to the environment's action space (provided\n that `action_scaling` is enabled).\n This applies to continuous action spaces only (`gym.spaces.Box`) and should be set to None\n for discrete spaces.\n When set to \"clip\", actions exceeding the [-1, 1] range are simply clipped to this\n range. When set to \"tanh\", a hyperbolic tangent function is applied, which smoothly\n constrains outputs to [-1, 1] while preserving gradients.\n The choice of bounding method affects both training dynamics and exploration behavior.\n Clipping provides hard boundaries but may create plateau regions in the gradient\n landscape, while tanh provides smoother transitions but can compress sensitivity\n near the boundaries.\n Should be set to None if the actor model inherently produces bounded outputs.\n Typically used together with `action_scaling=True`.\n \"\"\"\n super().__init__(\n action_space=action_space,\n observation_space=observation_space,\n action_scaling=action_scaling,\n action_bound_method=action_bound_method,\n )\n self.actor = actor\n\n def forward(\n self,\n batch: ObsBatchProtocol,\n state: dict | BatchProtocol | np.ndarray | None = None,\n **kwargs: Any,\n ) -> ModelOutputBatchProtocol:\n # TODO - ALGO-REFACTORING: marked for refactoring when Algorithm abstraction is introduced\n if self.action_type == \"discrete\":\n # If it's discrete, the \"actor\" is usually a critic that maps obs to action_values\n # which then could be turned into logits or a Categorigal\n action_values_BA, hidden_BH = self.actor(batch.obs, state=state, info=batch.info)\n act_B = action_values_BA.argmax(dim=1)\n result = Batch(logits=action_values_BA, act=act_B, state=hidden_BH)\n elif self.action_type == \"continuous\":\n # If it's continuous, the actor would usually deliver something like loc, scale determining a\n # Gaussian dist\n dist_input_BD, hidden_BH = self.actor(batch.obs, state=state, info=batch.info)\n result = Batch(logits=dist_input_BD, act=dist_input_BD, state=hidden_BH)\n else:\n raise RuntimeError(f\"Unknown {self.action_type=}, this shouldn't have happened!\")\n return cast(ModelOutputBatchProtocol, result)\n\n\nclass ImitationLearningAlgorithmMixin:\n def _imitation_update(\n self,\n batch: RolloutBatchProtocol,\n policy: ImitationPolicy,\n optim: Algorithm.Optimizer,\n ) -> ImitationTrainingStats:\n if policy.action_type == \"continuous\": # regression\n act = policy(batch).act\n act_target = to_torch(batch.act, dtype=torch.float32, device=act.device)\n loss = F.mse_loss(act, act_target)\n elif policy.action_type == \"discrete\": # classification\n act = F.log_softmax(policy(batch).logits, dim=-1)\n act_target = to_torch(batch.act, dtype=torch.long, device=act.device)\n loss = F.nll_loss(act, act_target)\n else:\n raise ValueError(policy.action_type)\n optim.step(loss)\n\n return ImitationTrainingStats(loss=loss.item())\n\n\nclass OffPolicyImitationLearning(\n OffPolicyAlgorithm[ImitationPolicy],\n ImitationLearningAlgorithmMixin,\n):\n \"\"\"Implementation of off-policy vanilla imitation learning.\"\"\"\n\n def __init__(\n self,\n *,\n policy: ImitationPolicy,\n optim: OptimizerFactory,\n ) -> None:\n \"\"\"\n :param policy: the policy\n :param optim: the optimizer factory\n \"\"\"\n super().__init__(\n policy=policy,\n )\n self.optim = self._create_optimizer(self.policy, optim)\n\n def _update_with_batch(\n self,\n batch: RolloutBatchProtocol,\n ) -> ImitationTrainingStats:\n return self._imitation_update(batch, self.policy, self.optim)\n\n\nclass OfflineImitationLearning(\n OfflineAlgorithm[ImitationPolicy],\n ImitationLearningAlgorithmMixin,\n):\n \"\"\"Implementation of offline vanilla imitation learning.\"\"\"\n\n def __init__(\n self,\n *,\n policy: ImitationPolicy,\n optim: OptimizerFactory,\n ) -> None:\n \"\"\"\n :param policy: the policy\n :param optim: the optimizer factory\n \"\"\"\n super().__init__(\n policy=policy,\n )\n self.optim = self._create_optimizer(self.policy, optim)\n\n def _update_with_batch(\n self,\n batch: RolloutBatchProtocol,\n ) -> ImitationTrainingStats:\n return self._imitation_update(batch, self.policy, self.optim)\n" }, { "path": "tianshou/algorithm/imitation/td3_bc.py", "content": "import torch\nimport torch.nn.functional as F\n\nfrom tianshou.algorithm import TD3\nfrom tianshou.algorithm.algorithm_base import OfflineAlgorithm\nfrom tianshou.algorithm.modelfree.ddpg import ContinuousDeterministicPolicy\nfrom tianshou.algorithm.modelfree.td3 import TD3TrainingStats\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import to_torch_as\nfrom tianshou.data.types import RolloutBatchProtocol\n\n\n# NOTE: This uses diamond inheritance to convert from off-policy to offline\nclass TD3BC(OfflineAlgorithm[ContinuousDeterministicPolicy], TD3): # type: ignore\n \"\"\"Implementation of TD3+BC. arXiv:2106.06860.\"\"\"\n\n def __init__(\n self,\n *,\n policy: ContinuousDeterministicPolicy,\n policy_optim: OptimizerFactory,\n critic: torch.nn.Module,\n critic_optim: OptimizerFactory,\n critic2: torch.nn.Module | None = None,\n critic2_optim: OptimizerFactory | None = None,\n tau: float = 0.005,\n gamma: float = 0.99,\n policy_noise: float = 0.2,\n update_actor_freq: int = 2,\n noise_clip: float = 0.5,\n alpha: float = 2.5,\n n_step_return_horizon: int = 1,\n ) -> None:\n \"\"\"\n :param policy: the policy\n :param policy_optim: the optimizer factory for the policy's model.\n :param critic: the first critic network. (s, a -> Q(s, a))\n :param critic_optim: the optimizer factory for the first critic network.\n :param critic2: the second critic network. (s, a -> Q(s, a)).\n If None, copy the first critic (via deepcopy).\n :param critic2_optim: the optimizer factory for the second critic network.\n If None, use the first critic's factory.\n :param tau: the soft update coefficient for target networks, controlling the rate at which\n target networks track the learned networks.\n When the parameters of the target network are updated with the current (source) network's\n parameters, a weighted average is used: target = tau * source + (1 - tau) * target.\n Smaller values (closer to 0) create more stable but slower learning as target networks\n change more gradually. Higher values (closer to 1) allow faster learning but may reduce\n stability.\n Typically set to a small value (0.001 to 0.01) for most reinforcement learning tasks.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param exploration_noise: add noise to action for exploration.\n This is useful when solving \"hard exploration\" problems.\n \"default\" is equivalent to GaussianNoise(sigma=0.1).\n :param policy_noise: scaling factor for the Gaussian noise added to target policy actions.\n This parameter implements target policy smoothing, a regularization technique described in the TD3 paper.\n The noise is sampled from a normal distribution and multiplied by this value before being added to actions.\n Higher values increase exploration in the target policy, helping to address function approximation error.\n The added noise is optionally clipped to a range determined by the noise_clip parameter.\n Typically set between 0.1 and 0.5 relative to the action scale of the environment.\n :param update_actor_freq: the frequency of actor network updates relative to critic network updates\n (the actor network is only updated once for every `update_actor_freq` critic updates).\n This implements the \"delayed\" policy updates from the TD3 algorithm, where the actor is\n updated less frequently than the critics.\n Higher values (e.g., 2-5) help stabilize training by allowing the critic to become more\n accurate before updating the policy.\n The default value of 2 follows the original TD3 paper's recommendation of updating the\n policy at half the rate of the Q-functions.\n :param noise_clip: defines the maximum absolute value of the noise added to target policy actions, i.e. noise values\n are clipped to the range [-noise_clip, noise_clip] (after generating and scaling the noise\n via `policy_noise`).\n This parameter implements bounded target policy smoothing as described in the TD3 paper.\n It prevents extreme noise values from causing unrealistic target values during training.\n Setting it 0.0 (or a negative value) disables clipping entirely.\n It is typically set to about twice the `policy_noise` value (e.g. 0.5 when `policy_noise` is 0.2).\n :param alpha: the value of alpha, which controls the weight for TD3 learning\n relative to behavior cloning.\n \"\"\"\n TD3.__init__(\n self,\n policy=policy,\n policy_optim=policy_optim,\n critic=critic,\n critic_optim=critic_optim,\n critic2=critic2,\n critic2_optim=critic2_optim,\n tau=tau,\n gamma=gamma,\n policy_noise=policy_noise,\n noise_clip=noise_clip,\n update_actor_freq=update_actor_freq,\n n_step_return_horizon=n_step_return_horizon,\n )\n self.alpha = alpha\n\n def _update_with_batch(self, batch: RolloutBatchProtocol) -> TD3TrainingStats:\n # critic 1&2\n td1, critic1_loss = self._minimize_critic_squared_loss(\n batch, self.critic, self.critic_optim\n )\n td2, critic2_loss = self._minimize_critic_squared_loss(\n batch, self.critic2, self.critic2_optim\n )\n batch.weight = (td1 + td2) / 2.0 # prio-buffer\n\n # actor\n if self._cnt % self.update_actor_freq == 0:\n act = self.policy(batch, eps=0.0).act\n q_value = self.critic(batch.obs, act)\n lmbda = self.alpha / q_value.abs().mean().detach()\n actor_loss = -lmbda * q_value.mean() + F.mse_loss(act, to_torch_as(batch.act, act))\n self._last = actor_loss.item()\n self.policy_optim.step(actor_loss)\n self._update_lagged_network_weights()\n self._cnt += 1\n\n return TD3TrainingStats(\n actor_loss=self._last,\n critic1_loss=critic1_loss.item(),\n critic2_loss=critic2_loss.item(),\n )\n" }, { "path": "tianshou/algorithm/modelbased/__init__.py", "content": "" }, { "path": "tianshou/algorithm/modelbased/icm.py", "content": "import numpy as np\nimport torch\nimport torch.nn.functional as F\n\nfrom tianshou.algorithm import Algorithm\nfrom tianshou.algorithm.algorithm_base import (\n OffPolicyAlgorithm,\n OffPolicyWrapperAlgorithm,\n OnPolicyAlgorithm,\n OnPolicyWrapperAlgorithm,\n TPolicy,\n TrainingStats,\n TrainingStatsWrapper,\n)\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import Batch, ReplayBuffer, to_numpy, to_torch\nfrom tianshou.data.batch import BatchProtocol\nfrom tianshou.data.types import RolloutBatchProtocol\nfrom tianshou.utils.net.discrete import IntrinsicCuriosityModule\n\n\nclass ICMTrainingStats(TrainingStatsWrapper):\n def __init__(\n self,\n wrapped_stats: TrainingStats,\n *,\n icm_loss: float,\n icm_forward_loss: float,\n icm_inverse_loss: float,\n ) -> None:\n self.icm_loss = icm_loss\n self.icm_forward_loss = icm_forward_loss\n self.icm_inverse_loss = icm_inverse_loss\n super().__init__(wrapped_stats)\n\n\nclass _ICMMixin:\n \"\"\"Implementation of the Intrinsic Curiosity Module (ICM) algorithm. arXiv:1705.05363.\"\"\"\n\n def __init__(\n self,\n *,\n model: IntrinsicCuriosityModule,\n optim: Algorithm.Optimizer,\n lr_scale: float,\n reward_scale: float,\n forward_loss_weight: float,\n ) -> None:\n \"\"\"\n :param model: the ICM model.\n :param optim: the optimizer factory.\n :param lr_scale: a multiplier that effectively scales the learning rate for the ICM model updates.\n Higher values increase the step size during optimization of the intrinsic curiosity module.\n Lower values decrease the step size, leading to more gradual learning of the curiosity mechanism.\n This parameter offers an alternative to directly adjusting the base learning rate in the optimizer.\n :param reward_scale: a multiplier that controls the magnitude of intrinsic rewards (curiosity-driven\n rewards generated by the agent itself) relative to extrinsic rewards (task-specific rewards provided\n by the environment).\n Scales the prediction error (curiosity signal) before adding it to the environment rewards.\n Higher values increase the agent's motivation to explore novel states.\n Lower values decrease the influence of curiosity relative to task-specific rewards.\n Setting to zero effectively disables intrinsic motivation while still learning the ICM model.\n :param forward_loss_weight: relative importance in [0, 1] of the forward model loss in relation to\n the inverse model loss.\n Controls the trade-off between state prediction and action prediction in the ICM algorithm.\n Higher values (> 0.5) prioritize learning to predict next states given current states and actions.\n Lower values (< 0.5) prioritize learning to predict actions given current and next states.\n The total loss combines both components:\n (1-forward_loss_weight)*inverse_loss + forward_loss_weight*forward_loss.\n \"\"\"\n self.model = model\n self.optim = optim\n self.lr_scale = lr_scale\n self.reward_scale = reward_scale\n self.forward_loss_weight = forward_loss_weight\n\n def _icm_preprocess_batch(\n self,\n batch: RolloutBatchProtocol,\n ) -> None:\n mse_loss, act_hat = self.model(batch.obs, batch.act, batch.obs_next)\n batch.policy = Batch(orig_rew=batch.rew, act_hat=act_hat, mse_loss=mse_loss)\n batch.rew += to_numpy(mse_loss * self.reward_scale)\n\n @staticmethod\n def _icm_postprocess_batch(batch: BatchProtocol) -> None:\n # restore original reward\n batch.rew = batch.policy.orig_rew\n\n def _icm_update(\n self,\n batch: RolloutBatchProtocol,\n original_stats: TrainingStats,\n ) -> ICMTrainingStats:\n act_hat = batch.policy.act_hat\n act = to_torch(batch.act, dtype=torch.long, device=act_hat.device)\n inverse_loss = F.cross_entropy(act_hat, act).mean()\n forward_loss = batch.policy.mse_loss.mean()\n loss = (\n (1 - self.forward_loss_weight) * inverse_loss + self.forward_loss_weight * forward_loss\n ) * self.lr_scale\n self.optim.step(loss)\n\n return ICMTrainingStats(\n original_stats,\n icm_loss=loss.item(),\n icm_forward_loss=forward_loss.item(),\n icm_inverse_loss=inverse_loss.item(),\n )\n\n\nclass ICMOffPolicyWrapper(OffPolicyWrapperAlgorithm[TPolicy], _ICMMixin):\n \"\"\"Implementation of the Intrinsic Curiosity Module (ICM) algorithm for off-policy learning. arXiv:1705.05363.\"\"\"\n\n def __init__(\n self,\n *,\n wrapped_algorithm: OffPolicyAlgorithm[TPolicy],\n model: IntrinsicCuriosityModule,\n optim: OptimizerFactory,\n lr_scale: float,\n reward_scale: float,\n forward_loss_weight: float,\n ) -> None:\n \"\"\"\n :param wrapped_algorithm: the base algorithm to which we want to add the ICM.\n :param model: the ICM model.\n :param optim: the optimizer factory for the ICM model.\n :param lr_scale: a multiplier that effectively scales the learning rate for the ICM model updates.\n Higher values increase the step size during optimization of the intrinsic curiosity module.\n Lower values decrease the step size, leading to more gradual learning of the curiosity mechanism.\n This parameter offers an alternative to directly adjusting the base learning rate in the optimizer.\n :param reward_scale: a multiplier that controls the magnitude of intrinsic rewards (curiosity-driven\n rewards generated by the agent itself) relative to extrinsic rewards (task-specific rewards provided\n by the environment).\n Scales the prediction error (curiosity signal) before adding it to the environment rewards.\n Higher values increase the agent's motivation to explore novel states.\n Lower values decrease the influence of curiosity relative to task-specific rewards.\n Setting to zero effectively disables intrinsic motivation while still learning the ICM model.\n :param forward_loss_weight: relative importance in [0, 1] of the forward model loss in relation to\n the inverse model loss.\n Controls the trade-off between state prediction and action prediction in the ICM algorithm.\n Higher values (> 0.5) prioritize learning to predict next states given current states and actions.\n Lower values (< 0.5) prioritize learning to predict actions given current and next states.\n The total loss combines both components:\n (1-forward_loss_weight)*inverse_loss + forward_loss_weight*forward_loss.\n \"\"\"\n OffPolicyWrapperAlgorithm.__init__(\n self,\n wrapped_algorithm=wrapped_algorithm,\n )\n _ICMMixin.__init__(\n self,\n model=model,\n optim=self._create_optimizer(model, optim),\n lr_scale=lr_scale,\n reward_scale=reward_scale,\n forward_loss_weight=forward_loss_weight,\n )\n\n def _preprocess_batch(\n self,\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n ) -> RolloutBatchProtocol:\n self._icm_preprocess_batch(batch)\n return super()._preprocess_batch(batch, buffer, indices)\n\n def _postprocess_batch(\n self,\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n ) -> None:\n super()._postprocess_batch(batch, buffer, indices)\n self._icm_postprocess_batch(batch)\n\n def _wrapper_update_with_batch(\n self,\n batch: RolloutBatchProtocol,\n original_stats: TrainingStats,\n ) -> ICMTrainingStats:\n return self._icm_update(batch, original_stats)\n\n\nclass ICMOnPolicyWrapper(OnPolicyWrapperAlgorithm[TPolicy], _ICMMixin):\n \"\"\"Implementation of the Intrinsic Curiosity Module (ICM) algorithm for on-policy learning. arXiv:1705.05363.\"\"\"\n\n def __init__(\n self,\n *,\n wrapped_algorithm: OnPolicyAlgorithm[TPolicy],\n model: IntrinsicCuriosityModule,\n optim: OptimizerFactory,\n lr_scale: float,\n reward_scale: float,\n forward_loss_weight: float,\n ) -> None:\n \"\"\"\n :param wrapped_algorithm: the base algorithm to which we want to add the ICM.\n :param model: the ICM model.\n :param optim: the optimizer factory for the ICM model.\n :param lr_scale: a multiplier that effectively scales the learning rate for the ICM model updates.\n Higher values increase the step size during optimization of the intrinsic curiosity module.\n Lower values decrease the step size, leading to more gradual learning of the curiosity mechanism.\n This parameter offers an alternative to directly adjusting the base learning rate in the optimizer.\n :param reward_scale: a multiplier that controls the magnitude of intrinsic rewards (curiosity-driven\n rewards generated by the agent itself) relative to extrinsic rewards (task-specific rewards provided\n by the environment).\n Scales the prediction error (curiosity signal) before adding it to the environment rewards.\n Higher values increase the agent's motivation to explore novel states.\n Lower values decrease the influence of curiosity relative to task-specific rewards.\n Setting to zero effectively disables intrinsic motivation while still learning the ICM model.\n :param forward_loss_weight: relative importance in [0, 1] of the forward model loss in relation to\n the inverse model loss.\n Controls the trade-off between state prediction and action prediction in the ICM algorithm.\n Higher values (> 0.5) prioritize learning to predict next states given current states and actions.\n Lower values (< 0.5) prioritize learning to predict actions given current and next states.\n The total loss combines both components:\n (1-forward_loss_weight)*inverse_loss + forward_loss_weight*forward_loss.\n \"\"\"\n OnPolicyWrapperAlgorithm.__init__(\n self,\n wrapped_algorithm=wrapped_algorithm,\n )\n _ICMMixin.__init__(\n self,\n model=model,\n optim=self._create_optimizer(model, optim),\n lr_scale=lr_scale,\n reward_scale=reward_scale,\n forward_loss_weight=forward_loss_weight,\n )\n\n def _preprocess_batch(\n self,\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n ) -> RolloutBatchProtocol:\n self._icm_preprocess_batch(batch)\n return super()._preprocess_batch(batch, buffer, indices)\n\n def _postprocess_batch(\n self,\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n ) -> None:\n super()._postprocess_batch(batch, buffer, indices)\n self._icm_postprocess_batch(batch)\n\n def _wrapper_update_with_batch(\n self,\n batch: RolloutBatchProtocol,\n batch_size: int | None,\n repeat: int,\n original_stats: TrainingStats,\n ) -> ICMTrainingStats:\n return self._icm_update(batch, original_stats)\n" }, { "path": "tianshou/algorithm/modelbased/psrl.py", "content": "from dataclasses import dataclass\nfrom typing import Any, cast\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\n\nfrom tianshou.algorithm.algorithm_base import (\n OnPolicyAlgorithm,\n Policy,\n TrainingStats,\n)\nfrom tianshou.data import Batch\nfrom tianshou.data.batch import BatchProtocol\nfrom tianshou.data.types import ActBatchProtocol, ObsBatchProtocol, RolloutBatchProtocol\n\n\n@dataclass(kw_only=True)\nclass PSRLTrainingStats(TrainingStats):\n psrl_rew_mean: float = 0.0\n psrl_rew_std: float = 0.0\n\n\nclass PSRLModel:\n \"\"\"Implementation of Posterior Sampling Reinforcement Learning Model.\"\"\"\n\n def __init__(\n self,\n trans_count_prior: np.ndarray,\n rew_mean_prior: np.ndarray,\n rew_std_prior: np.ndarray,\n gamma: float,\n epsilon: float,\n ) -> None:\n \"\"\"\n :param trans_count_prior: dirichlet prior (alphas), with shape\n (n_state, n_action, n_state).\n :param rew_mean_prior: means of the normal priors of rewards,\n with shape (n_state, n_action).\n :param rew_std_prior: standard deviations of the normal priors\n of rewards, with shape (n_state, n_action).\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param epsilon: for precision control in value iteration.\n \"\"\"\n self.trans_count = trans_count_prior\n self.n_state, self.n_action = rew_mean_prior.shape\n self.rew_mean = rew_mean_prior\n self.rew_std = rew_std_prior\n self.rew_square_sum = np.zeros_like(rew_mean_prior)\n self.rew_std_prior = rew_std_prior\n self.gamma = gamma\n self.rew_count = np.full(rew_mean_prior.shape, epsilon) # no weight\n self.eps = epsilon\n self.policy: np.ndarray\n self.value = np.zeros(self.n_state)\n self.updated = False\n self.__eps = np.finfo(np.float32).eps.item()\n\n def observe(\n self,\n trans_count: np.ndarray,\n rew_sum: np.ndarray,\n rew_square_sum: np.ndarray,\n rew_count: np.ndarray,\n ) -> None:\n \"\"\"Add data into memory pool.\n\n For rewards, we have a normal prior at first. After we observed a\n reward for a given state-action pair, we use the mean value of our\n observations instead of the prior mean as the posterior mean. The\n standard deviations are in inverse proportion to the number of the\n corresponding observations.\n\n :param trans_count: the number of observations, with shape\n (n_state, n_action, n_state).\n :param rew_sum: total rewards, with shape\n (n_state, n_action).\n :param rew_square_sum: total rewards' squares, with shape\n (n_state, n_action).\n :param rew_count: the number of rewards, with shape\n (n_state, n_action).\n \"\"\"\n self.updated = False\n self.trans_count += trans_count\n sum_count = self.rew_count + rew_count\n self.rew_mean = (self.rew_mean * self.rew_count + rew_sum) / sum_count\n self.rew_square_sum += rew_square_sum\n raw_std2 = self.rew_square_sum / sum_count - self.rew_mean**2\n self.rew_std = np.sqrt(\n 1 / (sum_count / (raw_std2 + self.__eps) + 1 / self.rew_std_prior**2),\n )\n self.rew_count = sum_count\n\n def sample_trans_prob(self) -> np.ndarray:\n return torch.distributions.Dirichlet(torch.from_numpy(self.trans_count)).sample().numpy()\n\n def sample_reward(self) -> np.ndarray:\n return np.random.normal(self.rew_mean, self.rew_std)\n\n def solve_policy(self) -> None:\n self.updated = True\n self.policy, self.value = self.value_iteration(\n self.sample_trans_prob(),\n self.sample_reward(),\n self.gamma,\n self.eps,\n self.value,\n )\n\n @staticmethod\n def value_iteration(\n trans_prob: np.ndarray,\n rew: np.ndarray,\n gamma: float,\n eps: float,\n value: np.ndarray,\n ) -> tuple[np.ndarray, np.ndarray]:\n \"\"\"Value iteration solver for MDPs.\n\n :param trans_prob: transition probabilities, with shape\n (n_state, n_action, n_state).\n :param rew: rewards, with shape (n_state, n_action).\n :param eps: for precision control.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param value: the initialize value of value array, with\n shape (n_state, ).\n\n :return: the optimal policy with shape (n_state, ).\n \"\"\"\n Q = rew + gamma * trans_prob.dot(value)\n new_value = Q.max(axis=1)\n while not np.allclose(new_value, value, eps):\n value = new_value\n Q = rew + gamma * trans_prob.dot(value)\n new_value = Q.max(axis=1)\n # this is to make sure if Q(s, a1) == Q(s, a2) -> choose a1/a2 randomly\n Q += eps * np.random.randn(*Q.shape)\n return Q.argmax(axis=1), new_value\n\n def __call__(\n self,\n obs: np.ndarray,\n state: Any = None,\n info: Any = None,\n ) -> np.ndarray:\n if not self.updated:\n self.solve_policy()\n return self.policy[obs]\n\n\nclass PSRLPolicy(Policy):\n def __init__(\n self,\n *,\n trans_count_prior: np.ndarray,\n rew_mean_prior: np.ndarray,\n rew_std_prior: np.ndarray,\n action_space: gym.spaces.Discrete,\n discount_factor: float = 0.99,\n epsilon: float = 0.01,\n observation_space: gym.Space | None = None,\n ) -> None:\n \"\"\"\n :param trans_count_prior: dirichlet prior (alphas), with shape\n (n_state, n_action, n_state).\n :param rew_mean_prior: means of the normal priors of rewards,\n with shape (n_state, n_action).\n :param rew_std_prior: standard deviations of the normal priors\n of rewards, with shape (n_state, n_action).\n :param action_space: the environment's action_space.\n :param epsilon: for precision control in value iteration.\n :param observation_space: the environment's observation space\n \"\"\"\n super().__init__(\n action_space=action_space,\n observation_space=observation_space,\n action_scaling=False,\n action_bound_method=None,\n )\n self.model = PSRLModel(\n trans_count_prior,\n rew_mean_prior,\n rew_std_prior,\n discount_factor,\n epsilon,\n )\n\n def forward(\n self,\n batch: ObsBatchProtocol,\n state: dict | BatchProtocol | np.ndarray | None = None,\n **kwargs: Any,\n ) -> ActBatchProtocol:\n \"\"\"Compute action over the given batch data with PSRL model.\n\n :return: A :class:`~tianshou.data.Batch` with \"act\" key containing\n the action.\n \"\"\"\n assert isinstance(batch.obs, np.ndarray), \"only support np.ndarray observation\"\n # TODO: shouldn't the model output a state as well if state is passed (i.e. RNNs are involved)?\n act = self.model(batch.obs, state=state, info=batch.info)\n return cast(ActBatchProtocol, Batch(act=act))\n\n\nclass PSRL(OnPolicyAlgorithm[PSRLPolicy]):\n \"\"\"Implementation of Posterior Sampling Reinforcement Learning (PSRL).\n\n Reference: Strens M., A Bayesian Framework for Reinforcement Learning, ICML, 2000.\n \"\"\"\n\n def __init__(\n self,\n *,\n policy: PSRLPolicy,\n add_done_loop: bool = False,\n ) -> None:\n \"\"\"\n :param policy: the policy\n :param add_done_loop: a flag indicating whether to add a self-loop transition for terminal states\n in the MDP.\n If True, whenever an episode terminates, an artificial transition from the terminal state\n back to itself is added to the transition counts for all actions.\n This modification can help stabilize learning for terminal states that have limited samples.\n Setting to True can be beneficial in environments where episodes frequently terminate,\n ensuring that terminal states receive sufficient updates to their value estimates.\n Default is False, which preserves the standard MDP formulation without artificial self-loops.\n \"\"\"\n super().__init__(\n policy=policy,\n )\n self._add_done_loop = add_done_loop\n\n def _update_with_batch(\n self, batch: RolloutBatchProtocol, batch_size: int | None, repeat: int\n ) -> PSRLTrainingStats:\n # NOTE: In contrast to other on-policy algorithms, this algorithm ignores\n # the batch_size and repeat arguments.\n # PSRL, being a Bayesian approach, updates its posterior distribution of\n # the MDP parameters based on the collected transition data as a whole,\n # rather than performing gradient-based updates that benefit from mini-batching.\n n_s, n_a = self.policy.model.n_state, self.policy.model.n_action\n trans_count = np.zeros((n_s, n_a, n_s))\n rew_sum = np.zeros((n_s, n_a))\n rew_square_sum = np.zeros((n_s, n_a))\n rew_count = np.zeros((n_s, n_a))\n for minibatch in batch.split(size=1):\n obs, act, obs_next = minibatch.obs, minibatch.act, minibatch.obs_next\n obs_next = cast(np.ndarray, obs_next)\n assert not isinstance(obs, Batch), \"Observations cannot be Batches here\"\n obs = cast(np.ndarray, obs)\n trans_count[obs, act, obs_next] += 1\n rew_sum[obs, act] += minibatch.rew\n rew_square_sum[obs, act] += minibatch.rew**2\n rew_count[obs, act] += 1\n if self._add_done_loop and minibatch.done:\n # special operation for terminal states: add a self-loop\n trans_count[obs_next, :, obs_next] += 1\n rew_count[obs_next, :] += 1\n self.policy.model.observe(trans_count, rew_sum, rew_square_sum, rew_count)\n\n return PSRLTrainingStats(\n psrl_rew_mean=float(self.policy.model.rew_mean.mean()),\n psrl_rew_std=float(self.policy.model.rew_std.mean()),\n )\n" }, { "path": "tianshou/algorithm/modelfree/__init__.py", "content": "" }, { "path": "tianshou/algorithm/modelfree/a2c.py", "content": "from abc import ABC\nfrom dataclasses import dataclass\nfrom typing import cast\n\nimport numpy as np\nimport torch\nimport torch.nn.functional as F\n\nfrom tianshou.algorithm.algorithm_base import (\n OnPolicyAlgorithm,\n TrainingStats,\n)\nfrom tianshou.algorithm.modelfree.reinforce import ProbabilisticActorPolicy\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import ReplayBuffer, SequenceSummaryStats, to_torch_as\nfrom tianshou.data.types import BatchWithAdvantagesProtocol, RolloutBatchProtocol\nfrom tianshou.utils import RunningMeanStd\nfrom tianshou.utils.net.common import ActorCritic\nfrom tianshou.utils.net.continuous import ContinuousCritic\nfrom tianshou.utils.net.discrete import DiscreteCritic\n\n\n@dataclass(kw_only=True)\nclass A2CTrainingStats(TrainingStats):\n loss: SequenceSummaryStats\n actor_loss: SequenceSummaryStats\n vf_loss: SequenceSummaryStats\n ent_loss: SequenceSummaryStats\n gradient_steps: int\n\n\nclass ActorCriticOnPolicyAlgorithm(OnPolicyAlgorithm[ProbabilisticActorPolicy], ABC):\n \"\"\"Abstract base class for actor-critic algorithms that use generalized advantage estimation (GAE).\"\"\"\n\n def __init__(\n self,\n *,\n policy: ProbabilisticActorPolicy,\n critic: torch.nn.Module | ContinuousCritic | DiscreteCritic,\n optim: OptimizerFactory,\n optim_include_actor: bool,\n max_grad_norm: float | None = None,\n gae_lambda: float = 0.95,\n max_batchsize: int = 256,\n gamma: float = 0.99,\n return_scaling: bool = False,\n ) -> None:\n \"\"\"\n :param critic: the critic network. (s -> V(s))\n :param optim: the optimizer factory.\n :param optim_include_actor: whether the optimizer shall include the actor network's parameters.\n Pass False for algorithms that shall update only the critic via the optimizer.\n :param max_grad_norm: the maximum L2 norm threshold for gradient clipping.\n When not None, gradients will be rescaled using to ensure their L2 norm does not\n exceed this value. This prevents exploding gradients and stabilizes training by\n limiting the magnitude of parameter updates.\n Set to None to disable gradient clipping.\n :param gae_lambda: the lambda parameter in [0, 1] for generalized advantage estimation (GAE).\n Controls the bias-variance tradeoff in advantage estimates, acting as a\n weighting factor for combining different n-step advantage estimators. Higher values\n (closer to 1) reduce bias but increase variance by giving more weight to longer\n trajectories, while lower values (closer to 0) reduce variance but increase bias\n by relying more on the immediate TD error and value function estimates. At λ=0,\n GAE becomes equivalent to the one-step TD error (high bias, low variance); at λ=1,\n it becomes equivalent to Monte Carlo advantage estimation (low bias, high variance).\n Intermediate values create a weighted average of n-step returns, with exponentially\n decaying weights for longer-horizon returns. Typically set between 0.9 and 0.99 for\n most policy gradient methods.\n :param max_batchsize: the maximum number of samples to process at once when computing\n generalized advantage estimation (GAE) and value function predictions.\n Controls memory usage by breaking large batches into smaller chunks processed sequentially.\n Higher values may increase speed but require more GPU/CPU memory; lower values\n reduce memory requirements but may increase computation time. Should be adjusted\n based on available hardware resources and total batch size of your training data.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param return_scaling: flag indicating whether to enable scaling of estimated returns by\n dividing them by their running standard deviation without centering the mean.\n This reduces the magnitude variation of advantages across different episodes while\n preserving their signs and relative ordering.\n The use of running statistics (rather than batch-specific scaling) means that early\n training experiences may be scaled differently than later ones as the statistics evolve.\n When enabled, this improves training stability in environments with highly variable\n reward scales and makes the algorithm less sensitive to learning rate settings.\n However, it may reduce the algorithm's ability to distinguish between episodes with\n different absolute return magnitudes.\n Best used in environments where the relative ordering of actions is more important\n than the absolute scale of returns.\n \"\"\"\n super().__init__(\n policy=policy,\n )\n self.critic = critic\n assert 0.0 <= gae_lambda <= 1.0, f\"GAE lambda should be in [0, 1] but got: {gae_lambda}\"\n self.gae_lambda = gae_lambda\n self.max_batchsize = max_batchsize\n if optim_include_actor:\n self.optim = self._create_optimizer(\n ActorCritic(self.policy.actor, self.critic),\n optim,\n max_grad_norm=max_grad_norm,\n )\n else:\n self.optim = self._create_optimizer(self.critic, optim, max_grad_norm=max_grad_norm)\n self.gamma = gamma\n self.return_scaling = return_scaling\n self.ret_rms = RunningMeanStd()\n self._eps = 1e-8\n\n def _add_returns_and_advantages(\n self,\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n ) -> BatchWithAdvantagesProtocol:\n \"\"\"Adds the returns and advantages to the given batch.\"\"\"\n v_s, v_s_ = [], []\n with torch.no_grad():\n for minibatch in batch.split(self.max_batchsize, shuffle=False, merge_last=True):\n v_s.append(self.critic(minibatch.obs))\n v_s_.append(self.critic(minibatch.obs_next))\n batch.v_s = torch.cat(v_s, dim=0).flatten() # old value\n v_s = batch.v_s.cpu().numpy()\n v_s_ = torch.cat(v_s_, dim=0).flatten().cpu().numpy()\n # when normalizing values, we do not minus self.ret_rms.mean to be numerically\n # consistent with OPENAI baselines' value normalization pipeline. Empirical\n # study also shows that \"minus mean\" will harm performances a tiny little bit\n # due to unknown reasons (on Mujoco envs, not confident, though).\n if self.return_scaling: # unnormalize v_s & v_s_\n v_s = v_s * np.sqrt(self.ret_rms.var + self._eps)\n v_s_ = v_s_ * np.sqrt(self.ret_rms.var + self._eps)\n unnormalized_returns, advantages = self.compute_episodic_return(\n batch,\n buffer,\n indices,\n v_s_,\n v_s,\n gamma=self.gamma,\n gae_lambda=self.gae_lambda,\n )\n if self.return_scaling:\n batch.returns = unnormalized_returns / np.sqrt(self.ret_rms.var + self._eps)\n self.ret_rms.update(unnormalized_returns)\n else:\n batch.returns = unnormalized_returns\n batch.returns = to_torch_as(batch.returns, batch.v_s)\n batch.adv = to_torch_as(advantages, batch.v_s)\n return cast(BatchWithAdvantagesProtocol, batch)\n\n\nclass A2C(ActorCriticOnPolicyAlgorithm):\n \"\"\"Implementation of (synchronous) Advantage Actor-Critic (A2C). arXiv:1602.01783.\"\"\"\n\n def __init__(\n self,\n *,\n policy: ProbabilisticActorPolicy,\n critic: torch.nn.Module | ContinuousCritic | DiscreteCritic,\n optim: OptimizerFactory,\n vf_coef: float = 0.5,\n ent_coef: float = 0.01,\n max_grad_norm: float | None = None,\n gae_lambda: float = 0.95,\n max_batchsize: int = 256,\n gamma: float = 0.99,\n return_scaling: bool = False,\n ) -> None:\n \"\"\"\n :param policy: the policy containing the actor network.\n :param critic: the critic network. (s -> V(s))\n :param optim: the optimizer factory.\n :param vf_coef: coefficient that weights the value loss relative to the actor loss in\n the overall loss function.\n Higher values prioritize accurate value function estimation over policy improvement.\n Controls the trade-off between policy optimization and value function fitting.\n Typically set between 0.5 and 1.0 for most actor-critic implementations.\n :param ent_coef: coefficient that weights the entropy bonus relative to the actor loss.\n Controls the exploration-exploitation trade-off by encouraging policy entropy.\n Higher values promote more exploration by encouraging a more uniform action distribution.\n Lower values focus more on exploitation of the current policy's knowledge.\n Typically set between 0.01 and 0.05 for most actor-critic implementations.\n :param max_grad_norm: the maximum L2 norm threshold for gradient clipping.\n When not None, gradients will be rescaled using to ensure their L2 norm does not\n exceed this value. This prevents exploding gradients and stabilizes training by\n limiting the magnitude of parameter updates.\n Set to None to disable gradient clipping.\n :param gae_lambda: the lambda parameter in [0, 1] for generalized advantage estimation (GAE).\n Controls the bias-variance tradeoff in advantage estimates, acting as a\n weighting factor for combining different n-step advantage estimators. Higher values\n (closer to 1) reduce bias but increase variance by giving more weight to longer\n trajectories, while lower values (closer to 0) reduce variance but increase bias\n by relying more on the immediate TD error and value function estimates. At λ=0,\n GAE becomes equivalent to the one-step TD error (high bias, low variance); at λ=1,\n it becomes equivalent to Monte Carlo advantage estimation (low bias, high variance).\n Intermediate values create a weighted average of n-step returns, with exponentially\n decaying weights for longer-horizon returns. Typically set between 0.9 and 0.99 for\n most policy gradient methods.\n :param max_batchsize: the maximum size of the batch when computing GAE.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param return_scaling: flag indicating whether to enable scaling of estimated returns by\n dividing them by their running standard deviation without centering the mean.\n This reduces the magnitude variation of advantages across different episodes while\n preserving their signs and relative ordering.\n The use of running statistics (rather than batch-specific scaling) means that early\n training experiences may be scaled differently than later ones as the statistics evolve.\n When enabled, this improves training stability in environments with highly variable\n reward scales and makes the algorithm less sensitive to learning rate settings.\n However, it may reduce the algorithm's ability to distinguish between episodes with\n different absolute return magnitudes.\n Best used in environments where the relative ordering of actions is more important\n than the absolute scale of returns.\n \"\"\"\n super().__init__(\n policy=policy,\n critic=critic,\n optim=optim,\n optim_include_actor=True,\n max_grad_norm=max_grad_norm,\n gae_lambda=gae_lambda,\n max_batchsize=max_batchsize,\n gamma=gamma,\n return_scaling=return_scaling,\n )\n self.vf_coef = vf_coef\n self.ent_coef = ent_coef\n self.max_grad_norm = max_grad_norm\n\n def _preprocess_batch(\n self,\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n ) -> BatchWithAdvantagesProtocol:\n batch = self._add_returns_and_advantages(batch, buffer, indices)\n batch.act = to_torch_as(batch.act, batch.v_s)\n return batch\n\n def _update_with_batch( # type: ignore[override]\n self,\n batch: BatchWithAdvantagesProtocol,\n batch_size: int | None,\n repeat: int,\n ) -> A2CTrainingStats:\n losses, actor_losses, vf_losses, ent_losses = [], [], [], []\n split_batch_size = batch_size or -1\n gradient_steps = 0\n for _ in range(repeat):\n for minibatch in batch.split(split_batch_size, merge_last=True):\n gradient_steps += 1\n\n # calculate loss for actor\n dist = self.policy(minibatch).dist\n log_prob = dist.log_prob(minibatch.act)\n log_prob = log_prob.reshape(len(minibatch.adv), -1).transpose(0, 1)\n actor_loss = -(log_prob * minibatch.adv).mean()\n # calculate loss for critic\n value = self.critic(minibatch.obs).flatten()\n vf_loss = F.mse_loss(minibatch.returns, value)\n # calculate regularization and overall loss\n ent_loss = dist.entropy().mean()\n loss = actor_loss + self.vf_coef * vf_loss - self.ent_coef * ent_loss\n self.optim.step(loss)\n actor_losses.append(actor_loss.item())\n vf_losses.append(vf_loss.item())\n ent_losses.append(ent_loss.item())\n losses.append(loss.item())\n\n loss_summary_stat = SequenceSummaryStats.from_sequence(losses)\n actor_loss_summary_stat = SequenceSummaryStats.from_sequence(actor_losses)\n vf_loss_summary_stat = SequenceSummaryStats.from_sequence(vf_losses)\n ent_loss_summary_stat = SequenceSummaryStats.from_sequence(ent_losses)\n\n return A2CTrainingStats(\n loss=loss_summary_stat,\n actor_loss=actor_loss_summary_stat,\n vf_loss=vf_loss_summary_stat,\n ent_loss=ent_loss_summary_stat,\n gradient_steps=gradient_steps,\n )\n" }, { "path": "tianshou/algorithm/modelfree/bdqn.py", "content": "from typing import cast\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom sensai.util.helper import mark_used\n\nfrom tianshou.algorithm.algorithm_base import TArrOrActBatch\nfrom tianshou.algorithm.modelfree.dqn import (\n DiscreteQLearningPolicy,\n QLearningOffPolicyAlgorithm,\n)\nfrom tianshou.algorithm.modelfree.reinforce import SimpleLossTrainingStats\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import Batch, ReplayBuffer, to_numpy, to_torch, to_torch_as\nfrom tianshou.data.batch import BatchProtocol\nfrom tianshou.data.types import (\n ActBatchProtocol,\n BatchWithReturnsProtocol,\n ModelOutputBatchProtocol,\n ObsBatchProtocol,\n RolloutBatchProtocol,\n)\nfrom tianshou.utils.net.common import BranchingNet\n\nmark_used(ActBatchProtocol)\n\n\nclass BDQNPolicy(DiscreteQLearningPolicy[BranchingNet]):\n def __init__(\n self,\n *,\n model: BranchingNet,\n action_space: gym.spaces.Discrete,\n observation_space: gym.Space | None = None,\n eps_training: float = 0.0,\n eps_inference: float = 0.0,\n ):\n \"\"\"\n :param model: BranchingNet mapping (obs, state, info) -> action_values_BA.\n :param action_space: the environment's action space\n :param observation_space: the environment's observation space.\n :param eps_training: the epsilon value for epsilon-greedy exploration during training.\n When collecting data for training, this is the probability of choosing a random action\n instead of the action chosen by the policy.\n A value of 0.0 means no exploration (fully greedy) and a value of 1.0 means full\n exploration (fully random).\n :param eps_inference: the epsilon value for epsilon-greedy exploration during inference,\n i.e. non-training cases (such as evaluation during test steps).\n The epsilon value is the probability of choosing a random action instead of the action\n chosen by the policy.\n A value of 0.0 means no exploration (fully greedy) and a value of 1.0 means full\n exploration (fully random).\n \"\"\"\n super().__init__(\n model=model,\n action_space=action_space,\n observation_space=observation_space,\n eps_training=eps_training,\n eps_inference=eps_inference,\n )\n\n def forward(\n self,\n batch: ObsBatchProtocol,\n state: dict | BatchProtocol | np.ndarray | None = None,\n model: torch.nn.Module | None = None,\n ) -> ModelOutputBatchProtocol:\n if model is None:\n model = self.model\n assert model is not None\n obs = batch.obs\n # TODO: this is very contrived, see also iqn.py\n obs_next_BO = obs.obs if hasattr(obs, \"obs\") else obs\n action_values_BA, hidden_BH = model(obs_next_BO, state=state, info=batch.info)\n act_B = to_numpy(action_values_BA.argmax(dim=-1))\n result = Batch(logits=action_values_BA, act=act_B, state=hidden_BH)\n return cast(ModelOutputBatchProtocol, result)\n\n def add_exploration_noise(\n self,\n act: TArrOrActBatch,\n batch: ObsBatchProtocol,\n ) -> TArrOrActBatch:\n eps = self.eps_training if self.is_within_training_step else self.eps_inference\n if np.isclose(eps, 0.0):\n return act\n if isinstance(act, np.ndarray):\n bsz = len(act)\n rand_mask = np.random.rand(bsz) < eps\n rand_act = np.random.randint(\n low=0,\n high=self.model.action_per_branch,\n size=(bsz, act.shape[-1]),\n )\n if hasattr(batch.obs, \"mask\"):\n rand_act += batch.obs.mask\n act[rand_mask] = rand_act[rand_mask]\n return act # type: ignore[return-value]\n else:\n raise NotImplementedError(\n f\"Currently only numpy arrays are supported, got {type(act)=}.\"\n )\n\n\nclass BDQN(QLearningOffPolicyAlgorithm[BDQNPolicy]):\n \"\"\"Implementation of the Branching Dueling Q-Network (BDQN) algorithm arXiv:1711.08946.\"\"\"\n\n def __init__(\n self,\n *,\n policy: BDQNPolicy,\n optim: OptimizerFactory,\n gamma: float = 0.99,\n target_update_freq: int = 0,\n is_double: bool = True,\n ) -> None:\n \"\"\"\n :param policy: policy\n :param optim: the optimizer factory for the policy's model.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param target_update_freq: the number of training iterations between each complete update of\n the target network.\n Controls how frequently the target Q-network parameters are updated with the current\n Q-network values.\n A value of 0 disables the target network entirely, using only a single network for both\n action selection and bootstrap targets.\n Higher values provide more stable learning targets but slow down the propagation of new\n value estimates. Lower positive values allow faster learning but may lead to instability\n due to rapidly changing targets.\n Typically set between 100-10000 for DQN variants, with exact values depending on environment\n complexity.\n :param is_double: flag indicating whether to use Double Q-learning for target value calculation.\n If True, the algorithm uses the online network to select actions and the target network to evaluate their Q-values.\n This decoupling helps reduce the overestimation bias that standard Q-learning is prone to.\n If False, the algorithm selects actions by directly taking the maximum Q-value from the target network.\n Note: This parameter is most effective when used with a target network (target_update_freq > 0).\n \"\"\"\n super().__init__(\n policy=policy,\n optim=optim,\n gamma=gamma,\n # BDQN implements its own returns computation (below), which supports only 1-step returns\n n_step_return_horizon=1,\n target_update_freq=target_update_freq,\n )\n self.is_double = is_double\n\n def _target_q(self, buffer: ReplayBuffer, indices: np.ndarray) -> torch.Tensor:\n obs_next_batch = Batch(\n obs=buffer[indices].obs_next,\n info=[None] * len(indices),\n ) # obs_next: s_{t+n}\n result = self.policy(obs_next_batch)\n if self.use_target_network:\n # target_Q = Q_old(s_, argmax(Q_new(s_, *)))\n target_q = self.policy(obs_next_batch, model=self.model_old).logits\n else:\n target_q = result.logits\n if self.is_double:\n act = np.expand_dims(self.policy(obs_next_batch).act, -1)\n act = to_torch(act, dtype=torch.long, device=target_q.device)\n else:\n act = target_q.max(-1).indices.unsqueeze(-1)\n return torch.gather(target_q, -1, act).squeeze()\n\n def _compute_return(\n self,\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indice: np.ndarray,\n gamma: float = 0.99,\n ) -> BatchWithReturnsProtocol:\n rew = batch.rew\n with torch.no_grad():\n target_q_torch = self._target_q(buffer, indice) # (bsz, ?)\n target_q = to_numpy(target_q_torch)\n end_flag = buffer.done.copy()\n end_flag[buffer.unfinished_index()] = True\n end_flag = end_flag[indice]\n mean_target_q = np.mean(target_q, -1) if len(target_q.shape) > 1 else target_q\n _target_q = rew + gamma * mean_target_q * (1 - end_flag)\n target_q = np.repeat(_target_q[..., None], self.policy.model.num_branches, axis=-1)\n target_q = np.repeat(target_q[..., None], self.policy.model.action_per_branch, axis=-1)\n\n batch.returns = to_torch_as(target_q, target_q_torch)\n if hasattr(batch, \"weight\"): # prio buffer update\n batch.weight = to_torch_as(batch.weight, target_q_torch)\n return cast(BatchWithReturnsProtocol, batch)\n\n def _preprocess_batch(\n self,\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n ) -> BatchWithReturnsProtocol:\n \"\"\"Compute the 1-step return for BDQ targets.\"\"\"\n return self._compute_return(batch, buffer, indices)\n\n def _update_with_batch( # type: ignore[override]\n self,\n batch: BatchWithReturnsProtocol,\n ) -> SimpleLossTrainingStats:\n self._periodically_update_lagged_network_weights()\n weight = batch.pop(\"weight\", 1.0)\n act = to_torch(batch.act, dtype=torch.long, device=batch.returns.device)\n q = self.policy(batch).logits\n act_mask = torch.zeros_like(q)\n act_mask = act_mask.scatter_(-1, act.unsqueeze(-1), 1)\n act_q = q * act_mask\n returns = batch.returns\n returns = returns * act_mask\n td_error = returns - act_q\n loss = (td_error.pow(2).sum(-1).mean(-1) * weight).mean()\n batch.weight = td_error.sum(-1).sum(-1) # prio-buffer\n self.optim.step(loss)\n\n return SimpleLossTrainingStats(loss=loss.item())\n" }, { "path": "tianshou/algorithm/modelfree/c51.py", "content": "import gymnasium as gym\nimport numpy as np\nimport torch\n\nfrom tianshou.algorithm.modelfree.dqn import (\n DiscreteQLearningPolicy,\n QLearningOffPolicyAlgorithm,\n)\nfrom tianshou.algorithm.modelfree.reinforce import LossSequenceTrainingStats\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import Batch, ReplayBuffer\nfrom tianshou.data.types import RolloutBatchProtocol\nfrom tianshou.utils.net.common import Net\n\n\nclass C51Policy(DiscreteQLearningPolicy):\n def __init__(\n self,\n model: torch.nn.Module | Net,\n action_space: gym.spaces.Space,\n observation_space: gym.Space | None = None,\n num_atoms: int = 51,\n v_min: float = -10.0,\n v_max: float = 10.0,\n eps_training: float = 0.0,\n eps_inference: float = 0.0,\n ):\n \"\"\"\n :param model: a model following the rules (s_B -> action_values_BA)\n :param num_atoms: the number of atoms in the support set of the\n value distribution. Default to 51.\n :param v_min: the value of the smallest atom in the support set.\n Default to -10.0.\n :param v_max: the value of the largest atom in the support set.\n Default to 10.0.\n :param eps_training: the epsilon value for epsilon-greedy exploration during training.\n When collecting data for training, this is the probability of choosing a random action\n instead of the action chosen by the policy.\n A value of 0.0 means no exploration (fully greedy) and a value of 1.0 means full\n exploration (fully random).\n :param eps_inference: the epsilon value for epsilon-greedy exploration during inference,\n i.e. non-training cases (such as evaluation during test steps).\n The epsilon value is the probability of choosing a random action instead of the action\n chosen by the policy.\n A value of 0.0 means no exploration (fully greedy) and a value of 1.0 means full\n exploration (fully random).\n \"\"\"\n assert isinstance(action_space, gym.spaces.Discrete)\n super().__init__(\n model=model,\n action_space=action_space,\n observation_space=observation_space,\n eps_training=eps_training,\n eps_inference=eps_inference,\n )\n assert num_atoms > 1, f\"num_atoms should be greater than 1 but got: {num_atoms}\"\n assert v_min < v_max, f\"v_max should be larger than v_min, but got {v_min=} and {v_max=}\"\n self.num_atoms = num_atoms\n self.v_min = v_min\n self.v_max = v_max\n self.support = torch.nn.Parameter(\n torch.linspace(self.v_min, self.v_max, self.num_atoms),\n requires_grad=False,\n )\n\n def compute_q_value(self, logits: torch.Tensor, mask: np.ndarray | None) -> torch.Tensor:\n return super().compute_q_value((logits * self.support).sum(2), mask)\n\n\nclass C51(QLearningOffPolicyAlgorithm[C51Policy]):\n \"\"\"Implementation of Categorical Deep Q-Network. arXiv:1707.06887.\"\"\"\n\n def __init__(\n self,\n *,\n policy: C51Policy,\n optim: OptimizerFactory,\n gamma: float = 0.99,\n n_step_return_horizon: int = 1,\n target_update_freq: int = 0,\n ) -> None:\n \"\"\"\n :param policy: a policy following the rules (s -> action_values_BA)\n :param optim: the optimizer factory for the policy's model.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param n_step_return_horizon: the number of future steps (> 0) to consider when computing temporal\n difference (TD) targets. Controls the balance between TD learning and Monte Carlo methods:\n higher values reduce bias (by relying less on potentially inaccurate value estimates)\n but increase variance (by incorporating more environmental stochasticity and reducing\n the averaging effect). A value of 1 corresponds to standard TD learning with immediate\n bootstrapping, while very large values approach Monte Carlo-like estimation that uses\n complete episode returns.\n :param target_update_freq: the number of training iterations between each complete update of\n the target network.\n Controls how frequently the target Q-network parameters are updated with the current\n Q-network values.\n A value of 0 disables the target network entirely, using only a single network for both\n action selection and bootstrap targets.\n Higher values provide more stable learning targets but slow down the propagation of new\n value estimates. Lower positive values allow faster learning but may lead to instability\n due to rapidly changing targets.\n Typically set between 100-10000 for DQN variants, with exact values depending on environment\n complexity.\n \"\"\"\n super().__init__(\n policy=policy,\n optim=optim,\n gamma=gamma,\n n_step_return_horizon=n_step_return_horizon,\n target_update_freq=target_update_freq,\n )\n self.delta_z = (policy.v_max - policy.v_min) / (policy.num_atoms - 1)\n\n def _target_q(self, buffer: ReplayBuffer, indices: np.ndarray) -> torch.Tensor:\n return self.policy.support.repeat(len(indices), 1) # shape: [bsz, num_atoms]\n\n def _target_dist(self, batch: RolloutBatchProtocol) -> torch.Tensor:\n obs_next_batch = Batch(obs=batch.obs_next, info=[None] * len(batch))\n if self.use_target_network:\n act = self.policy(obs_next_batch).act\n next_dist = self.policy(obs_next_batch, model=self.model_old).logits\n else:\n next_batch = self.policy(obs_next_batch)\n act = next_batch.act\n next_dist = next_batch.logits\n next_dist = next_dist[np.arange(len(act)), act, :]\n target_support = batch.returns.clamp(self.policy.v_min, self.policy.v_max)\n # An amazing trick for calculating the projection gracefully.\n # ref: https://github.com/ShangtongZhang/DeepRL\n target_dist = (\n 1\n - (target_support.unsqueeze(1) - self.policy.support.view(1, -1, 1)).abs()\n / self.delta_z\n ).clamp(0, 1) * next_dist.unsqueeze(1)\n return target_dist.sum(-1)\n\n def _update_with_batch(\n self,\n batch: RolloutBatchProtocol,\n ) -> LossSequenceTrainingStats:\n self._periodically_update_lagged_network_weights()\n with torch.no_grad():\n target_dist = self._target_dist(batch)\n weight = batch.pop(\"weight\", 1.0)\n curr_dist = self.policy(batch).logits\n act = batch.act\n curr_dist = curr_dist[np.arange(len(act)), act, :]\n cross_entropy = -(target_dist * torch.log(curr_dist + 1e-8)).sum(1)\n loss = (cross_entropy * weight).mean()\n # ref: https://github.com/Kaixhin/Rainbow/blob/master/agent.py L94-100\n batch.weight = cross_entropy.detach() # prio-buffer\n self.optim.step(loss)\n\n return LossSequenceTrainingStats(loss=loss.item())\n" }, { "path": "tianshou/algorithm/modelfree/ddpg.py", "content": "import warnings\nfrom abc import ABC\nfrom dataclasses import dataclass\nfrom typing import Any, Generic, Literal, TypeVar, cast\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom sensai.util.helper import mark_used\n\nfrom tianshou.algorithm import Algorithm\nfrom tianshou.algorithm.algorithm_base import (\n LaggedNetworkPolyakUpdateAlgorithmMixin,\n OffPolicyAlgorithm,\n Policy,\n TArrOrActBatch,\n TPolicy,\n TrainingStats,\n)\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import Batch, ReplayBuffer\nfrom tianshou.data.batch import BatchProtocol\nfrom tianshou.data.types import (\n ActBatchProtocol,\n ActStateBatchProtocol,\n BatchWithReturnsProtocol,\n ObsBatchProtocol,\n RolloutBatchProtocol,\n)\nfrom tianshou.exploration import BaseNoise, GaussianNoise\nfrom tianshou.utils.net.continuous import (\n AbstractContinuousActorDeterministic,\n ContinuousCritic,\n)\n\nmark_used(ActBatchProtocol)\n\n\n@dataclass(kw_only=True)\nclass DDPGTrainingStats(TrainingStats):\n actor_loss: float\n critic_loss: float\n\n\nclass ContinuousPolicyWithExplorationNoise(Policy, ABC):\n def __init__(\n self,\n *,\n exploration_noise: BaseNoise | Literal[\"default\"] | None = None,\n action_space: gym.Space,\n observation_space: gym.Space | None = None,\n action_scaling: bool = True,\n action_bound_method: Literal[\"clip\"] | None = \"clip\",\n ):\n \"\"\"\n :param exploration_noise: noise model for adding noise to continuous actions\n for exploration. This is useful when solving \"hard exploration\" problems.\n \"default\" is equivalent to GaussianNoise(sigma=0.1).\n :param action_space: the environment's action_space.\n :param observation_space: the environment's observation space\n :param action_scaling: flag indicating whether, for continuous action spaces, actions\n should be scaled from the standard neural network output range [-1, 1] to the\n environment's action space range [action_space.low, action_space.high].\n This applies to continuous action spaces only (gym.spaces.Box) and has no effect\n for discrete spaces.\n When enabled, policy outputs are expected to be in the normalized range [-1, 1]\n (after bounding), and are then linearly transformed to the actual required range.\n This improves neural network training stability, allows the same algorithm to work\n across environments with different action ranges, and standardizes exploration\n strategies.\n Should be disabled if the actor model already produces outputs in the correct range.\n :param action_bound_method: the method used for bounding actions in continuous action spaces\n to the range [-1, 1] before scaling them to the environment's action space (provided\n that `action_scaling` is enabled).\n This applies to continuous action spaces only (`gym.spaces.Box`) and should be set to None\n for discrete spaces.\n When set to \"clip\", actions exceeding the [-1, 1] range are simply clipped to this\n range. When set to \"tanh\", a hyperbolic tangent function is applied, which smoothly\n constrains outputs to [-1, 1] while preserving gradients.\n The choice of bounding method affects both training dynamics and exploration behavior.\n Clipping provides hard boundaries but may create plateau regions in the gradient\n landscape, while tanh provides smoother transitions but can compress sensitivity\n near the boundaries.\n Should be set to None if the actor model inherently produces bounded outputs.\n Typically used together with `action_scaling=True`.\n \"\"\"\n super().__init__(\n action_space=action_space,\n observation_space=observation_space,\n action_scaling=action_scaling,\n action_bound_method=action_bound_method,\n )\n if exploration_noise == \"default\":\n exploration_noise = GaussianNoise(sigma=0.1)\n self.exploration_noise = exploration_noise\n\n def set_exploration_noise(self, noise: BaseNoise | None) -> None:\n \"\"\"Set the exploration noise.\"\"\"\n self.exploration_noise = noise\n\n def add_exploration_noise(\n self,\n act: TArrOrActBatch,\n batch: ObsBatchProtocol,\n ) -> TArrOrActBatch:\n if self.exploration_noise is None:\n return act\n if isinstance(act, np.ndarray):\n return act + self.exploration_noise(act.shape)\n warnings.warn(\"Cannot add exploration noise to non-numpy_array action.\")\n return act\n\n\nclass ContinuousDeterministicPolicy(ContinuousPolicyWithExplorationNoise):\n \"\"\"A policy for continuous action spaces that uses an actor which directly maps states to actions.\"\"\"\n\n def __init__(\n self,\n *,\n actor: AbstractContinuousActorDeterministic,\n exploration_noise: BaseNoise | Literal[\"default\"] | None = None,\n action_space: gym.Space,\n observation_space: gym.Space | None = None,\n action_scaling: bool = True,\n action_bound_method: Literal[\"clip\"] | None = \"clip\",\n ):\n \"\"\"\n :param actor: The actor network following the rules (s -> actions)\n :param exploration_noise: add noise to continuous actions for exploration;\n set to None for discrete action spaces.\n This is useful when solving \"hard exploration\" problems.\n \"default\" is equivalent to GaussianNoise(sigma=0.1).\n :param action_space: the environment's action space.\n :param tau: the soft update coefficient for target networks, controlling the rate at which\n target networks track the learned networks.\n When the parameters of the target network are updated with the current (source) network's\n parameters, a weighted average is used: target = tau * source + (1 - tau) * target.\n Smaller values (closer to 0) create more stable but slower learning as target networks\n change more gradually. Higher values (closer to 1) allow faster learning but may reduce\n stability.\n Typically set to a small value (0.001 to 0.01) for most reinforcement learning tasks.\n :param observation_space: the environment's observation space.\n :param action_scaling: flag indicating whether, for continuous action spaces, actions\n should be scaled from the standard neural network output range [-1, 1] to the\n environment's action space range [action_space.low, action_space.high].\n This applies to continuous action spaces only (gym.spaces.Box) and has no effect\n for discrete spaces.\n When enabled, policy outputs are expected to be in the normalized range [-1, 1]\n (after bounding), and are then linearly transformed to the actual required range.\n This improves neural network training stability, allows the same algorithm to work\n across environments with different action ranges, and standardizes exploration\n strategies.\n Should be disabled if the actor model already produces outputs in the correct range.\n :param action_bound_method: method to bound action to range [-1, 1].\n \"\"\"\n if action_scaling and not np.isclose(actor.max_action, 1.0):\n warnings.warn(\n \"action_scaling and action_bound_method are only intended to deal\"\n \"with unbounded model action space, but find actor model bound\"\n f\"action space with max_action={actor.max_action}.\"\n \"Consider using unbounded=True option of the actor model,\"\n \"or set action_scaling to False and action_bound_method to None.\",\n )\n super().__init__(\n exploration_noise=exploration_noise,\n action_space=action_space,\n observation_space=observation_space,\n action_scaling=action_scaling,\n action_bound_method=action_bound_method,\n )\n self.actor = actor\n\n def forward(\n self,\n batch: ObsBatchProtocol,\n state: dict | BatchProtocol | np.ndarray | None = None,\n model: torch.nn.Module | None = None,\n **kwargs: Any,\n ) -> ActStateBatchProtocol:\n \"\"\"Compute action over the given batch data.\n\n :return: A :class:`~tianshou.data.Batch` which has 2 keys:\n\n * ``act`` the action.\n * ``state`` the hidden state.\n \"\"\"\n if model is None:\n model = self.actor\n actions, hidden = model(batch.obs, state=state, info=batch.info)\n return cast(ActStateBatchProtocol, Batch(act=actions, state=hidden))\n\n\nTActBatchProtocol = TypeVar(\"TActBatchProtocol\", bound=ActBatchProtocol)\n\n\nclass ActorCriticOffPolicyAlgorithm(\n OffPolicyAlgorithm[TPolicy],\n LaggedNetworkPolyakUpdateAlgorithmMixin,\n Generic[TPolicy, TActBatchProtocol],\n ABC,\n):\n \"\"\"Base class for actor-critic off-policy algorithms that use a lagged critic\n as a target network.\n\n Its implementation of `process_fn` adds the n-step return to the batch, using the\n Q-values computed by the target network (lagged critic, `critic_old`) in order to compute the\n reward-to-go.\n\n Specializations can override the action computation (`_target_q_compute_action`) or the\n Q-value computation based on these actions (`_target_q_compute_value`) to customize the\n target Q-value computation.\n The default implementation assumes a continuous action space where a critic receives a\n state/observation and an action; for discrete action spaces, where the critic receives only\n a state/observation, the method `_target_q_compute_value` must be overridden.\n \"\"\"\n\n def __init__(\n self,\n *,\n policy: TPolicy,\n policy_optim: OptimizerFactory,\n critic: torch.nn.Module,\n critic_optim: OptimizerFactory,\n tau: float = 0.005,\n gamma: float = 0.99,\n n_step_return_horizon: int = 1,\n ) -> None:\n \"\"\"\n :param policy: the policy\n :param policy_optim: the optimizer factory for the policy's model.\n :param critic: the critic network.\n For continuous action spaces: (s, a -> Q(s, a)).\n For discrete action spaces: (s -> ).\n **NOTE**: The default implementation of `_target_q_compute_value` assumes\n a continuous action space; override this method if using discrete actions.\n :param critic_optim: the optimizer factory for the critic network.\n :param tau: the soft update coefficient for target networks, controlling the rate at which\n target networks track the learned networks.\n When the parameters of the target network are updated with the current (source) network's\n parameters, a weighted average is used: target = tau * source + (1 - tau) * target.\n Smaller values (closer to 0) create more stable but slower learning as target networks\n change more gradually. Higher values (closer to 1) allow faster learning but may reduce\n stability.\n Typically set to a small value (0.001 to 0.01) for most reinforcement learning tasks.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n \"\"\"\n assert 0.0 <= tau <= 1.0, f\"tau should be in [0, 1] but got: {tau}\"\n assert 0.0 <= gamma <= 1.0, f\"gamma should be in [0, 1] but got: {gamma}\"\n super().__init__(\n policy=policy,\n )\n LaggedNetworkPolyakUpdateAlgorithmMixin.__init__(self, tau=tau)\n self.policy_optim = self._create_optimizer(policy, policy_optim)\n self.critic = critic\n self.critic_old = self._add_lagged_network(self.critic)\n self.critic_optim = self._create_optimizer(self.critic, critic_optim)\n self.gamma = gamma\n self.n_step_return_horizon = n_step_return_horizon\n\n @staticmethod\n def _minimize_critic_squared_loss(\n batch: RolloutBatchProtocol,\n critic: torch.nn.Module,\n optimizer: Algorithm.Optimizer,\n ) -> tuple[torch.Tensor, torch.Tensor]:\n \"\"\"Takes an optimizer step to minimize the squared loss of the critic given a batch of data.\n\n :param batch: the batch containing the observations, actions, returns, and (optionally) weights.\n :param critic: the critic network to minimize the loss for.\n :param optimizer: the optimizer for the critic's parameters.\n :return: a pair (`td`, `loss`), where `td` is the tensor of errors (current - target) and `loss` is the MSE loss.\n \"\"\"\n weight = getattr(batch, \"weight\", 1.0)\n current_q = critic(batch.obs, batch.act).flatten()\n target_q = batch.returns.flatten()\n td = current_q - target_q\n critic_loss = (td.pow(2) * weight).mean()\n optimizer.step(critic_loss)\n return td, critic_loss\n\n def _preprocess_batch(\n self,\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n ) -> RolloutBatchProtocol | BatchWithReturnsProtocol:\n # add the n-step return to the batch, which the critic (Q-functions) seeks to match,\n # based the Q-values computed by the target network (lagged critic)\n return self.compute_nstep_return(\n batch=batch,\n buffer=buffer,\n indices=indices,\n target_q_fn=self._target_q,\n gamma=self.gamma,\n n_step=self.n_step_return_horizon,\n )\n\n def _target_q_compute_action(self, obs_batch: Batch) -> TActBatchProtocol:\n \"\"\"\n Computes the action to be taken for the given batch (containing the observations)\n within the context of Q-value target computation.\n\n :param obs_batch: the batch containing the observations.\n :return: batch containing the actions to be taken.\n \"\"\"\n return self.policy(obs_batch)\n\n def _target_q_compute_value(\n self, obs_batch: Batch, act_batch: TActBatchProtocol\n ) -> torch.Tensor:\n \"\"\"\n Computes the target Q-value given a batch with observations and actions taken.\n\n :param obs_batch: the batch containing the observations.\n :param act_batch: the batch containing the actions taken.\n :return: a tensor containing the target Q-values.\n \"\"\"\n # compute the target Q-value using the lagged critic network (target network)\n return self.critic_old(obs_batch.obs, act_batch.act)\n\n def _target_q(self, buffer: ReplayBuffer, indices: np.ndarray) -> torch.Tensor:\n \"\"\"\n Computes the target Q-value for the given buffer and indices.\n\n :param buffer: the replay buffer\n :param indices: the indices within the buffer to compute the target Q-value for\n \"\"\"\n obs_next_batch = Batch(\n obs=buffer[indices].obs_next,\n info=[None] * len(indices),\n ) # obs_next: s_{t+n}\n act_batch = self._target_q_compute_action(obs_next_batch)\n return self._target_q_compute_value(obs_next_batch, act_batch)\n\n\nclass DDPG(\n ActorCriticOffPolicyAlgorithm[ContinuousDeterministicPolicy, ActBatchProtocol],\n):\n \"\"\"Implementation of Deep Deterministic Policy Gradient. arXiv:1509.02971.\"\"\"\n\n def __init__(\n self,\n *,\n policy: ContinuousDeterministicPolicy,\n policy_optim: OptimizerFactory,\n critic: torch.nn.Module | ContinuousCritic,\n critic_optim: OptimizerFactory,\n tau: float = 0.005,\n gamma: float = 0.99,\n n_step_return_horizon: int = 1,\n ) -> None:\n \"\"\"\n :param policy: the policy\n :param policy_optim: the optimizer factory for the policy's model.\n :param critic: the critic network. (s, a -> Q(s, a))\n :param critic_optim: the optimizer factory for the critic network.\n :param tau: the soft update coefficient for target networks, controlling the rate at which\n target networks track the learned networks.\n When the parameters of the target network are updated with the current (source) network's\n parameters, a weighted average is used: target = tau * source + (1 - tau) * target.\n Smaller values (closer to 0) create more stable but slower learning as target networks\n change more gradually. Higher values (closer to 1) allow faster learning but may reduce\n stability.\n Typically set to a small value (0.001 to 0.01) for most reinforcement learning tasks.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param n_step_return_horizon: the number of future steps (> 0) to consider when computing temporal\n difference (TD) targets. Controls the balance between TD learning and Monte Carlo methods:\n higher values reduce bias (by relying less on potentially inaccurate value estimates)\n but increase variance (by incorporating more environmental stochasticity and reducing\n the averaging effect). A value of 1 corresponds to standard TD learning with immediate\n bootstrapping, while very large values approach Monte Carlo-like estimation that uses\n complete episode returns.\n \"\"\"\n super().__init__(\n policy=policy,\n policy_optim=policy_optim,\n critic=critic,\n critic_optim=critic_optim,\n tau=tau,\n gamma=gamma,\n n_step_return_horizon=n_step_return_horizon,\n )\n self.actor_old = self._add_lagged_network(self.policy.actor)\n\n def _target_q_compute_action(self, obs_batch: Batch) -> ActBatchProtocol:\n # compute the action using the lagged actor network\n return self.policy(obs_batch, model=self.actor_old)\n\n def _update_with_batch(self, batch: RolloutBatchProtocol) -> DDPGTrainingStats:\n # critic\n td, critic_loss = self._minimize_critic_squared_loss(batch, self.critic, self.critic_optim)\n batch.weight = td # prio-buffer\n # actor\n actor_loss = -self.critic(batch.obs, self.policy(batch).act).mean()\n self.policy_optim.step(actor_loss)\n self._update_lagged_network_weights()\n\n return DDPGTrainingStats(actor_loss=actor_loss.item(), critic_loss=critic_loss.item())\n" }, { "path": "tianshou/algorithm/modelfree/discrete_sac.py", "content": "from dataclasses import dataclass\nfrom typing import Any, TypeVar, cast\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.distributions import Categorical\n\nfrom tianshou.algorithm.algorithm_base import Policy\nfrom tianshou.algorithm.modelfree.sac import Alpha, SACTrainingStats\nfrom tianshou.algorithm.modelfree.td3 import ActorDualCriticsOffPolicyAlgorithm\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import Batch, to_torch\nfrom tianshou.data.batch import BatchProtocol\nfrom tianshou.data.types import (\n DistBatchProtocol,\n ObsBatchProtocol,\n RolloutBatchProtocol,\n)\nfrom tianshou.utils.net.discrete import DiscreteCritic\n\n\n@dataclass\nclass DiscreteSACTrainingStats(SACTrainingStats):\n pass\n\n\nTDiscreteSACTrainingStats = TypeVar(\"TDiscreteSACTrainingStats\", bound=DiscreteSACTrainingStats)\n\n\nclass DiscreteSACPolicy(Policy):\n def __init__(\n self,\n *,\n actor: torch.nn.Module,\n deterministic_eval: bool = True,\n action_space: gym.Space,\n observation_space: gym.Space | None = None,\n ):\n \"\"\"\n :param actor: the actor network following the rules (s -> dist_input_BD),\n where the distribution input is for a `Categorical` distribution.\n :param deterministic_eval: flag indicating whether the policy should use deterministic\n actions (using the mode of the action distribution) instead of stochastic ones\n (using random sampling) during evaluation.\n When enabled, the policy will always select the most probable action according to\n the learned distribution during evaluation phases, while still using stochastic\n sampling during training. This creates a clear distinction between exploration\n (training) and exploitation (evaluation) behaviors.\n Deterministic actions are generally preferred for final deployment and reproducible\n evaluation as they provide consistent behavior, reduce variance in performance\n metrics, and are more interpretable for human observers.\n Note that this parameter only affects behavior when the policy is not within a\n training step. When collecting rollouts for training, actions remain stochastic\n regardless of this setting to maintain proper exploration behaviour.\n :param action_space: the environment's action_space.\n :param observation_space: the environment's observation space\n \"\"\"\n assert isinstance(action_space, gym.spaces.Discrete)\n super().__init__(\n action_space=action_space,\n observation_space=observation_space,\n )\n self.actor = actor\n self.deterministic_eval = deterministic_eval\n\n def forward(\n self,\n batch: ObsBatchProtocol,\n state: dict | BatchProtocol | np.ndarray | None = None,\n **kwargs: Any,\n ) -> Batch:\n logits_BA, hidden_BH = self.actor(batch.obs, state=state, info=batch.info)\n dist = Categorical(logits=logits_BA)\n act_B = (\n dist.mode\n if self.deterministic_eval and not self.is_within_training_step\n else dist.sample()\n )\n return Batch(logits=logits_BA, act=act_B, state=hidden_BH, dist=dist)\n\n\nclass DiscreteSAC(ActorDualCriticsOffPolicyAlgorithm[DiscreteSACPolicy, DistBatchProtocol]):\n \"\"\"Implementation of SAC for Discrete Action Settings. arXiv:1910.07207.\"\"\"\n\n def __init__(\n self,\n *,\n policy: DiscreteSACPolicy,\n policy_optim: OptimizerFactory,\n critic: torch.nn.Module | DiscreteCritic,\n critic_optim: OptimizerFactory,\n critic2: torch.nn.Module | DiscreteCritic | None = None,\n critic2_optim: OptimizerFactory | None = None,\n tau: float = 0.005,\n gamma: float = 0.99,\n alpha: float | Alpha = 0.2,\n n_step_return_horizon: int = 1,\n ) -> None:\n \"\"\"\n :param policy: the policy\n :param policy_optim: the optimizer factory for the policy's model.\n :param critic: the first critic network. (s -> ).\n :param critic_optim: the optimizer factory for the first critic network.\n :param critic2: the second critic network. (s -> ).\n If None, copy the first critic (via deepcopy).\n :param critic2_optim: the optimizer factory for the second critic network.\n If None, use the first critic's factory.\n :param tau: the soft update coefficient for target networks, controlling the rate at which\n target networks track the learned networks.\n When the parameters of the target network are updated with the current (source) network's\n parameters, a weighted average is used: target = tau * source + (1 - tau) * target.\n Smaller values (closer to 0) create more stable but slower learning as target networks\n change more gradually. Higher values (closer to 1) allow faster learning but may reduce\n stability.\n Typically set to a small value (0.001 to 0.01) for most reinforcement learning tasks.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param alpha: the entropy regularization coefficient alpha or an object\n which can be used to automatically tune it (e.g. an instance of `AutoAlpha`).\n :param n_step_return_horizon: the number of future steps (> 0) to consider when computing temporal\n difference (TD) targets. Controls the balance between TD learning and Monte Carlo methods:\n higher values reduce bias (by relying less on potentially inaccurate value estimates)\n but increase variance (by incorporating more environmental stochasticity and reducing\n the averaging effect). A value of 1 corresponds to standard TD learning with immediate\n bootstrapping, while very large values approach Monte Carlo-like estimation that uses\n complete episode returns.\n \"\"\"\n super().__init__(\n policy=policy,\n policy_optim=policy_optim,\n critic=critic,\n critic_optim=critic_optim,\n critic2=critic2,\n critic2_optim=critic2_optim,\n tau=tau,\n gamma=gamma,\n n_step_return_horizon=n_step_return_horizon,\n )\n self.alpha = Alpha.from_float_or_instance(alpha)\n\n def _target_q_compute_value(\n self, obs_batch: Batch, act_batch: DistBatchProtocol\n ) -> torch.Tensor:\n dist = cast(Categorical, act_batch.dist)\n target_q = dist.probs * torch.min(\n self.critic_old(obs_batch.obs),\n self.critic2_old(obs_batch.obs),\n )\n return target_q.sum(dim=-1) + self.alpha.value * dist.entropy()\n\n def _update_with_batch(self, batch: RolloutBatchProtocol) -> TDiscreteSACTrainingStats: # type: ignore\n weight = batch.pop(\"weight\", 1.0)\n target_q = batch.returns.flatten()\n act = to_torch(batch.act[:, np.newaxis], device=target_q.device, dtype=torch.long)\n\n # critic 1\n current_q1 = self.critic(batch.obs).gather(1, act).flatten()\n td1 = current_q1 - target_q\n critic1_loss = (td1.pow(2) * weight).mean()\n self.critic_optim.step(critic1_loss)\n\n # critic 2\n current_q2 = self.critic2(batch.obs).gather(1, act).flatten()\n td2 = current_q2 - target_q\n critic2_loss = (td2.pow(2) * weight).mean()\n self.critic2_optim.step(critic2_loss)\n\n batch.weight = (td1 + td2) / 2.0 # prio-buffer\n\n # actor\n dist = self.policy(batch).dist\n entropy = dist.entropy()\n with torch.no_grad():\n current_q1a = self.critic(batch.obs)\n current_q2a = self.critic2(batch.obs)\n q = torch.min(current_q1a, current_q2a)\n actor_loss = -(self.alpha.value * entropy + (dist.probs * q).sum(dim=-1)).mean()\n self.policy_optim.step(actor_loss)\n\n alpha_loss = self.alpha.update(entropy.detach())\n\n self._update_lagged_network_weights()\n\n return DiscreteSACTrainingStats( # type: ignore[return-value]\n actor_loss=actor_loss.item(),\n critic1_loss=critic1_loss.item(),\n critic2_loss=critic2_loss.item(),\n alpha=self.alpha.value,\n alpha_loss=alpha_loss,\n )\n" }, { "path": "tianshou/algorithm/modelfree/dqn.py", "content": "import logging\nfrom abc import ABC, abstractmethod\nfrom typing import Any, Generic, TypeVar, cast\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom gymnasium.spaces.discrete import Discrete\nfrom sensai.util.helper import mark_used\n\nfrom tianshou.algorithm import Algorithm\nfrom tianshou.algorithm.algorithm_base import (\n LaggedNetworkFullUpdateAlgorithmMixin,\n OffPolicyAlgorithm,\n Policy,\n TArrOrActBatch,\n)\nfrom tianshou.algorithm.modelfree.reinforce import (\n SimpleLossTrainingStats,\n)\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import Batch, ReplayBuffer, to_numpy, to_torch_as\nfrom tianshou.data.types import (\n ActBatchProtocol,\n BatchWithReturnsProtocol,\n ModelOutputBatchProtocol,\n ObsBatchProtocol,\n RolloutBatchProtocol,\n)\nfrom tianshou.utils.lagged_network import EvalModeModuleWrapper\nfrom tianshou.utils.net.common import Net\n\nmark_used(ActBatchProtocol)\n\nTModel = TypeVar(\"TModel\", bound=torch.nn.Module | Net)\nlog = logging.getLogger(__name__)\n\n\nclass DiscreteQLearningPolicy(Policy, Generic[TModel]):\n def __init__(\n self,\n *,\n model: TModel,\n action_space: gym.spaces.Space,\n observation_space: gym.Space | None = None,\n eps_training: float = 0.0,\n eps_inference: float = 0.0,\n ) -> None:\n \"\"\"\n :param model: a model mapping (obs, state, info) to action_values_BA.\n :param action_space: the environment's action space\n :param observation_space: the environment's observation space.\n :param eps_training: the epsilon value for epsilon-greedy exploration during training.\n When collecting data for training, this is the probability of choosing a random action\n instead of the action chosen by the policy.\n A value of 0.0 means no exploration (fully greedy) and a value of 1.0 means full\n exploration (fully random).\n :param eps_inference: the epsilon value for epsilon-greedy exploration during inference,\n i.e. non-training cases (such as evaluation during test steps).\n The epsilon value is the probability of choosing a random action instead of the action\n chosen by the policy.\n A value of 0.0 means no exploration (fully greedy) and a value of 1.0 means full\n exploration (fully random).\n \"\"\"\n super().__init__(\n action_space=action_space,\n observation_space=observation_space,\n action_scaling=False,\n action_bound_method=None,\n )\n self.action_space = cast(Discrete, self.action_space)\n self.model = model\n self.eps_training = eps_training\n self.eps_inference = eps_inference\n\n def set_eps_training(self, eps: float) -> None:\n \"\"\"\n Sets the epsilon value for epsilon-greedy exploration during training.\n\n :param eps: the epsilon value for epsilon-greedy exploration during training.\n When collecting data for training, this is the probability of choosing a random action\n instead of the action chosen by the policy.\n A value of 0.0 means no exploration (fully greedy) and a value of 1.0 means full\n exploration (fully random).\n \"\"\"\n self.eps_training = eps\n\n def set_eps_inference(self, eps: float) -> None:\n \"\"\"\n Sets the epsilon value for epsilon-greedy exploration during inference.\n\n :param eps: the epsilon value for epsilon-greedy exploration during inference,\n i.e. non-training cases (such as evaluation during test steps).\n The epsilon value is the probability of choosing a random action instead of the action\n chosen by the policy.\n A value of 0.0 means no exploration (fully greedy) and a value of 1.0 means full\n exploration (fully random).\n \"\"\"\n self.eps_inference = eps\n\n def forward(\n self,\n batch: ObsBatchProtocol,\n state: Any | None = None,\n model: torch.nn.Module | None = None,\n ) -> ModelOutputBatchProtocol:\n \"\"\"Compute action over the given batch data.\n\n If you need to mask the action, please add a \"mask\" into batch.obs, for\n example, if we have an environment that has \"0/1/2\" three actions:\n ::\n\n batch == Batch(\n obs=Batch(\n obs=\"original obs, with batch_size=1 for demonstration\",\n mask=np.array([[False, True, False]]),\n # action 1 is available\n # action 0 and 2 are unavailable\n ),\n ...\n )\n\n :param batch:\n :param state: optional hidden state (for RNNs)\n :param model: if not passed will use `self.model`. Typically used to pass\n the lagged target network instead of using the current model.\n :return: A :class:`~tianshou.data.Batch` which has 3 keys:\n\n * ``act`` the action.\n * ``logits`` the network's raw output.\n * ``state`` the hidden state.\n \"\"\"\n if model is None:\n model = self.model\n obs = batch.obs\n mask = getattr(obs, \"mask\", None)\n # TODO: this is convoluted! See also other places where this is done.\n obs_arr = obs.obs if hasattr(obs, \"obs\") else obs\n action_values_BA, hidden_BH = model(obs_arr, state=state, info=batch.info)\n q = self.compute_q_value(action_values_BA, mask)\n act_B = to_numpy(q.argmax(dim=1))\n result = Batch(logits=action_values_BA, act=act_B, state=hidden_BH)\n return cast(ModelOutputBatchProtocol, result)\n\n def compute_q_value(self, logits: torch.Tensor, mask: np.ndarray | None) -> torch.Tensor:\n \"\"\"Compute the q value based on the network's raw output and action mask.\"\"\"\n if mask is not None:\n # the masked q value should be smaller than logits.min()\n min_value = logits.min() - logits.max() - 1.0\n logits = logits + to_torch_as(1 - mask, logits) * min_value\n return logits\n\n def add_exploration_noise(\n self,\n act: TArrOrActBatch,\n batch: ObsBatchProtocol,\n ) -> TArrOrActBatch:\n eps = self.eps_training if self.is_within_training_step else self.eps_inference\n if np.isclose(eps, 0.0):\n return act\n if isinstance(act, np.ndarray):\n batch_size = len(act)\n rand_mask = np.random.rand(batch_size) < eps\n self.action_space = cast(Discrete, self.action_space) # for mypy\n action_num = int(self.action_space.n)\n q = np.random.rand(batch_size, action_num) # [0, 1]\n if hasattr(batch.obs, \"mask\"):\n q += batch.obs.mask\n rand_act = q.argmax(axis=1)\n act[rand_mask] = rand_act[rand_mask]\n return act # type: ignore[return-value]\n raise NotImplementedError(\n f\"Currently only numpy array is supported for action, but got {type(act)}\"\n )\n\n\nTDQNPolicy = TypeVar(\"TDQNPolicy\", bound=DiscreteQLearningPolicy)\n\n\nclass QLearningOffPolicyAlgorithm(\n OffPolicyAlgorithm[TDQNPolicy], LaggedNetworkFullUpdateAlgorithmMixin, ABC\n):\n \"\"\"\n Base class for Q-learning off-policy algorithms that use a Q-function to compute the\n n-step return.\n It optionally uses a lagged model, which is used as a target network and which is\n fully updated periodically.\n \"\"\"\n\n def __init__(\n self,\n *,\n policy: TDQNPolicy,\n optim: OptimizerFactory,\n gamma: float = 0.99,\n n_step_return_horizon: int = 1,\n target_update_freq: int = 0,\n ) -> None:\n \"\"\"\n :param policy: the policy\n :param optim: the optimizer factory for the policy's model.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param n_step_return_horizon: the number of future steps (> 0) to consider when computing temporal\n difference (TD) targets. Controls the balance between TD learning and Monte Carlo methods:\n higher values reduce bias (by relying less on potentially inaccurate value estimates)\n but increase variance (by incorporating more environmental stochasticity and reducing\n the averaging effect). A value of 1 corresponds to standard TD learning with immediate\n bootstrapping, while very large values approach Monte Carlo-like estimation that uses\n complete episode returns.\n :param target_update_freq: the number of training iterations between each complete update of\n the target network.\n Controls how frequently the target Q-network parameters are updated with the current\n Q-network values.\n A value of 0 disables the target network entirely, using only a single network for both\n action selection and bootstrap targets.\n Higher values provide more stable learning targets but slow down the propagation of new\n value estimates. Lower positive values allow faster learning but may lead to instability\n due to rapidly changing targets.\n Typically set between 100-10000 for DQN variants, with exact values depending on environment\n complexity.\n \"\"\"\n super().__init__(\n policy=policy,\n )\n self.optim = self._create_policy_optimizer(optim)\n LaggedNetworkFullUpdateAlgorithmMixin.__init__(self)\n assert 0.0 <= gamma <= 1.0, f\"discount factor should be in [0, 1] but got: {gamma}\"\n self.gamma = gamma\n assert n_step_return_horizon > 0, (\n f\"n_step_return_horizon should be greater than 0 but got: {n_step_return_horizon}\"\n )\n self.n_step = n_step_return_horizon\n self.target_update_freq = target_update_freq\n # TODO: 1 would be a more reasonable initialization given how it is incremented\n self._iter = 0\n self.model_old: EvalModeModuleWrapper | None = (\n self._add_lagged_network(self.policy.model) if self.use_target_network else None\n )\n\n def _create_policy_optimizer(self, optim: OptimizerFactory) -> Algorithm.Optimizer:\n return self._create_optimizer(self.policy, optim)\n\n @property\n def use_target_network(self) -> bool:\n return self.target_update_freq > 0\n\n @abstractmethod\n def _target_q(self, buffer: ReplayBuffer, indices: np.ndarray) -> torch.Tensor:\n pass\n\n def _preprocess_batch(\n self,\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n ) -> BatchWithReturnsProtocol:\n \"\"\"Compute the n-step return for Q-learning targets.\n\n More details can be found at\n :meth:`~tianshou.policy.BasePolicy.compute_nstep_return`.\n \"\"\"\n return self.compute_nstep_return(\n batch=batch,\n buffer=buffer,\n indices=indices,\n target_q_fn=self._target_q,\n gamma=self.gamma,\n n_step=self.n_step,\n )\n\n def _periodically_update_lagged_network_weights(self) -> None:\n \"\"\"\n Periodically updates the parameters of the lagged target network (if any), i.e.\n every n-th call (where n=`target_update_freq`), the target network's parameters\n are fully updated with the model's parameters.\n \"\"\"\n if self.use_target_network and self._iter % self.target_update_freq == 0:\n self._update_lagged_network_weights()\n self._iter += 1\n\n\nclass DQN(\n QLearningOffPolicyAlgorithm[TDQNPolicy],\n Generic[TDQNPolicy],\n):\n \"\"\"Implementation of Deep Q Network. arXiv:1312.5602.\n\n Implementation of Double Q-Learning. arXiv:1509.06461.\n\n Implementation of Dueling DQN. arXiv:1511.06581 (the dueling DQN is\n implemented in the network side, not here).\n \"\"\"\n\n def __init__(\n self,\n *,\n policy: TDQNPolicy,\n optim: OptimizerFactory,\n gamma: float = 0.99,\n n_step_return_horizon: int = 1,\n target_update_freq: int = 0,\n is_double: bool = True,\n huber_loss_delta: float | None = None,\n ) -> None:\n \"\"\"\n :param policy: the policy\n :param optim: the optimizer factory for the policy's model.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param n_step_return_horizon: the number of future steps (> 0) to consider when computing temporal\n difference (TD) targets. Controls the balance between TD learning and Monte Carlo methods:\n higher values reduce bias (by relying less on potentially inaccurate value estimates)\n but increase variance (by incorporating more environmental stochasticity and reducing\n the averaging effect). A value of 1 corresponds to standard TD learning with immediate\n bootstrapping, while very large values approach Monte Carlo-like estimation that uses\n complete episode returns.\n :param target_update_freq: the number of training iterations between each complete update of\n the target network.\n Controls how frequently the target Q-network parameters are updated with the current\n Q-network values.\n A value of 0 disables the target network entirely, using only a single network for both\n action selection and bootstrap targets.\n Higher values provide more stable learning targets but slow down the propagation of new\n value estimates. Lower positive values allow faster learning but may lead to instability\n due to rapidly changing targets.\n Typically set between 100-10000 for DQN variants, with exact values depending on environment\n complexity.\n :param is_double: flag indicating whether to use the Double DQN algorithm for target value computation.\n If True, the algorithm uses the online network to select actions and the target network to\n evaluate their Q-values. This approach helps reduce the overestimation bias in Q-learning\n by decoupling action selection from action evaluation.\n If False, the algorithm follows the vanilla DQN method that directly takes the maximum Q-value\n from the target network.\n Note: Double Q-learning will only be effective when a target network is used (target_update_freq > 0).\n :param huber_loss_delta: controls whether to use the Huber loss instead of the MSE loss for the TD error\n and the threshold for the Huber loss.\n If None, the MSE loss is used.\n If not None, uses the Huber loss as described in the Nature DQN paper (nature14236) with the given delta,\n which limits the influence of outliers.\n Unlike the MSE loss where the gradients grow linearly with the error magnitude, the Huber\n loss causes the gradients to plateau at a constant value for large errors, providing more stable training.\n NOTE: The magnitude of delta should depend on the scale of the returns obtained in the environment.\n \"\"\"\n super().__init__(\n policy=policy,\n optim=optim,\n gamma=gamma,\n n_step_return_horizon=n_step_return_horizon,\n target_update_freq=target_update_freq,\n )\n self.is_double = is_double\n self.huber_loss_delta = huber_loss_delta\n\n def _target_q(self, buffer: ReplayBuffer, indices: np.ndarray) -> torch.Tensor:\n obs_next_batch = Batch(\n obs=buffer[indices].obs_next,\n info=[None] * len(indices),\n ) # obs_next: s_{t+n}\n result = self.policy(obs_next_batch)\n if self.use_target_network:\n # target_Q = Q_old(s_, argmax(Q_new(s_, *)))\n target_q = self.policy(obs_next_batch, model=self.model_old).logits\n else:\n target_q = result.logits\n if self.is_double:\n return target_q[np.arange(len(result.act)), result.act]\n # Nature DQN, over estimate\n return target_q.max(dim=1)[0]\n\n def _update_with_batch(\n self,\n batch: RolloutBatchProtocol,\n ) -> SimpleLossTrainingStats:\n self._periodically_update_lagged_network_weights()\n weight = batch.pop(\"weight\", 1.0)\n q = self.policy(batch).logits\n q = q[np.arange(len(q)), batch.act]\n returns = to_torch_as(batch.returns.flatten(), q)\n td_error = returns - q\n\n if self.huber_loss_delta is not None:\n y = q.reshape(-1, 1)\n t = returns.reshape(-1, 1)\n loss = torch.nn.functional.huber_loss(\n y, t, delta=self.huber_loss_delta, reduction=\"mean\"\n )\n else:\n loss = (td_error.pow(2) * weight).mean()\n\n batch.weight = td_error # prio-buffer\n self.optim.step(loss)\n\n return SimpleLossTrainingStats(loss=loss.item())\n" }, { "path": "tianshou/algorithm/modelfree/fqf.py", "content": "from dataclasses import dataclass\nfrom typing import Any, cast\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nimport torch.nn.functional as F\nfrom overrides import override\n\nfrom tianshou.algorithm import QRDQN, Algorithm\nfrom tianshou.algorithm.modelfree.dqn import DiscreteQLearningPolicy\nfrom tianshou.algorithm.modelfree.qrdqn import QRDQNPolicy\nfrom tianshou.algorithm.modelfree.reinforce import SimpleLossTrainingStats\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import Batch, ReplayBuffer, to_numpy\nfrom tianshou.data.types import FQFBatchProtocol, ObsBatchProtocol, RolloutBatchProtocol\nfrom tianshou.utils.net.discrete import FractionProposalNetwork, FullQuantileFunction\n\n\n@dataclass(kw_only=True)\nclass FQFTrainingStats(SimpleLossTrainingStats):\n quantile_loss: float\n fraction_loss: float\n entropy_loss: float\n\n\nclass FQFPolicy(QRDQNPolicy):\n def __init__(\n self,\n *,\n model: FullQuantileFunction,\n fraction_model: FractionProposalNetwork,\n action_space: gym.spaces.Space,\n observation_space: gym.Space | None = None,\n eps_training: float = 0.0,\n eps_inference: float = 0.0,\n ):\n \"\"\"\n :param model: a model following the rules (s_B -> action_values_BA)\n :param fraction_model: a FractionProposalNetwork for\n proposing fractions/quantiles given state.\n :param action_space: the environment's action space\n :param observation_space: the environment's observation space.\n :param eps_training: the epsilon value for epsilon-greedy exploration during training.\n When collecting data for training, this is the probability of choosing a random action\n instead of the action chosen by the policy.\n A value of 0.0 means no exploration (fully greedy) and a value of 1.0 means full\n exploration (fully random).\n :param eps_inference: the epsilon value for epsilon-greedy exploration during inference,\n i.e. non-training cases (such as evaluation during test steps).\n The epsilon value is the probability of choosing a random action instead of the action\n chosen by the policy.\n A value of 0.0 means no exploration (fully greedy) and a value of 1.0 means full\n exploration (fully random).\n \"\"\"\n assert isinstance(action_space, gym.spaces.Discrete)\n super().__init__(\n model=model,\n action_space=action_space,\n observation_space=observation_space,\n eps_training=eps_training,\n eps_inference=eps_inference,\n )\n self.fraction_model = fraction_model\n\n def forward( # type: ignore\n self,\n batch: ObsBatchProtocol,\n state: dict | Batch | np.ndarray | None = None,\n model: FullQuantileFunction | None = None,\n fractions: Batch | None = None,\n **kwargs: Any,\n ) -> FQFBatchProtocol:\n if model is None:\n model = self.model\n obs = batch.obs\n # TODO: this is convoluted! See also other places where this is done\n obs_next = obs.obs if hasattr(obs, \"obs\") else obs\n if fractions is None:\n (logits, fractions, quantiles_tau), hidden = model(\n obs_next,\n propose_model=self.fraction_model,\n state=state,\n info=batch.info,\n )\n else:\n (logits, _, quantiles_tau), hidden = model(\n obs_next,\n propose_model=self.fraction_model,\n fractions=fractions,\n state=state,\n info=batch.info,\n )\n weighted_logits = (fractions.taus[:, 1:] - fractions.taus[:, :-1]).unsqueeze(1) * logits\n q = DiscreteQLearningPolicy.compute_q_value(\n self, weighted_logits.sum(2), getattr(obs, \"mask\", None)\n )\n act = to_numpy(q.max(dim=1)[1])\n result = Batch(\n logits=logits,\n act=act,\n state=hidden,\n fractions=fractions,\n quantiles_tau=quantiles_tau,\n )\n return cast(FQFBatchProtocol, result)\n\n\nclass FQF(QRDQN[FQFPolicy]):\n \"\"\"Implementation of Fully Parameterized Quantile Function for Distributional Reinforcement Learning. arXiv:1911.02140.\"\"\"\n\n def __init__(\n self,\n *,\n policy: FQFPolicy,\n optim: OptimizerFactory,\n fraction_optim: OptimizerFactory,\n gamma: float = 0.99,\n num_fractions: int = 32,\n ent_coef: float = 0.0,\n n_step_return_horizon: int = 1,\n target_update_freq: int = 0,\n ) -> None:\n \"\"\"\n :param policy: the policy\n :param optim: the optimizer factory for the policy's main Q-function model\n :param fraction_optim: the optimizer factory for the policy's fraction model\n :param action_space: the environment's action space.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param num_fractions: the number of fractions to use.\n :param ent_coef: coefficient that weights the entropy bonus relative to the actor loss.\n Controls the exploration-exploitation trade-off by encouraging policy entropy.\n Higher values promote more exploration by encouraging a more uniform action distribution.\n Lower values focus more on exploitation of the current policy's knowledge.\n Typically set between 0.01 and 0.05 for most actor-critic implementations.\n :param n_step_return_horizon: the number of future steps (> 0) to consider when computing temporal\n difference (TD) targets. Controls the balance between TD learning and Monte Carlo methods:\n higher values reduce bias (by relying less on potentially inaccurate value estimates)\n but increase variance (by incorporating more environmental stochasticity and reducing\n the averaging effect). A value of 1 corresponds to standard TD learning with immediate\n bootstrapping, while very large values approach Monte Carlo-like estimation that uses\n complete episode returns.\n :param target_update_freq: the number of training iterations between each complete update of\n the target network.\n Controls how frequently the target Q-network parameters are updated with the current\n Q-network values.\n A value of 0 disables the target network entirely, using only a single network for both\n action selection and bootstrap targets.\n Higher values provide more stable learning targets but slow down the propagation of new\n value estimates. Lower positive values allow faster learning but may lead to instability\n due to rapidly changing targets.\n Typically set between 100-10000 for DQN variants, with exact values depending on environment\n complexity.\n \"\"\"\n super().__init__(\n policy=policy,\n optim=optim,\n gamma=gamma,\n num_quantiles=num_fractions,\n n_step_return_horizon=n_step_return_horizon,\n target_update_freq=target_update_freq,\n )\n self.ent_coef = ent_coef\n self.fraction_optim = self._create_optimizer(self.policy.fraction_model, fraction_optim)\n\n @override\n def _create_policy_optimizer(self, optim: OptimizerFactory) -> Algorithm.Optimizer:\n # Override to leave out the fraction model (use main model only), as we want\n # to use a separate optimizer for the fraction model\n return self._create_optimizer(self.policy.model, optim)\n\n def _target_q(self, buffer: ReplayBuffer, indices: np.ndarray) -> torch.Tensor:\n obs_next_batch = Batch(\n obs=buffer[indices].obs_next,\n info=[None] * len(indices),\n ) # obs_next: s_{t+n}\n if self.use_target_network:\n result = self.policy(obs_next_batch)\n act, fractions = result.act, result.fractions\n next_dist = self.policy(\n obs_next_batch, model=self.model_old, fractions=fractions\n ).logits\n else:\n next_batch = self.policy(obs_next_batch)\n act = next_batch.act\n next_dist = next_batch.logits\n return next_dist[np.arange(len(act)), act, :]\n\n def _update_with_batch(\n self,\n batch: RolloutBatchProtocol,\n ) -> FQFTrainingStats:\n self._periodically_update_lagged_network_weights()\n weight = batch.pop(\"weight\", 1.0)\n out = self.policy(batch)\n curr_dist_orig = out.logits\n taus, tau_hats = out.fractions.taus, out.fractions.tau_hats\n act = batch.act\n curr_dist = curr_dist_orig[np.arange(len(act)), act, :].unsqueeze(2)\n target_dist = batch.returns.unsqueeze(1)\n # calculate each element's difference between curr_dist and target_dist\n dist_diff = F.smooth_l1_loss(target_dist, curr_dist, reduction=\"none\")\n huber_loss = (\n (\n dist_diff\n * (tau_hats.unsqueeze(2) - (target_dist - curr_dist).detach().le(0.0).float()).abs()\n )\n .sum(-1)\n .mean(1)\n )\n quantile_loss = (huber_loss * weight).mean()\n # ref: https://github.com/ku2482/fqf-iqn-qrdqn.pytorch/\n # blob/master/fqf_iqn_qrdqn/agent/qrdqn_agent.py L130\n batch.weight = dist_diff.detach().abs().sum(-1).mean(1) # prio-buffer\n # calculate fraction loss\n with torch.no_grad():\n sa_quantile_hats = curr_dist_orig[np.arange(len(act)), act, :]\n sa_quantiles = out.quantiles_tau[np.arange(len(act)), act, :]\n # ref: https://github.com/ku2482/fqf-iqn-qrdqn.pytorch/\n # blob/master/fqf_iqn_qrdqn/agent/fqf_agent.py L169\n values_1 = sa_quantiles - sa_quantile_hats[:, :-1]\n signs_1 = sa_quantiles > torch.cat(\n [sa_quantile_hats[:, :1], sa_quantiles[:, :-1]],\n dim=1,\n )\n\n values_2 = sa_quantiles - sa_quantile_hats[:, 1:]\n signs_2 = sa_quantiles < torch.cat(\n [sa_quantiles[:, 1:], sa_quantile_hats[:, -1:]],\n dim=1,\n )\n\n gradient_of_taus = torch.where(signs_1, values_1, -values_1) + torch.where(\n signs_2,\n values_2,\n -values_2,\n )\n fraction_loss = (gradient_of_taus * taus[:, 1:-1]).sum(1).mean()\n # calculate entropy loss\n entropy_loss = out.fractions.entropies.mean()\n fraction_entropy_loss = fraction_loss - self.ent_coef * entropy_loss\n self.fraction_optim.step(fraction_entropy_loss, retain_graph=True)\n self.optim.step(quantile_loss)\n\n return FQFTrainingStats(\n loss=quantile_loss.item() + fraction_entropy_loss.item(),\n quantile_loss=quantile_loss.item(),\n fraction_loss=fraction_loss.item(),\n entropy_loss=entropy_loss.item(),\n )\n" }, { "path": "tianshou/algorithm/modelfree/iqn.py", "content": "from typing import Any, cast\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nimport torch.nn.functional as F\n\nfrom tianshou.algorithm import QRDQN\nfrom tianshou.algorithm.modelfree.qrdqn import QRDQNPolicy\nfrom tianshou.algorithm.modelfree.reinforce import SimpleLossTrainingStats\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import Batch, to_numpy\nfrom tianshou.data.batch import BatchProtocol\nfrom tianshou.data.types import (\n ObsBatchProtocol,\n QuantileRegressionBatchProtocol,\n RolloutBatchProtocol,\n)\n\n\nclass IQNPolicy(QRDQNPolicy):\n def __init__(\n self,\n *,\n model: torch.nn.Module,\n action_space: gym.spaces.Space,\n sample_size: int = 32,\n online_sample_size: int = 8,\n target_sample_size: int = 8,\n observation_space: gym.Space | None = None,\n eps_training: float = 0.0,\n eps_inference: float = 0.0,\n ) -> None:\n \"\"\"\n :param model:\n :param action_space: the environment's action space\n :param sample_size:\n :param online_sample_size:\n :param target_sample_size:\n :param observation_space: the environment's observation space\n :param eps_training: the epsilon value for epsilon-greedy exploration during training.\n When collecting data for training, this is the probability of choosing a random action\n instead of the action chosen by the policy.\n A value of 0.0 means no exploration (fully greedy) and a value of 1.0 means full\n exploration (fully random).\n :param eps_inference: the epsilon value for epsilon-greedy exploration during inference,\n i.e. non-training cases (such as evaluation during test steps).\n The epsilon value is the probability of choosing a random action instead of the action\n chosen by the policy.\n A value of 0.0 means no exploration (fully greedy) and a value of 1.0 means full\n exploration (fully random).\n \"\"\"\n assert isinstance(action_space, gym.spaces.Discrete)\n assert sample_size > 1, f\"sample_size should be greater than 1 but got: {sample_size}\"\n assert online_sample_size > 1, (\n f\"online_sample_size should be greater than 1 but got: {online_sample_size}\"\n )\n assert target_sample_size > 1, (\n f\"target_sample_size should be greater than 1 but got: {target_sample_size}\"\n )\n super().__init__(\n model=model,\n action_space=action_space,\n observation_space=observation_space,\n eps_training=eps_training,\n eps_inference=eps_inference,\n )\n self.sample_size = sample_size\n self.online_sample_size = online_sample_size\n self.target_sample_size = target_sample_size\n\n def forward(\n self,\n batch: ObsBatchProtocol,\n state: dict | BatchProtocol | np.ndarray | None = None,\n model: torch.nn.Module | None = None,\n **kwargs: Any,\n ) -> QuantileRegressionBatchProtocol:\n is_model_old = model is not None\n if is_model_old:\n sample_size = self.target_sample_size\n elif self.training:\n sample_size = self.online_sample_size\n else:\n sample_size = self.sample_size\n if model is None:\n model = self.model\n obs = batch.obs\n # TODO: this seems very contrived!\n obs_next = obs.obs if hasattr(obs, \"obs\") else obs\n (logits, taus), hidden = model(\n obs_next,\n sample_size=sample_size,\n state=state,\n info=batch.info,\n )\n q = self.compute_q_value(logits, getattr(obs, \"mask\", None))\n act = to_numpy(q.max(dim=1)[1])\n result = Batch(logits=logits, act=act, state=hidden, taus=taus)\n return cast(QuantileRegressionBatchProtocol, result)\n\n\nclass IQN(QRDQN[IQNPolicy]):\n \"\"\"Implementation of Implicit Quantile Network. arXiv:1806.06923.\"\"\"\n\n def __init__(\n self,\n *,\n policy: IQNPolicy,\n optim: OptimizerFactory,\n gamma: float = 0.99,\n num_quantiles: int = 200,\n n_step_return_horizon: int = 1,\n target_update_freq: int = 0,\n ) -> None:\n \"\"\"\n :param policy: the policy\n :param optim: the optimizer factory for the policy's model\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param num_quantiles: the number of quantile midpoints in the inverse\n cumulative distribution function of the value.\n :param n_step_return_horizon: the number of future steps (> 0) to consider when computing temporal\n difference (TD) targets. Controls the balance between TD learning and Monte Carlo methods:\n higher values reduce bias (by relying less on potentially inaccurate value estimates)\n but increase variance (by incorporating more environmental stochasticity and reducing\n the averaging effect). A value of 1 corresponds to standard TD learning with immediate\n bootstrapping, while very large values approach Monte Carlo-like estimation that uses\n complete episode returns.\n :param target_update_freq: the number of training iterations between each complete update of\n the target network.\n Controls how frequently the target Q-network parameters are updated with the current\n Q-network values.\n A value of 0 disables the target network entirely, using only a single network for both\n action selection and bootstrap targets.\n Higher values provide more stable learning targets but slow down the propagation of new\n value estimates. Lower positive values allow faster learning but may lead to instability\n due to rapidly changing targets.\n Typically set between 100-10000 for DQN variants, with exact values depending on environment\n complexity.\n \"\"\"\n super().__init__(\n policy=policy,\n optim=optim,\n gamma=gamma,\n num_quantiles=num_quantiles,\n n_step_return_horizon=n_step_return_horizon,\n target_update_freq=target_update_freq,\n )\n\n def _update_with_batch(\n self,\n batch: RolloutBatchProtocol,\n ) -> SimpleLossTrainingStats:\n self._periodically_update_lagged_network_weights()\n weight = batch.pop(\"weight\", 1.0)\n action_batch = self.policy(batch)\n curr_dist, taus = action_batch.logits, action_batch.taus\n act = batch.act\n curr_dist = curr_dist[np.arange(len(act)), act, :].unsqueeze(2)\n target_dist = batch.returns.unsqueeze(1)\n # calculate each element's difference between curr_dist and target_dist\n dist_diff = F.smooth_l1_loss(target_dist, curr_dist, reduction=\"none\")\n huber_loss = (\n (\n dist_diff\n * (taus.unsqueeze(2) - (target_dist - curr_dist).detach().le(0.0).float()).abs()\n )\n .sum(-1)\n .mean(1)\n )\n loss = (huber_loss * weight).mean()\n # ref: https://github.com/ku2482/fqf-iqn-qrdqn.pytorch/\n # blob/master/fqf_iqn_qrdqn/agent/qrdqn_agent.py L130\n batch.weight = dist_diff.detach().abs().sum(-1).mean(1) # prio-buffer\n self.optim.step(loss)\n\n return SimpleLossTrainingStats(loss=loss.item())\n" }, { "path": "tianshou/algorithm/modelfree/npg.py", "content": "from dataclasses import dataclass\nfrom typing import Any\n\nimport numpy as np\nimport torch\nimport torch.nn.functional as F\nfrom torch import nn\nfrom torch.distributions import kl_divergence\n\nfrom tianshou.algorithm.algorithm_base import TrainingStats\nfrom tianshou.algorithm.modelfree.a2c import ActorCriticOnPolicyAlgorithm\nfrom tianshou.algorithm.modelfree.reinforce import ProbabilisticActorPolicy\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import ReplayBuffer, SequenceSummaryStats, to_torch_as\nfrom tianshou.data.types import BatchWithAdvantagesProtocol, RolloutBatchProtocol\nfrom tianshou.utils.net.continuous import ContinuousCritic\nfrom tianshou.utils.net.discrete import DiscreteCritic\n\n\n@dataclass(kw_only=True)\nclass NPGTrainingStats(TrainingStats):\n actor_loss: SequenceSummaryStats\n vf_loss: SequenceSummaryStats\n kl: SequenceSummaryStats\n\n\nclass NPG(ActorCriticOnPolicyAlgorithm):\n \"\"\"Implementation of Natural Policy Gradient.\n\n https://proceedings.neurips.cc/paper/2001/file/4b86abe48d358ecf194c56c69108433e-Paper.pdf\n \"\"\"\n\n def __init__(\n self,\n *,\n policy: ProbabilisticActorPolicy,\n critic: torch.nn.Module | ContinuousCritic | DiscreteCritic,\n optim: OptimizerFactory,\n optim_critic_iters: int = 5,\n trust_region_size: float = 0.5,\n advantage_normalization: bool = True,\n gae_lambda: float = 0.95,\n max_batchsize: int = 256,\n gamma: float = 0.99,\n return_scaling: bool = False,\n ) -> None:\n \"\"\"\n :param policy: the policy containing the actor network.\n :param critic: the critic network. (s -> V(s))\n :param optim: the optimizer factory for the critic network.\n :param optim_critic_iters: the number of optimization steps performed on the critic network\n for each policy (actor) update.\n Controls the learning rate balance between critic and actor.\n Higher values prioritize critic accuracy by training the value function more\n extensively before each policy update, which can improve stability but slow down\n training. Lower values maintain a more even learning pace between policy and value\n function but may lead to less reliable advantage estimates.\n Typically set between 1 and 10, depending on the complexity of the value function.\n :param trust_region_size: the parameter delta - a scalar multiplier for policy updates in the natural gradient direction.\n The mathematical meaning is the trust region size, which is the maximum KL divergence\n allowed between the old and new policy distributions.\n Controls how far the policy parameters move in the calculated direction\n during each update. Higher values allow for faster learning but may cause instability\n or policy deterioration; lower values provide more stable but slower learning. Unlike\n regular policy gradients, natural gradients already account for the local geometry of\n the parameter space, making this step size more robust to different parameterizations.\n Typically set between 0.1 and 1.0 for most reinforcement learning tasks.\n :param advantage_normalization: whether to do per mini-batch advantage\n normalization.\n :param gae_lambda: the lambda parameter in [0, 1] for generalized advantage estimation (GAE).\n Controls the bias-variance tradeoff in advantage estimates, acting as a\n weighting factor for combining different n-step advantage estimators. Higher values\n (closer to 1) reduce bias but increase variance by giving more weight to longer\n trajectories, while lower values (closer to 0) reduce variance but increase bias\n by relying more on the immediate TD error and value function estimates. At λ=0,\n GAE becomes equivalent to the one-step TD error (high bias, low variance); at λ=1,\n it becomes equivalent to Monte Carlo advantage estimation (low bias, high variance).\n Intermediate values create a weighted average of n-step returns, with exponentially\n decaying weights for longer-horizon returns. Typically set between 0.9 and 0.99 for\n most policy gradient methods.\n :param max_batchsize: the maximum number of samples to process at once when computing\n generalized advantage estimation (GAE) and value function predictions.\n Controls memory usage by breaking large batches into smaller chunks processed sequentially.\n Higher values may increase speed but require more GPU/CPU memory; lower values\n reduce memory requirements but may increase computation time. Should be adjusted\n based on available hardware resources and total batch size of your training data.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param return_scaling: flag indicating whether to enable scaling of estimated returns by\n dividing them by their running standard deviation without centering the mean.\n This reduces the magnitude variation of advantages across different episodes while\n preserving their signs and relative ordering.\n The use of running statistics (rather than batch-specific scaling) means that early\n training experiences may be scaled differently than later ones as the statistics evolve.\n When enabled, this improves training stability in environments with highly variable\n reward scales and makes the algorithm less sensitive to learning rate settings.\n However, it may reduce the algorithm's ability to distinguish between episodes with\n different absolute return magnitudes.\n Best used in environments where the relative ordering of actions is more important\n than the absolute scale of returns.\n \"\"\"\n super().__init__(\n policy=policy,\n critic=critic,\n optim=optim,\n optim_include_actor=False,\n gae_lambda=gae_lambda,\n max_batchsize=max_batchsize,\n gamma=gamma,\n return_scaling=return_scaling,\n )\n self.advantage_normalization = advantage_normalization\n self.optim_critic_iters = optim_critic_iters\n self.trust_region_size = trust_region_size\n # adjusts Hessian-vector product calculation for numerical stability\n self._damping = 0.1\n\n def _preprocess_batch(\n self,\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n ) -> BatchWithAdvantagesProtocol:\n batch = self._add_returns_and_advantages(batch, buffer, indices)\n batch.act = to_torch_as(batch.act, batch.v_s)\n old_log_prob = []\n with torch.no_grad():\n for minibatch in batch.split(self.max_batchsize, shuffle=False, merge_last=True):\n old_log_prob.append(self.policy(minibatch).dist.log_prob(minibatch.act))\n batch.logp_old = torch.cat(old_log_prob, dim=0)\n if self.advantage_normalization:\n batch.adv = (batch.adv - batch.adv.mean()) / batch.adv.std()\n return batch\n\n def _update_with_batch( # type: ignore[override]\n self,\n batch: BatchWithAdvantagesProtocol,\n batch_size: int | None,\n repeat: int,\n ) -> NPGTrainingStats:\n actor_losses, vf_losses, kls = [], [], []\n split_batch_size = batch_size or -1\n for _ in range(repeat):\n for minibatch in batch.split(split_batch_size, merge_last=True):\n # optimize actor\n # direction: calculate villia gradient\n dist = self.policy(minibatch).dist\n log_prob = dist.log_prob(minibatch.act)\n log_prob = log_prob.reshape(log_prob.size(0), -1).transpose(0, 1)\n actor_loss = -(log_prob * minibatch.adv).mean()\n flat_grads = self._get_flat_grad(\n actor_loss, self.policy.actor, retain_graph=True\n ).detach()\n\n # direction: calculate natural gradient\n with torch.no_grad():\n old_dist = self.policy(minibatch).dist\n\n kl = kl_divergence(old_dist, dist).mean()\n # calculate first order gradient of kl with respect to theta\n flat_kl_grad = self._get_flat_grad(kl, self.policy.actor, create_graph=True)\n search_direction = -self._conjugate_gradients(flat_grads, flat_kl_grad, nsteps=10)\n\n # step\n with torch.no_grad():\n flat_params = torch.cat(\n [param.data.view(-1) for param in self.policy.actor.parameters()],\n )\n new_flat_params = flat_params + self.trust_region_size * search_direction\n self._set_from_flat_params(self.policy.actor, new_flat_params)\n new_dist = self.policy(minibatch).dist\n kl = kl_divergence(old_dist, new_dist).mean()\n\n # optimize critic\n for _ in range(self.optim_critic_iters):\n value = self.critic(minibatch.obs).flatten()\n vf_loss = F.mse_loss(minibatch.returns, value)\n self.optim.step(vf_loss)\n\n actor_losses.append(actor_loss.item())\n vf_losses.append(vf_loss.item())\n kls.append(kl.item())\n\n return NPGTrainingStats(\n actor_loss=SequenceSummaryStats.from_sequence(actor_losses),\n vf_loss=SequenceSummaryStats.from_sequence(vf_losses),\n kl=SequenceSummaryStats.from_sequence(kls),\n )\n\n def _MVP(self, v: torch.Tensor, flat_kl_grad: torch.Tensor) -> torch.Tensor:\n \"\"\"Matrix vector product.\"\"\"\n # caculate second order gradient of kl with respect to theta\n kl_v = (flat_kl_grad * v).sum()\n flat_kl_grad_grad = self._get_flat_grad(kl_v, self.policy.actor, retain_graph=True).detach()\n return flat_kl_grad_grad + v * self._damping\n\n def _conjugate_gradients(\n self,\n minibatch: torch.Tensor,\n flat_kl_grad: torch.Tensor,\n nsteps: int = 10,\n residual_tol: float = 1e-10,\n ) -> torch.Tensor:\n x = torch.zeros_like(minibatch)\n r, p = minibatch.clone(), minibatch.clone()\n # Note: should be 'r, p = minibatch - MVP(x)', but for x=0, MVP(x)=0.\n # Change if doing warm start.\n rdotr = r.dot(r)\n for _ in range(nsteps):\n z = self._MVP(p, flat_kl_grad)\n alpha = rdotr / p.dot(z)\n x += alpha * p\n r -= alpha * z\n new_rdotr = r.dot(r)\n if new_rdotr < residual_tol:\n break\n p = r + new_rdotr / rdotr * p\n rdotr = new_rdotr\n return x\n\n def _get_flat_grad(self, y: torch.Tensor, model: nn.Module, **kwargs: Any) -> torch.Tensor:\n grads = torch.autograd.grad(y, model.parameters(), **kwargs) # type: ignore\n return torch.cat([grad.reshape(-1) for grad in grads])\n\n def _set_from_flat_params(self, model: nn.Module, flat_params: torch.Tensor) -> nn.Module:\n prev_ind = 0\n for param in model.parameters():\n flat_size = int(np.prod(list(param.size())))\n param.data.copy_(flat_params[prev_ind : prev_ind + flat_size].view(param.size()))\n prev_ind += flat_size\n return model\n" }, { "path": "tianshou/algorithm/modelfree/ppo.py", "content": "from typing import cast\n\nimport numpy as np\nimport torch\n\nfrom tianshou.algorithm import A2C\nfrom tianshou.algorithm.modelfree.a2c import A2CTrainingStats\nfrom tianshou.algorithm.modelfree.reinforce import ProbabilisticActorPolicy\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import ReplayBuffer, SequenceSummaryStats, to_torch_as\nfrom tianshou.data.types import LogpOldProtocol, RolloutBatchProtocol\nfrom tianshou.utils.net.continuous import ContinuousCritic\nfrom tianshou.utils.net.discrete import DiscreteCritic\n\n\nclass PPO(A2C):\n \"\"\"Implementation of Proximal Policy Optimization. arXiv:1707.06347.\"\"\"\n\n def __init__(\n self,\n *,\n policy: ProbabilisticActorPolicy,\n critic: torch.nn.Module | ContinuousCritic | DiscreteCritic,\n optim: OptimizerFactory,\n eps_clip: float = 0.2,\n dual_clip: float | None = None,\n value_clip: bool = False,\n advantage_normalization: bool = True,\n recompute_advantage: bool = False,\n vf_coef: float = 0.5,\n ent_coef: float = 0.01,\n max_grad_norm: float | None = None,\n gae_lambda: float = 0.95,\n max_batchsize: int = 256,\n gamma: float = 0.99,\n return_scaling: bool = False,\n ) -> None:\n r\"\"\"\n :param policy: the policy containing the actor network.\n :param critic: the critic network. (s -> V(s))\n :param optim: the optimizer factory for the policy's actor network and the critic networks.\n :param eps_clip: determines the range of allowed change in the policy during a policy update:\n The ratio of action probabilities indicated by the new and old policy is\n constrained to stay in the interval [1 - eps_clip, 1 + eps_clip].\n Small values thus force the new policy to stay close to the old policy.\n Typical values range between 0.1 and 0.3, the value of 0.2 is recommended\n in the original PPO paper.\n The optimal value depends on the environment; more stochastic environments may\n need larger values.\n :param dual_clip: a clipping parameter (denoted as c in the literature) that prevents\n excessive pessimism in policy updates for negative-advantage actions.\n Excessive pessimism occurs when the policy update too strongly reduces the probability\n of selecting actions that led to negative advantages, potentially eliminating useful\n actions based on limited negative experiences.\n When enabled (c > 1), the objective for negative advantages becomes:\n max(min(r(θ)A, clip(r(θ), 1-ε, 1+ε)A), c*A), where min(r(θ)A, clip(r(θ), 1-ε, 1+ε)A)\n is the original single-clipping objective determined by `eps_clip`.\n This creates a floor on negative policy gradients, maintaining some probability\n of exploring actions despite initial negative outcomes.\n Larger values (e.g., 2.0 to 5.0) maintain more exploration, while values closer\n to 1.0 provide less protection against pessimistic updates.\n Set to None to disable dual clipping.\n :param value_clip: flag indicating whether to enable clipping for value function updates.\n When enabled, restricts how much the value function estimate can change from its\n previous prediction, using the same clipping range as the policy updates (eps_clip).\n This stabilizes training by preventing large fluctuations in value estimates,\n particularly useful in environments with high reward variance.\n The clipped value loss uses a pessimistic approach, taking the maximum of the\n original and clipped value errors:\n max((returns - value)², (returns - v_clipped)²)\n Setting to True often improves training stability but may slow convergence.\n Implementation follows the approach mentioned in arXiv:1811.02553v3 Sec. 4.1.\n :param advantage_normalization: whether to do per mini-batch advantage\n normalization.\n :param recompute_advantage: whether to recompute advantage every update\n repeat according to https://arxiv.org/pdf/2006.05990.pdf Sec. 3.5.\n :param vf_coef: coefficient that weights the value loss relative to the actor loss in\n the overall loss function.\n Higher values prioritize accurate value function estimation over policy improvement.\n Controls the trade-off between policy optimization and value function fitting.\n Typically set between 0.5 and 1.0 for most actor-critic implementations.\n :param ent_coef: coefficient that weights the entropy bonus relative to the actor loss.\n Controls the exploration-exploitation trade-off by encouraging policy entropy.\n Higher values promote more exploration by encouraging a more uniform action distribution.\n Lower values focus more on exploitation of the current policy's knowledge.\n Typically set between 0.01 and 0.05 for most actor-critic implementations.\n :param max_grad_norm: the maximum L2 norm threshold for gradient clipping.\n When not None, gradients will be rescaled using to ensure their L2 norm does not\n exceed this value. This prevents exploding gradients and stabilizes training by\n limiting the magnitude of parameter updates.\n Set to None to disable gradient clipping.\n :param gae_lambda: the lambda parameter in [0, 1] for generalized advantage estimation (GAE).\n Controls the bias-variance tradeoff in advantage estimates, acting as a\n weighting factor for combining different n-step advantage estimators. Higher values\n (closer to 1) reduce bias but increase variance by giving more weight to longer\n trajectories, while lower values (closer to 0) reduce variance but increase bias\n by relying more on the immediate TD error and value function estimates. At λ=0,\n GAE becomes equivalent to the one-step TD error (high bias, low variance); at λ=1,\n it becomes equivalent to Monte Carlo advantage estimation (low bias, high variance).\n Intermediate values create a weighted average of n-step returns, with exponentially\n decaying weights for longer-horizon returns. Typically set between 0.9 and 0.99 for\n most policy gradient methods.\n :param max_batchsize: the maximum size of the batch when computing GAE.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param return_scaling: flag indicating whether to enable scaling of estimated returns by\n dividing them by their running standard deviation without centering the mean.\n This reduces the magnitude variation of advantages across different episodes while\n preserving their signs and relative ordering.\n The use of running statistics (rather than batch-specific scaling) means that early\n training experiences may be scaled differently than later ones as the statistics evolve.\n When enabled, this improves training stability in environments with highly variable\n reward scales and makes the algorithm less sensitive to learning rate settings.\n However, it may reduce the algorithm's ability to distinguish between episodes with\n different absolute return magnitudes.\n Best used in environments where the relative ordering of actions is more important\n than the absolute scale of returns.\n \"\"\"\n assert dual_clip is None or dual_clip > 1.0, (\n f\"Dual-clip PPO parameter should greater than 1.0 but got {dual_clip}\"\n )\n\n super().__init__(\n policy=policy,\n critic=critic,\n optim=optim,\n vf_coef=vf_coef,\n ent_coef=ent_coef,\n max_grad_norm=max_grad_norm,\n gae_lambda=gae_lambda,\n max_batchsize=max_batchsize,\n gamma=gamma,\n return_scaling=return_scaling,\n )\n self.eps_clip = eps_clip\n self.dual_clip = dual_clip\n self.value_clip = value_clip\n self.advantage_normalization = advantage_normalization\n self.recompute_adv = recompute_advantage\n\n def _preprocess_batch(\n self,\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n ) -> LogpOldProtocol:\n if self.recompute_adv:\n # buffer input `buffer` and `indices` to be used in `_update_with_batch()`.\n self._buffer, self._indices = buffer, indices\n batch = self._add_returns_and_advantages(batch, buffer, indices)\n batch.act = to_torch_as(batch.act, batch.v_s)\n logp_old = []\n with torch.no_grad():\n for minibatch in batch.split(self.max_batchsize, shuffle=False, merge_last=True):\n logp_old.append(self.policy(minibatch).dist.log_prob(minibatch.act))\n batch.logp_old = torch.cat(logp_old, dim=0).flatten()\n return cast(LogpOldProtocol, batch)\n\n def _update_with_batch( # type: ignore[override]\n self,\n batch: LogpOldProtocol,\n batch_size: int | None,\n repeat: int,\n ) -> A2CTrainingStats:\n losses, clip_losses, vf_losses, ent_losses = [], [], [], []\n gradient_steps = 0\n split_batch_size = batch_size or -1\n for step in range(repeat):\n if self.recompute_adv and step > 0:\n batch = cast(\n LogpOldProtocol,\n self._add_returns_and_advantages(batch, self._buffer, self._indices),\n )\n for minibatch in batch.split(split_batch_size, merge_last=True):\n gradient_steps += 1\n # calculate loss for actor\n advantages = minibatch.adv\n dist = self.policy(minibatch).dist\n if self.advantage_normalization:\n mean, std = advantages.mean(), advantages.std()\n advantages = (advantages - mean) / (std + self._eps) # per-batch norm\n ratios = (dist.log_prob(minibatch.act) - minibatch.logp_old).exp().float()\n ratios = ratios.reshape(ratios.size(0), -1).transpose(0, 1)\n surr1 = ratios * advantages\n surr2 = ratios.clamp(1.0 - self.eps_clip, 1.0 + self.eps_clip) * advantages\n if self.dual_clip:\n clip1 = torch.min(surr1, surr2)\n clip2 = torch.max(clip1, self.dual_clip * advantages)\n clip_loss = -torch.where(advantages < 0, clip2, clip1).mean()\n else:\n clip_loss = -torch.min(surr1, surr2).mean()\n # calculate loss for critic\n value = self.critic(minibatch.obs).flatten()\n if self.value_clip:\n v_clip = minibatch.v_s + (value - minibatch.v_s).clamp(\n -self.eps_clip,\n self.eps_clip,\n )\n vf1 = (minibatch.returns - value).pow(2)\n vf2 = (minibatch.returns - v_clip).pow(2)\n vf_loss = torch.max(vf1, vf2).mean()\n else:\n vf_loss = (minibatch.returns - value).pow(2).mean()\n # calculate regularization and overall loss\n ent_loss = dist.entropy().mean()\n loss = clip_loss + self.vf_coef * vf_loss - self.ent_coef * ent_loss\n self.optim.step(loss)\n clip_losses.append(clip_loss.item())\n vf_losses.append(vf_loss.item())\n ent_losses.append(ent_loss.item())\n losses.append(loss.item())\n\n return A2CTrainingStats(\n loss=SequenceSummaryStats.from_sequence(losses),\n actor_loss=SequenceSummaryStats.from_sequence(clip_losses),\n vf_loss=SequenceSummaryStats.from_sequence(vf_losses),\n ent_loss=SequenceSummaryStats.from_sequence(ent_losses),\n gradient_steps=gradient_steps,\n )\n" }, { "path": "tianshou/algorithm/modelfree/qrdqn.py", "content": "import warnings\nfrom typing import Generic, TypeVar\n\nimport numpy as np\nimport torch\nimport torch.nn.functional as F\n\nfrom tianshou.algorithm.modelfree.dqn import (\n DiscreteQLearningPolicy,\n QLearningOffPolicyAlgorithm,\n)\nfrom tianshou.algorithm.modelfree.reinforce import SimpleLossTrainingStats\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import Batch, ReplayBuffer\nfrom tianshou.data.types import RolloutBatchProtocol\n\n\nclass QRDQNPolicy(DiscreteQLearningPolicy):\n def compute_q_value(self, logits: torch.Tensor, mask: np.ndarray | None) -> torch.Tensor:\n return super().compute_q_value(logits.mean(2), mask)\n\n\nTQRDQNPolicy = TypeVar(\"TQRDQNPolicy\", bound=QRDQNPolicy)\n\n\nclass QRDQN(\n QLearningOffPolicyAlgorithm[TQRDQNPolicy],\n Generic[TQRDQNPolicy],\n):\n \"\"\"Implementation of Quantile Regression Deep Q-Network. arXiv:1710.10044.\"\"\"\n\n def __init__(\n self,\n *,\n policy: TQRDQNPolicy,\n optim: OptimizerFactory,\n gamma: float = 0.99,\n num_quantiles: int = 200,\n n_step_return_horizon: int = 1,\n target_update_freq: int = 0,\n ) -> None:\n \"\"\"\n :param policy: the policy\n :param optim: the optimizer factory for the policy's model.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param num_quantiles: the number of quantiles used to represent the return distribution for each action.\n Determines the granularity of the approximated distribution function.\n Higher values provide a more fine-grained approximation of the true return distribution but\n increase computational and memory requirements.\n Lower values reduce computational cost but may not capture the distribution accurately enough.\n The original QRDQN paper used 200 quantiles for Atari environments.\n Must be greater than 1, as at least two quantiles are needed to represent a distribution.\n :param n_step_return_horizon: the number of future steps (> 0) to consider when computing temporal\n difference (TD) targets. Controls the balance between TD learning and Monte Carlo methods:\n higher values reduce bias (by relying less on potentially inaccurate value estimates)\n but increase variance (by incorporating more environmental stochasticity and reducing\n the averaging effect). A value of 1 corresponds to standard TD learning with immediate\n bootstrapping, while very large values approach Monte Carlo-like estimation that uses\n complete episode returns.\n :param target_update_freq: the number of training iterations between each complete update of\n the target network.\n Controls how frequently the target Q-network parameters are updated with the current\n Q-network values.\n A value of 0 disables the target network entirely, using only a single network for both\n action selection and bootstrap targets.\n Higher values provide more stable learning targets but slow down the propagation of new\n value estimates. Lower positive values allow faster learning but may lead to instability\n due to rapidly changing targets.\n Typically set between 100-10000 for DQN variants, with exact values depending on environment\n complexity.\n \"\"\"\n assert num_quantiles > 1, f\"num_quantiles should be greater than 1 but got: {num_quantiles}\"\n super().__init__(\n policy=policy,\n optim=optim,\n gamma=gamma,\n n_step_return_horizon=n_step_return_horizon,\n target_update_freq=target_update_freq,\n )\n self.num_quantiles = num_quantiles\n tau = torch.linspace(0, 1, self.num_quantiles + 1)\n self.tau_hat = torch.nn.Parameter(\n ((tau[:-1] + tau[1:]) / 2).view(1, -1, 1),\n requires_grad=False,\n )\n warnings.filterwarnings(\"ignore\", message=\"Using a target size\")\n\n def _target_q(self, buffer: ReplayBuffer, indices: np.ndarray) -> torch.Tensor:\n obs_next_batch = Batch(\n obs=buffer[indices].obs_next,\n info=[None] * len(indices),\n ) # obs_next: s_{t+n}\n if self.use_target_network:\n act = self.policy(obs_next_batch).act\n next_dist = self.policy(obs_next_batch, model=self.model_old).logits\n else:\n next_batch = self.policy(obs_next_batch)\n act = next_batch.act\n next_dist = next_batch.logits\n return next_dist[np.arange(len(act)), act, :]\n\n def _update_with_batch(\n self,\n batch: RolloutBatchProtocol,\n ) -> SimpleLossTrainingStats:\n self._periodically_update_lagged_network_weights()\n weight = batch.pop(\"weight\", 1.0)\n curr_dist = self.policy(batch).logits\n act = batch.act\n curr_dist = curr_dist[np.arange(len(act)), act, :].unsqueeze(2)\n target_dist = batch.returns.unsqueeze(1)\n # calculate each element's difference between curr_dist and target_dist\n dist_diff = F.smooth_l1_loss(target_dist, curr_dist, reduction=\"none\")\n huber_loss = (\n (dist_diff * (self.tau_hat - (target_dist - curr_dist).detach().le(0.0).float()).abs())\n .sum(-1)\n .mean(1)\n )\n loss = (huber_loss * weight).mean()\n # ref: https://github.com/ku2482/fqf-iqn-qrdqn.pytorch/\n # blob/master/fqf_iqn_qrdqn/agent/qrdqn_agent.py L130\n batch.weight = dist_diff.detach().abs().sum(-1).mean(1) # prio-buffer\n self.optim.step(loss)\n\n return SimpleLossTrainingStats(loss=loss.item())\n" }, { "path": "tianshou/algorithm/modelfree/rainbow.py", "content": "from dataclasses import dataclass\n\nfrom torch import nn\n\nfrom tianshou.algorithm.modelfree.c51 import C51, C51Policy\nfrom tianshou.algorithm.modelfree.reinforce import LossSequenceTrainingStats\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data.types import RolloutBatchProtocol\nfrom tianshou.utils.lagged_network import EvalModeModuleWrapper\nfrom tianshou.utils.net.discrete import NoisyLinear\n\n\n@dataclass(kw_only=True)\nclass RainbowTrainingStats:\n loss: float\n\n\nclass RainbowDQN(C51):\n \"\"\"Implementation of Rainbow DQN. arXiv:1710.02298.\"\"\"\n\n def __init__(\n self,\n *,\n policy: C51Policy,\n optim: OptimizerFactory,\n gamma: float = 0.99,\n n_step_return_horizon: int = 1,\n target_update_freq: int = 0,\n ) -> None:\n \"\"\"\n :param policy: a policy following the rules (s -> action_values_BA)\n :param optim: the optimizer factory for the policy's model.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param n_step_return_horizon: the number of future steps (> 0) to consider when computing temporal\n difference (TD) targets. Controls the balance between TD learning and Monte Carlo methods:\n higher values reduce bias (by relying less on potentially inaccurate value estimates)\n but increase variance (by incorporating more environmental stochasticity and reducing\n the averaging effect). A value of 1 corresponds to standard TD learning with immediate\n bootstrapping, while very large values approach Monte Carlo-like estimation that uses\n complete episode returns.\n :param target_update_freq: the number of training iterations between each complete update of\n the target network.\n Controls how frequently the target Q-network parameters are updated with the current\n Q-network values.\n A value of 0 disables the target network entirely, using only a single network for both\n action selection and bootstrap targets.\n Higher values provide more stable learning targets but slow down the propagation of new\n value estimates. Lower positive values allow faster learning but may lead to instability\n due to rapidly changing targets.\n Typically set between 100-10000 for DQN variants, with exact values depending on environment\n complexity.\n \"\"\"\n super().__init__(\n policy=policy,\n optim=optim,\n gamma=gamma,\n n_step_return_horizon=n_step_return_horizon,\n target_update_freq=target_update_freq,\n )\n\n self.model_old: nn.Module | None # type: ignore[assignment]\n # Remove the wrapper that forces eval mode for the target network,\n # because Rainbow requires it to be set to train mode for sampling noise\n # in NoisyLinear layers to take effect.\n # (minor violation of Liskov Substitution Principle)\n if self.use_target_network:\n assert isinstance(self.model_old, EvalModeModuleWrapper)\n self.model_old = self.model_old.module\n\n @staticmethod\n def _sample_noise(model: nn.Module) -> bool:\n \"\"\"Sample the random noises of NoisyLinear modules in the model.\n\n Returns True if at least one NoisyLinear submodule was found.\n\n :param model: a PyTorch module which may have NoisyLinear submodules.\n :returns: True if model has at least one NoisyLinear submodule;\n otherwise, False.\n \"\"\"\n sampled_any_noise = False\n for m in model.modules():\n if isinstance(m, NoisyLinear):\n m.sample()\n sampled_any_noise = True\n return sampled_any_noise\n\n def _update_with_batch(\n self,\n batch: RolloutBatchProtocol,\n ) -> LossSequenceTrainingStats:\n self._sample_noise(self.policy.model)\n if self.use_target_network:\n assert self.model_old is not None\n self._sample_noise(self.model_old)\n return super()._update_with_batch(batch)\n" }, { "path": "tianshou/algorithm/modelfree/redq.py", "content": "from dataclasses import dataclass\nfrom typing import Any, Literal, TypeVar, cast\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.distributions import Independent, Normal\n\nfrom tianshou.algorithm.modelfree.ddpg import (\n ActorCriticOffPolicyAlgorithm,\n ContinuousPolicyWithExplorationNoise,\n DDPGTrainingStats,\n)\nfrom tianshou.algorithm.modelfree.sac import Alpha\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import Batch\nfrom tianshou.data.types import (\n DistLogProbBatchProtocol,\n ObsBatchProtocol,\n RolloutBatchProtocol,\n)\nfrom tianshou.exploration import BaseNoise\nfrom tianshou.utils.net.continuous import ContinuousActorProbabilistic\n\n\n@dataclass\nclass REDQTrainingStats(DDPGTrainingStats):\n \"\"\"A data structure for storing loss statistics of the REDQ learn step.\"\"\"\n\n alpha: float | None = None\n alpha_loss: float | None = None\n\n\nTREDQTrainingStats = TypeVar(\"TREDQTrainingStats\", bound=REDQTrainingStats)\n\n\nclass REDQPolicy(ContinuousPolicyWithExplorationNoise):\n def __init__(\n self,\n *,\n actor: torch.nn.Module | ContinuousActorProbabilistic,\n exploration_noise: BaseNoise | Literal[\"default\"] | None = None,\n action_space: gym.spaces.Space,\n deterministic_eval: bool = True,\n action_scaling: bool = True,\n action_bound_method: Literal[\"clip\"] | None = \"clip\",\n observation_space: gym.Space | None = None,\n ):\n \"\"\"\n :param actor: The actor network following the rules (s -> model_output)\n :param action_space: the environment's action_space.\n :param deterministic_eval: flag indicating whether the policy should use deterministic\n actions (using the mode of the action distribution) instead of stochastic ones\n (using random sampling) during evaluation.\n When enabled, the policy will always select the most probable action according to\n the learned distribution during evaluation phases, while still using stochastic\n sampling during training. This creates a clear distinction between exploration\n (training) and exploitation (evaluation) behaviors.\n Deterministic actions are generally preferred for final deployment and reproducible\n evaluation as they provide consistent behavior, reduce variance in performance\n metrics, and are more interpretable for human observers.\n Note that this parameter only affects behavior when the policy is not within a\n training step. When collecting rollouts for training, actions remain stochastic\n regardless of this setting to maintain proper exploration behaviour.\n :param observation_space: the environment's observation space\n :param action_scaling: flag indicating whether, for continuous action spaces, actions\n should be scaled from the standard neural network output range [-1, 1] to the\n environment's action space range [action_space.low, action_space.high].\n This applies to continuous action spaces only (gym.spaces.Box) and has no effect\n for discrete spaces.\n When enabled, policy outputs are expected to be in the normalized range [-1, 1]\n (after bounding), and are then linearly transformed to the actual required range.\n This improves neural network training stability, allows the same algorithm to work\n across environments with different action ranges, and standardizes exploration\n strategies.\n Should be disabled if the actor model already produces outputs in the correct range.\n :param action_bound_method: the method used for bounding actions in continuous action spaces\n to the range [-1, 1] before scaling them to the environment's action space (provided\n that `action_scaling` is enabled).\n This applies to continuous action spaces only (`gym.spaces.Box`) and should be set to None\n for discrete spaces.\n When set to \"clip\", actions exceeding the [-1, 1] range are simply clipped to this\n range. When set to \"tanh\", a hyperbolic tangent function is applied, which smoothly\n constrains outputs to [-1, 1] while preserving gradients.\n The choice of bounding method affects both training dynamics and exploration behavior.\n Clipping provides hard boundaries but may create plateau regions in the gradient\n landscape, while tanh provides smoother transitions but can compress sensitivity\n near the boundaries.\n Should be set to None if the actor model inherently produces bounded outputs.\n Typically used together with `action_scaling=True`.\n \"\"\"\n super().__init__(\n exploration_noise=exploration_noise,\n action_space=action_space,\n observation_space=observation_space,\n action_scaling=action_scaling,\n action_bound_method=action_bound_method,\n )\n self.actor = actor\n self.deterministic_eval = deterministic_eval\n self._eps = np.finfo(np.float32).eps.item()\n\n def forward( # type: ignore\n self,\n batch: ObsBatchProtocol,\n state: dict | Batch | np.ndarray | None = None,\n **kwargs: Any,\n ) -> DistLogProbBatchProtocol:\n (loc_B, scale_B), h_BH = self.actor(batch.obs, state=state, info=batch.info)\n dist = Independent(Normal(loc_B, scale_B), 1)\n if self.deterministic_eval and not self.is_within_training_step:\n act_B = dist.mode\n else:\n act_B = dist.rsample()\n log_prob = dist.log_prob(act_B).unsqueeze(-1)\n # apply correction for Tanh squashing when computing logprob from Gaussian\n # You can check out the original SAC paper (arXiv 1801.01290): Eq 21.\n # in appendix C to get some understanding of this equation.\n squashed_action = torch.tanh(act_B)\n log_prob = log_prob - torch.log((1 - squashed_action.pow(2)) + self._eps).sum(\n -1,\n keepdim=True,\n )\n result = Batch(\n logits=(loc_B, scale_B),\n act=squashed_action,\n state=h_BH,\n dist=dist,\n log_prob=log_prob,\n )\n return cast(DistLogProbBatchProtocol, result)\n\n\nclass REDQ(ActorCriticOffPolicyAlgorithm[REDQPolicy, DistLogProbBatchProtocol]):\n \"\"\"Implementation of REDQ. arXiv:2101.05982.\"\"\"\n\n def __init__(\n self,\n *,\n policy: REDQPolicy,\n policy_optim: OptimizerFactory,\n critic: torch.nn.Module,\n critic_optim: OptimizerFactory,\n ensemble_size: int = 10,\n subset_size: int = 2,\n tau: float = 0.005,\n gamma: float = 0.99,\n alpha: float | Alpha = 0.2,\n n_step_return_horizon: int = 1,\n actor_delay: int = 20,\n deterministic_eval: bool = True,\n target_mode: Literal[\"mean\", \"min\"] = \"min\",\n ) -> None:\n \"\"\"\n :param policy: the policy\n :param policy_optim: the optimizer factory for the policy's model.\n :param critic: the critic network. (s, a -> Q(s, a))\n :param critic_optim: the optimizer factory for the critic network.\n :param ensemble_size: the total number of critic networks in the ensemble.\n This parameter implements the randomized ensemble approach described in REDQ.\n The algorithm maintains `ensemble_size` different critic networks that all share the same\n architecture. During target value computation, a random subset of these networks (determined\n by `subset_size`) is used.\n Larger values increase the diversity of the ensemble but require more memory and computation.\n The original paper recommends a value of 10 for most tasks, balancing performance and\n computational efficiency.\n :param subset_size: the number of critic networks randomly selected from the ensemble for\n computing target Q-values.\n During each update, the algorithm samples `subset_size` networks from the ensemble of\n `ensemble_size` networks without replacement.\n The target Q-value is then calculated as either the minimum or mean (based on `target_mode`)\n of the predictions from this subset.\n Smaller values increase randomization and sample efficiency but may introduce more variance.\n Larger values provide more stable estimates but reduce the benefits of randomization.\n The REDQ paper recommends a value of 2 for optimal sample efficiency.\n Must satisfy 0 < subset_size <= ensemble_size.\n :param tau: the soft update coefficient for target networks, controlling the rate at which\n target networks track the learned networks.\n When the parameters of the target network are updated with the current (source) network's\n parameters, a weighted average is used: target = tau * source + (1 - tau) * target.\n Smaller values (closer to 0) create more stable but slower learning as target networks\n change more gradually. Higher values (closer to 1) allow faster learning but may reduce\n stability.\n Typically set to a small value (0.001 to 0.01) for most reinforcement learning tasks.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param alpha: the entropy regularization coefficient, which balances exploration and exploitation.\n This coefficient controls how much the agent values randomness in its policy versus\n pursuing higher rewards.\n Higher values (e.g., 0.5-1.0) strongly encourage exploration by rewarding the agent\n for maintaining diverse action choices, even if this means selecting some lower-value actions.\n Lower values (e.g., 0.01-0.1) prioritize exploitation, allowing the policy to become\n more focused on the highest-value actions.\n A value of 0 would completely remove entropy regularization, potentially leading to\n premature convergence to suboptimal deterministic policies.\n Can be provided as a fixed float (0.2 is a reasonable default) or as an instance of,\n in particular, class `AutoAlpha` for automatic tuning during training.\n :param n_step_return_horizon: the number of future steps (> 0) to consider when computing temporal\n difference (TD) targets. Controls the balance between TD learning and Monte Carlo methods:\n higher values reduce bias (by relying less on potentially inaccurate value estimates)\n but increase variance (by incorporating more environmental stochasticity and reducing\n the averaging effect). A value of 1 corresponds to standard TD learning with immediate\n bootstrapping, while very large values approach Monte Carlo-like estimation that uses\n complete episode returns.\n :param actor_delay: the number of critic updates performed before each actor update.\n The actor network is only updated once for every actor_delay critic updates, implementing\n a delayed policy update strategy similar to TD3.\n Larger values stabilize training by allowing critics to become more accurate before policy updates.\n Smaller values allow the policy to adapt more quickly but may lead to less stable learning.\n The REDQ paper recommends a value of 20 for most tasks.\n :param target_mode: the method used to aggregate Q-values from the subset of critic networks.\n Can be either \"min\" or \"mean\".\n If \"min\", uses the minimum Q-value across the selected subset of critics for each state-action pair.\n If \"mean\", uses the average Q-value across the selected subset of critics.\n Using \"min\" helps prevent overestimation bias but may lead to more conservative value estimates.\n Using \"mean\" provides more optimistic value estimates but may suffer from overestimation bias.\n Default is \"min\" following the conservative value estimation approach common in recent Q-learning\n algorithms.\n \"\"\"\n if target_mode not in (\"min\", \"mean\"):\n raise ValueError(f\"Unsupported target_mode: {target_mode}\")\n if not 0 < subset_size <= ensemble_size:\n raise ValueError(\n f\"Invalid choice of ensemble size or subset size. \"\n f\"Should be 0 < {subset_size=} <= {ensemble_size=}\",\n )\n super().__init__(\n policy=policy,\n policy_optim=policy_optim,\n critic=critic,\n critic_optim=critic_optim,\n tau=tau,\n gamma=gamma,\n n_step_return_horizon=n_step_return_horizon,\n )\n self.ensemble_size = ensemble_size\n self.subset_size = subset_size\n\n self.target_mode = target_mode\n self.critic_gradient_step = 0\n self.actor_delay = actor_delay\n self.deterministic_eval = deterministic_eval\n self.__eps = np.finfo(np.float32).eps.item()\n\n self._last_actor_loss = 0.0 # only for logging purposes\n\n self.alpha = Alpha.from_float_or_instance(alpha)\n\n def _target_q_compute_value(\n self, obs_batch: Batch, act_batch: DistLogProbBatchProtocol\n ) -> torch.Tensor:\n a_ = act_batch.act\n sample_ensemble_idx = np.random.choice(self.ensemble_size, self.subset_size, replace=False)\n qs = self.critic_old(obs_batch.obs, a_)[sample_ensemble_idx, ...]\n if self.target_mode == \"min\":\n target_q, _ = torch.min(qs, dim=0)\n elif self.target_mode == \"mean\":\n target_q = torch.mean(qs, dim=0)\n else:\n raise ValueError(f\"Invalid target_mode: {self.target_mode}\")\n\n target_q -= self.alpha.value * act_batch.log_prob\n\n return target_q\n\n def _update_with_batch(self, batch: RolloutBatchProtocol) -> REDQTrainingStats: # type: ignore\n # critic ensemble\n weight = getattr(batch, \"weight\", 1.0)\n current_qs = self.critic(batch.obs, batch.act).flatten(1)\n target_q = batch.returns.flatten()\n td = current_qs - target_q\n critic_loss = (td.pow(2) * weight).mean()\n self.critic_optim.step(critic_loss)\n batch.weight = torch.mean(td, dim=0) # prio-buffer\n self.critic_gradient_step += 1\n\n alpha_loss = None\n # actor\n if self.critic_gradient_step % self.actor_delay == 0:\n obs_result = self.policy(batch)\n a = obs_result.act\n current_qa = self.critic(batch.obs, a).mean(dim=0).flatten()\n actor_loss = (self.alpha.value * obs_result.log_prob.flatten() - current_qa).mean()\n self.policy_optim.step(actor_loss)\n\n # The entropy of a Gaussian policy can be expressed as -log_prob + a constant (which we ignore)\n entropy = -obs_result.log_prob.detach()\n alpha_loss = self.alpha.update(entropy)\n\n self._last_actor_loss = actor_loss.item()\n\n self._update_lagged_network_weights()\n\n return REDQTrainingStats(\n actor_loss=self._last_actor_loss,\n critic_loss=critic_loss.item(),\n alpha=self.alpha.value,\n alpha_loss=alpha_loss,\n )\n" }, { "path": "tianshou/algorithm/modelfree/reinforce.py", "content": "import logging\nimport warnings\nfrom collections.abc import Callable\nfrom dataclasses import dataclass\nfrom typing import Literal, TypeVar, cast\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\n\nfrom tianshou.algorithm import Algorithm\nfrom tianshou.algorithm.algorithm_base import (\n OnPolicyAlgorithm,\n Policy,\n TrainingStats,\n)\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import (\n Batch,\n ReplayBuffer,\n SequenceSummaryStats,\n to_torch,\n to_torch_as,\n)\nfrom tianshou.data.batch import BatchProtocol\nfrom tianshou.data.types import (\n BatchWithReturnsProtocol,\n DistBatchProtocol,\n ObsBatchProtocol,\n RolloutBatchProtocol,\n)\nfrom tianshou.utils import RunningMeanStd\nfrom tianshou.utils.net.common import (\n AbstractContinuousActorProbabilistic,\n AbstractDiscreteActor,\n ActionReprNet,\n)\nfrom tianshou.utils.net.discrete import dist_fn_categorical_from_logits\n\nlog = logging.getLogger(__name__)\n\n\n# Dimension Naming Convention\n# B - Batch Size\n# A - Action\n# D - Dist input (usually 2, loc and scale)\n# H - Dimension of hidden, can be None\n\nTDistFnContinuous = Callable[\n [tuple[torch.Tensor, torch.Tensor]],\n torch.distributions.Distribution,\n]\nTDistFnDiscrete = Callable[[torch.Tensor], torch.distributions.Distribution]\n\nTDistFnDiscrOrCont = TDistFnContinuous | TDistFnDiscrete\n\n\n@dataclass(kw_only=True)\nclass LossSequenceTrainingStats(TrainingStats):\n loss: SequenceSummaryStats\n\n\n@dataclass(kw_only=True)\nclass SimpleLossTrainingStats(TrainingStats):\n loss: float\n\n\nclass ProbabilisticActorPolicy(Policy):\n \"\"\"\n A policy that outputs (representations of) probability distributions from which\n actions can be sampled.\n \"\"\"\n\n def __init__(\n self,\n *,\n actor: AbstractContinuousActorProbabilistic | AbstractDiscreteActor | ActionReprNet,\n dist_fn: TDistFnDiscrOrCont,\n deterministic_eval: bool = False,\n action_space: gym.Space,\n observation_space: gym.Space | None = None,\n action_scaling: bool = True,\n action_bound_method: Literal[\"clip\", \"tanh\"] | None = \"clip\",\n ) -> None:\n \"\"\"\n :param actor: the actor network following the rules:\n If `self.action_type == \"discrete\"`: (`s_B` -> `action_values_BA`).\n If `self.action_type == \"continuous\"`: (`s_B` -> `dist_input_BD`).\n :param dist_fn: the function/type which creates a distribution from the actor output,\n i.e. it maps the tensor(s) generated by the actor to a torch distribution.\n For continuous action spaces, the output is typically a pair of tensors\n (mean, std) and the distribution is a Gaussian distribution.\n For discrete action spaces, the output is typically a tensor of unnormalized\n log probabilities (\"logits\" in PyTorch terminology) or a tensor of probabilities\n which can serve as the parameters of a Categorical distribution.\n Note that if the actor uses softmax activation in its final layer, it will produce\n probabilities, whereas if it uses no activation, it can be considered as producing\n \"logits\".\n As a user, you are responsible for ensuring that the distribution\n is compatible with the output of the actor model and the action space.\n :param deterministic_eval: flag indicating whether the policy should use deterministic\n actions (using the mode of the action distribution) instead of stochastic ones\n (using random sampling) during evaluation.\n When enabled, the policy will always select the most probable action according to\n the learned distribution during evaluation phases, while still using stochastic\n sampling during training. This creates a clear distinction between exploration\n (training) and exploitation (evaluation) behaviors.\n Deterministic actions are generally preferred for final deployment and reproducible\n evaluation as they provide consistent behavior, reduce variance in performance\n metrics, and are more interpretable for human observers.\n Note that this parameter only affects behavior when the policy is not within a\n training step. When collecting rollouts for training, actions remain stochastic\n regardless of this setting to maintain proper exploration behaviour.\n :param action_space: the environment's action space.\n :param observation_space: the environment's observation space.\n :param action_scaling: flag indicating whether, for continuous action spaces, actions\n should be scaled from the standard neural network output range [-1, 1] to the\n environment's action space range [action_space.low, action_space.high].\n This applies to continuous action spaces only (gym.spaces.Box) and has no effect\n for discrete spaces.\n When enabled, policy outputs are expected to be in the normalized range [-1, 1]\n (after bounding), and are then linearly transformed to the actual required range.\n This improves neural network training stability, allows the same algorithm to work\n across environments with different action ranges, and standardizes exploration\n strategies.\n Should be disabled if the actor model already produces outputs in the correct range.\n :param action_bound_method: the method used for bounding actions in continuous action spaces\n to the range [-1, 1] before scaling them to the environment's action space (provided\n that `action_scaling` is enabled).\n This applies to continuous action spaces only (`gym.spaces.Box`) and should be set to None\n for discrete spaces.\n When set to \"clip\", actions exceeding the [-1, 1] range are simply clipped to this\n range. When set to \"tanh\", a hyperbolic tangent function is applied, which smoothly\n constrains outputs to [-1, 1] while preserving gradients.\n The choice of bounding method affects both training dynamics and exploration behavior.\n Clipping provides hard boundaries but may create plateau regions in the gradient\n landscape, while tanh provides smoother transitions but can compress sensitivity\n near the boundaries.\n Should be set to None if the actor model inherently produces bounded outputs.\n Typically used together with `action_scaling=True`.\n \"\"\"\n super().__init__(\n action_space=action_space,\n observation_space=observation_space,\n action_scaling=action_scaling,\n action_bound_method=action_bound_method,\n )\n if action_scaling:\n try:\n max_action = float(actor.max_action)\n if np.isclose(max_action, 1.0):\n warnings.warn(\n \"action_scaling and action_bound_method are only intended \"\n \"to deal with unbounded model action space, but found actor model \"\n f\"bound action space with max_action={actor.max_action}. \"\n \"Consider using unbounded=True option of the actor model, \"\n \"or set action_scaling to False and action_bound_method to None.\",\n )\n except BaseException:\n pass\n\n self.actor = actor\n self.dist_fn = dist_fn\n self._eps = 1e-8\n self.deterministic_eval = deterministic_eval\n\n def forward(\n self,\n batch: ObsBatchProtocol,\n state: dict | BatchProtocol | np.ndarray | None = None,\n ) -> DistBatchProtocol:\n \"\"\"Compute action over the given batch data by applying the actor.\n\n Will sample from the dist_fn, if appropriate.\n Returns a new object representing the processed batch data\n (contrary to other methods that modify the input batch inplace).\n \"\"\"\n action_dist_input_BD, hidden_BH = self.actor(batch.obs, state=state, info=batch.info)\n # in the case that self.action_type == \"discrete\", the dist should always be Categorical, and D=A\n # therefore action_dist_input_BD is equivalent to logits_BA\n # If discrete, dist_fn will typically map loc, scale to a distribution (usually a Gaussian)\n # the action_dist_input_BD in that case is a tuple of loc_B, scale_B and needs to be unpacked\n dist = self.dist_fn(action_dist_input_BD)\n\n act_B = (\n dist.mode\n if self.deterministic_eval and not self.is_within_training_step\n else dist.sample()\n )\n # act is of dimension BA in continuous case and of dimension B in discrete\n result = Batch(logits=action_dist_input_BD, act=act_B, state=hidden_BH, dist=dist)\n return cast(DistBatchProtocol, result)\n\n\nclass DiscreteActorPolicy(ProbabilisticActorPolicy):\n def __init__(\n self,\n *,\n actor: AbstractDiscreteActor | ActionReprNet,\n dist_fn: TDistFnDiscrete = dist_fn_categorical_from_logits,\n deterministic_eval: bool = False,\n action_space: gym.Space,\n observation_space: gym.Space | None = None,\n ) -> None:\n \"\"\"\n :param actor: the actor network following the rules: (`s_B` -> `dist_input_BD`).\n :param dist_fn: the function/type which creates a distribution from the actor output,\n i.e. it maps the tensor(s) generated by the actor to a torch distribution.\n For discrete action spaces, the output is typically a tensor of unnormalized\n log probabilities (\"logits\" in PyTorch terminology) or a tensor of probabilities\n which serve as the parameters of a Categorical distribution.\n Note that if the actor uses softmax activation in its final layer, it will produce\n probabilities, whereas if it uses no activation, it can be considered as producing\n \"logits\".\n As a user, you are responsible for ensuring that the distribution\n is compatible with the output of the actor model and the action space.\n :param deterministic_eval: flag indicating whether the policy should use deterministic\n actions (using the mode of the action distribution) instead of stochastic ones\n (using random sampling) during evaluation.\n When enabled, the policy will always select the most probable action according to\n the learned distribution during evaluation phases, while still using stochastic\n sampling during training. This creates a clear distinction between exploration\n (training) and exploitation (evaluation) behaviors.\n Deterministic actions are generally preferred for final deployment and reproducible\n evaluation as they provide consistent behavior, reduce variance in performance\n metrics, and are more interpretable for human observers.\n Note that this parameter only affects behavior when the policy is not within a\n training step. When collecting rollouts for training, actions remain stochastic\n regardless of this setting to maintain proper exploration behaviour.\n :param action_space: the environment's (discrete) action space.\n :param observation_space: the environment's observation space.\n \"\"\"\n if not isinstance(action_space, gym.spaces.Discrete):\n raise ValueError(f\"Action space must be an instance of Discrete; got {action_space}\")\n super().__init__(\n actor=actor,\n dist_fn=dist_fn,\n deterministic_eval=deterministic_eval,\n action_space=action_space,\n observation_space=observation_space,\n action_scaling=False,\n action_bound_method=None,\n )\n\n\nTActorPolicy = TypeVar(\"TActorPolicy\", bound=ProbabilisticActorPolicy)\n\n\nclass DiscountedReturnComputation:\n def __init__(\n self,\n gamma: float = 0.99,\n return_standardization: bool = False,\n ):\n \"\"\"\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param return_standardization: whether to standardize episode returns\n by subtracting the running mean and dividing by the running standard deviation.\n Note that this is known to be detrimental to performance in many cases!\n \"\"\"\n assert 0.0 <= gamma <= 1.0, \"discount factor gamma should be in [0, 1]\"\n self.gamma = gamma\n self.return_standardization = return_standardization\n self.ret_rms = RunningMeanStd()\n self.eps = 1e-8\n\n def add_discounted_returns(\n self, batch: RolloutBatchProtocol, buffer: ReplayBuffer, indices: np.ndarray\n ) -> BatchWithReturnsProtocol:\n r\"\"\"Compute the discounted returns (Monte Carlo estimates) for each transition.\n\n They are added to the batch under the field `returns`.\n Note: this function will modify the input batch!\n\n .. math::\n G_t = \\sum_{i=t}^T \\gamma^{i-t}r_i\n\n where :math:`T` is the terminal time step, :math:`\\gamma` is the\n discount factor, :math:`\\gamma \\in [0, 1]`.\n\n :param batch: a data batch which contains several episodes of data in\n sequential order. Mind that the end of each finished episode of batch\n should be marked by done flag, unfinished (or collecting) episodes will be\n recognized by buffer.unfinished_index().\n :param buffer: the corresponding replay buffer.\n :param indices: tell batch's location in buffer, batch is equal\n to buffer[indices].\n \"\"\"\n v_s_ = np.full(indices.shape, self.ret_rms.mean)\n # gae_lambda = 1.0 means we use Monte Carlo estimate\n unnormalized_returns, _ = Algorithm.compute_episodic_return(\n batch,\n buffer,\n indices,\n v_s_=v_s_,\n gamma=self.gamma,\n gae_lambda=1.0,\n )\n if self.return_standardization:\n batch.returns = (unnormalized_returns - self.ret_rms.mean) / np.sqrt(\n self.ret_rms.var + self.eps,\n )\n self.ret_rms.update(unnormalized_returns)\n else:\n batch.returns = unnormalized_returns\n return cast(BatchWithReturnsProtocol, batch)\n\n\nclass Reinforce(OnPolicyAlgorithm[ProbabilisticActorPolicy]):\n \"\"\"Implementation of the REINFORCE (a.k.a. vanilla policy gradient) algorithm.\"\"\"\n\n def __init__(\n self,\n *,\n policy: ProbabilisticActorPolicy,\n gamma: float = 0.99,\n return_standardization: bool = False,\n optim: OptimizerFactory,\n ) -> None:\n \"\"\"\n :param policy: the policy\n :param optim: the optimizer factory for the policy's model.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param return_standardization: if True, will scale/standardize returns\n by subtracting the running mean and dividing by the running standard deviation.\n Can be detrimental to performance!\n \"\"\"\n super().__init__(\n policy=policy,\n )\n self.discounted_return_computation = DiscountedReturnComputation(\n gamma=gamma,\n return_standardization=return_standardization,\n )\n self.optim = self._create_optimizer(self.policy, optim)\n\n def _preprocess_batch(\n self,\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n ) -> BatchWithReturnsProtocol:\n return self.discounted_return_computation.add_discounted_returns(\n batch,\n buffer,\n indices,\n )\n\n # Needs BatchWithReturnsProtocol, which violates the substitution principle. But not a problem since it's a private method and\n # the remainder of the class was adjusted to provide the correct batch\n def _update_with_batch( # type: ignore[override]\n self,\n batch: BatchWithReturnsProtocol,\n batch_size: int | None,\n repeat: int,\n ) -> LossSequenceTrainingStats:\n losses = []\n split_batch_size = batch_size or -1\n for _ in range(repeat):\n for minibatch in batch.split(split_batch_size, merge_last=True):\n result = self.policy(minibatch)\n dist = result.dist\n act = to_torch_as(minibatch.act, result.act)\n ret = to_torch(minibatch.returns, torch.float, result.act.device)\n log_prob = dist.log_prob(act).reshape(len(ret), -1).transpose(0, 1)\n loss = -(log_prob * ret).mean()\n self.optim.step(loss)\n losses.append(loss.item())\n\n return LossSequenceTrainingStats(loss=SequenceSummaryStats.from_sequence(losses))\n" }, { "path": "tianshou/algorithm/modelfree/sac.py", "content": "from abc import ABC, abstractmethod\nfrom dataclasses import dataclass\nfrom typing import Any, Generic, Literal, TypeVar, Union, cast\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom torch.distributions import Independent, Normal\n\nfrom tianshou.algorithm.algorithm_base import TrainingStats\nfrom tianshou.algorithm.modelfree.ddpg import ContinuousPolicyWithExplorationNoise\nfrom tianshou.algorithm.modelfree.td3 import ActorDualCriticsOffPolicyAlgorithm\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import Batch\nfrom tianshou.data.types import (\n DistLogProbBatchProtocol,\n ObsBatchProtocol,\n RolloutBatchProtocol,\n)\nfrom tianshou.exploration import BaseNoise\nfrom tianshou.utils.conversion import to_optional_float\nfrom tianshou.utils.net.continuous import ContinuousActorProbabilistic\n\n\ndef correct_log_prob_gaussian_tanh(\n log_prob: torch.Tensor,\n tanh_squashed_action: torch.Tensor,\n eps: float = np.finfo(np.float32).eps.item(),\n) -> torch.Tensor:\n \"\"\"Apply correction for Tanh squashing when computing `log_prob` from Gaussian.\n\n See equation 21 in the original `SAC paper `_.\n\n :param log_prob: log probability of the action\n :param tanh_squashed_action: action squashed to values in (-1, 1) range by tanh\n :param eps: epsilon for numerical stability\n \"\"\"\n log_prob_correction = torch.log(1 - tanh_squashed_action.pow(2) + eps).sum(-1, keepdim=True)\n return log_prob - log_prob_correction\n\n\n@dataclass(kw_only=True)\nclass SACTrainingStats(TrainingStats):\n actor_loss: float\n critic1_loss: float\n critic2_loss: float\n alpha: float | None = None\n alpha_loss: float | None = None\n\n\nTSACTrainingStats = TypeVar(\"TSACTrainingStats\", bound=SACTrainingStats)\n\n\nclass SACPolicy(ContinuousPolicyWithExplorationNoise):\n def __init__(\n self,\n *,\n actor: torch.nn.Module | ContinuousActorProbabilistic,\n exploration_noise: BaseNoise | Literal[\"default\"] | None = None,\n deterministic_eval: bool = True,\n action_scaling: bool = True,\n action_space: gym.Space,\n observation_space: gym.Space | None = None,\n ):\n \"\"\"\n :param actor: the actor network following the rules (s -> dist_input_BD)\n :param exploration_noise: add noise to action for exploration.\n This is useful when solving \"hard exploration\" problems.\n \"default\" is equivalent to GaussianNoise(sigma=0.1).\n :param deterministic_eval: flag indicating whether the policy should use deterministic\n actions (using the mode of the action distribution) instead of stochastic ones\n (using random sampling) during evaluation.\n When enabled, the policy will always select the most probable action according to\n the learned distribution during evaluation phases, while still using stochastic\n sampling during training. This creates a clear distinction between exploration\n (training) and exploitation (evaluation) behaviors.\n Deterministic actions are generally preferred for final deployment and reproducible\n evaluation as they provide consistent behavior, reduce variance in performance\n metrics, and are more interpretable for human observers.\n Note that this parameter only affects behavior when the policy is not within a\n training step. When collecting rollouts for training, actions remain stochastic\n regardless of this setting to maintain proper exploration behaviour.\n :param action_scaling: flag indicating whether, for continuous action spaces, actions\n should be scaled from the standard neural network output range [-1, 1] to the\n environment's action space range [action_space.low, action_space.high].\n This applies to continuous action spaces only (gym.spaces.Box) and has no effect\n for discrete spaces.\n When enabled, policy outputs are expected to be in the normalized range [-1, 1]\n (after bounding), and are then linearly transformed to the actual required range.\n This improves neural network training stability, allows the same algorithm to work\n across environments with different action ranges, and standardizes exploration\n strategies.\n Should be disabled if the actor model already produces outputs in the correct range.\n :param action_space: the environment's action_space.\n :param observation_space: the environment's observation space\n \"\"\"\n super().__init__(\n exploration_noise=exploration_noise,\n action_space=action_space,\n observation_space=observation_space,\n action_scaling=action_scaling,\n # actions already squashed by tanh\n action_bound_method=None,\n )\n self.actor = actor\n self.deterministic_eval = deterministic_eval\n\n def forward( # type: ignore\n self,\n batch: ObsBatchProtocol,\n state: dict | Batch | np.ndarray | None = None,\n **kwargs: Any,\n ) -> DistLogProbBatchProtocol:\n (loc_B, scale_B), hidden_BH = self.actor(batch.obs, state=state, info=batch.info)\n dist = Independent(Normal(loc=loc_B, scale=scale_B), 1)\n if self.deterministic_eval and not self.is_within_training_step:\n act_B = dist.mode\n else:\n act_B = dist.rsample()\n log_prob = dist.log_prob(act_B).unsqueeze(-1)\n\n squashed_action = torch.tanh(act_B)\n log_prob = correct_log_prob_gaussian_tanh(log_prob, squashed_action)\n result = Batch(\n logits=(loc_B, scale_B),\n act=squashed_action,\n state=hidden_BH,\n dist=dist,\n log_prob=log_prob,\n )\n return cast(DistLogProbBatchProtocol, result)\n\n\nclass Alpha(ABC):\n \"\"\"Defines the interface for the entropy regularization coefficient alpha.\"\"\"\n\n @staticmethod\n def from_float_or_instance(alpha: Union[float, \"Alpha\"]) -> \"Alpha\":\n if isinstance(alpha, float):\n return FixedAlpha(alpha)\n elif isinstance(alpha, Alpha):\n return alpha\n else:\n raise ValueError(f\"Expected float or Alpha instance, but got {alpha=}\")\n\n @property\n @abstractmethod\n def value(self) -> float:\n \"\"\"Retrieves the current value of alpha.\"\"\"\n\n @abstractmethod\n def update(self, entropy: torch.Tensor) -> float | None:\n \"\"\"\n Updates the alpha value based on the entropy.\n\n :param entropy: the entropy of the policy.\n :return: the loss value if alpha is auto-tuned, otherwise None.\n \"\"\"\n return None\n\n\nclass FixedAlpha(Alpha):\n \"\"\"Represents a fixed entropy regularization coefficient alpha.\"\"\"\n\n def __init__(self, alpha: float):\n self._value = alpha\n\n @property\n def value(self) -> float:\n return self._value\n\n def update(self, entropy: torch.Tensor) -> float | None:\n return None\n\n\nclass AutoAlpha(torch.nn.Module, Alpha):\n \"\"\"Represents an entropy regularization coefficient alpha that is automatically tuned.\"\"\"\n\n def __init__(self, target_entropy: float, log_alpha: float, optim: OptimizerFactory):\n \"\"\"\n :param target_entropy: the target entropy value.\n For discrete action spaces, it is usually `-log(|A|)` for a balance between stochasticity\n and determinism or `-log(1/|A|)=log(|A|)` for maximum stochasticity or, more generally,\n `lambda*log(|A|)`, e.g. with `lambda` close to 1 (e.g. 0.98) for pronounced stochasticity.\n For continuous action spaces, it is usually `-dim(A)` for a balance between stochasticity\n and determinism, with similar generalizations as for discrete action spaces.\n :param log_alpha: the (initial) value of the log of the entropy regularization coefficient alpha.\n :param optim: the factory with which to create the optimizer for `log_alpha`.\n \"\"\"\n super().__init__()\n self._target_entropy = target_entropy\n self._log_alpha = torch.nn.Parameter(torch.tensor(log_alpha))\n self._optim, lr_scheduler = optim.create_instances(self)\n if lr_scheduler is not None:\n raise ValueError(\n f\"Learning rate schedulers are not supported by {self.__class__.__name__}\"\n )\n\n @property\n def value(self) -> float:\n return self._log_alpha.detach().exp().item()\n\n def update(self, entropy: torch.Tensor) -> float:\n entropy_deficit = self._target_entropy - entropy\n alpha_loss = -(self._log_alpha * entropy_deficit).mean()\n self._optim.zero_grad()\n alpha_loss.backward()\n self._optim.step()\n return alpha_loss.item()\n\n\nclass SAC(\n ActorDualCriticsOffPolicyAlgorithm[SACPolicy, DistLogProbBatchProtocol],\n Generic[TSACTrainingStats],\n):\n \"\"\"Implementation of Soft Actor-Critic. arXiv:1812.05905.\"\"\"\n\n def __init__(\n self,\n *,\n policy: SACPolicy,\n policy_optim: OptimizerFactory,\n critic: torch.nn.Module,\n critic_optim: OptimizerFactory,\n critic2: torch.nn.Module | None = None,\n critic2_optim: OptimizerFactory | None = None,\n tau: float = 0.005,\n gamma: float = 0.99,\n alpha: float | Alpha = 0.2,\n n_step_return_horizon: int = 1,\n deterministic_eval: bool = True,\n ) -> None:\n \"\"\"\n :param policy: the policy\n :param policy_optim: the optimizer factory for the policy's model.\n :param critic: the first critic network. (s, a -> Q(s, a))\n :param critic_optim: the optimizer factory for the first critic network.\n :param critic2: the second critic network. (s, a -> Q(s, a)).\n If None, copy the first critic (via deepcopy).\n :param critic2_optim: the optimizer factory for the second critic network.\n If None, use the first critic's factory.\n :param tau: the soft update coefficient for target networks, controlling the rate at which\n target networks track the learned networks.\n When the parameters of the target network are updated with the current (source) network's\n parameters, a weighted average is used: target = tau * source + (1 - tau) * target.\n Smaller values (closer to 0) create more stable but slower learning as target networks\n change more gradually. Higher values (closer to 1) allow faster learning but may reduce\n stability.\n Typically set to a small value (0.001 to 0.01) for most reinforcement learning tasks.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param alpha: the entropy regularization coefficient, which balances exploration and exploitation.\n This coefficient controls how much the agent values randomness in its policy versus\n pursuing higher rewards.\n Higher values (e.g., 0.5-1.0) strongly encourage exploration by rewarding the agent\n for maintaining diverse action choices, even if this means selecting some lower-value actions.\n Lower values (e.g., 0.01-0.1) prioritize exploitation, allowing the policy to become\n more focused on the highest-value actions.\n A value of 0 would completely remove entropy regularization, potentially leading to\n premature convergence to suboptimal deterministic policies.\n Can be provided as a fixed float (0.2 is a reasonable default) or as an instance of,\n in particular, class `AutoAlpha` for automatic tuning during training.\n :param n_step_return_horizon: the number of future steps (> 0) to consider when computing temporal\n difference (TD) targets. Controls the balance between TD learning and Monte Carlo methods:\n higher values reduce bias (by relying less on potentially inaccurate value estimates)\n but increase variance (by incorporating more environmental stochasticity and reducing\n the averaging effect). A value of 1 corresponds to standard TD learning with immediate\n bootstrapping, while very large values approach Monte Carlo-like estimation that uses\n complete episode returns.\n \"\"\"\n super().__init__(\n policy=policy,\n policy_optim=policy_optim,\n critic=critic,\n critic_optim=critic_optim,\n critic2=critic2,\n critic2_optim=critic2_optim,\n tau=tau,\n gamma=gamma,\n n_step_return_horizon=n_step_return_horizon,\n )\n self.deterministic_eval = deterministic_eval\n self.alpha = Alpha.from_float_or_instance(alpha)\n self._check_field_validity()\n\n def _check_field_validity(self) -> None:\n if not isinstance(self.policy.action_space, gym.spaces.Box):\n raise ValueError(\n f\"SACPolicy only supports gym.spaces.Box, but got {self.action_space=}.\"\n f\"Please use DiscreteSACPolicy for discrete action spaces.\",\n )\n\n def _target_q_compute_value(\n self, obs_batch: Batch, act_batch: DistLogProbBatchProtocol\n ) -> torch.Tensor:\n min_q_value = super()._target_q_compute_value(obs_batch, act_batch)\n return min_q_value - self.alpha.value * act_batch.log_prob\n\n def _update_with_batch(self, batch: RolloutBatchProtocol) -> TSACTrainingStats: # type: ignore\n # critic 1&2\n td1, critic1_loss = self._minimize_critic_squared_loss(\n batch, self.critic, self.critic_optim\n )\n td2, critic2_loss = self._minimize_critic_squared_loss(\n batch, self.critic2, self.critic2_optim\n )\n batch.weight = (td1 + td2) / 2.0 # prio-buffer\n\n # actor\n obs_result = self.policy(batch)\n act = obs_result.act\n current_q1a = self.critic(batch.obs, act).flatten()\n current_q2a = self.critic2(batch.obs, act).flatten()\n actor_loss = (\n self.alpha.value * obs_result.log_prob.flatten() - torch.min(current_q1a, current_q2a)\n ).mean()\n self.policy_optim.step(actor_loss)\n\n # The entropy of a Gaussian policy can be expressed as -log_prob + a constant (which we ignore)\n entropy = -obs_result.log_prob.detach()\n alpha_loss = self.alpha.update(entropy)\n\n self._update_lagged_network_weights()\n\n return SACTrainingStats( # type: ignore[return-value]\n actor_loss=actor_loss.item(),\n critic1_loss=critic1_loss.item(),\n critic2_loss=critic2_loss.item(),\n alpha=to_optional_float(self.alpha.value),\n alpha_loss=to_optional_float(alpha_loss),\n )\n" }, { "path": "tianshou/algorithm/modelfree/td3.py", "content": "from abc import ABC\nfrom copy import deepcopy\nfrom dataclasses import dataclass\nfrom typing import Any\n\nimport torch\n\nfrom tianshou.algorithm.algorithm_base import (\n TPolicy,\n TrainingStats,\n)\nfrom tianshou.algorithm.modelfree.ddpg import (\n ActorCriticOffPolicyAlgorithm,\n ContinuousDeterministicPolicy,\n TActBatchProtocol,\n)\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import Batch\nfrom tianshou.data.types import (\n ActStateBatchProtocol,\n RolloutBatchProtocol,\n)\n\n\n@dataclass(kw_only=True)\nclass TD3TrainingStats(TrainingStats):\n actor_loss: float\n critic1_loss: float\n critic2_loss: float\n\n\nclass ActorDualCriticsOffPolicyAlgorithm(\n ActorCriticOffPolicyAlgorithm[TPolicy, TActBatchProtocol],\n ABC,\n):\n \"\"\"A base class for off-policy algorithms with two critics, where the target Q-value is computed as the minimum\n of the two lagged critics' values.\n \"\"\"\n\n def __init__(\n self,\n *,\n policy: Any,\n policy_optim: OptimizerFactory,\n critic: torch.nn.Module,\n critic_optim: OptimizerFactory,\n critic2: torch.nn.Module | None = None,\n critic2_optim: OptimizerFactory | None = None,\n tau: float = 0.005,\n gamma: float = 0.99,\n n_step_return_horizon: int = 1,\n ) -> None:\n \"\"\"\n :param policy: the policy\n :param policy_optim: the optimizer factory for the policy's model.\n :param critic: the first critic network.\n For continuous action spaces: (s, a -> Q(s, a)).\n **NOTE**: The default implementation of `_target_q_compute_value` assumes\n a continuous action space; override this method if using discrete actions.\n :param critic_optim: the optimizer factory for the first critic network.\n :param critic2: the second critic network (analogous functionality to the first).\n If None, copy the first critic (via deepcopy).\n :param critic2_optim: the optimizer factory for the second critic network.\n If None, use the first critic's factory.\n :param tau: the soft update coefficient for target networks, controlling the rate at which\n target networks track the learned networks.\n When the parameters of the target network are updated with the current (source) network's\n parameters, a weighted average is used: target = tau * source + (1 - tau) * target.\n Smaller values (closer to 0) create more stable but slower learning as target networks\n change more gradually. Higher values (closer to 1) allow faster learning but may reduce\n stability.\n Typically set to a small value (0.001 to 0.01) for most reinforcement learning tasks.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n \"\"\"\n super().__init__(\n policy=policy,\n policy_optim=policy_optim,\n critic=critic,\n critic_optim=critic_optim,\n tau=tau,\n gamma=gamma,\n n_step_return_horizon=n_step_return_horizon,\n )\n self.critic2 = critic2 or deepcopy(critic)\n self.critic2_old = self._add_lagged_network(self.critic2)\n self.critic2_optim = self._create_optimizer(self.critic2, critic2_optim or critic_optim)\n\n def _target_q_compute_value(\n self, obs_batch: Batch, act_batch: TActBatchProtocol\n ) -> torch.Tensor:\n # compute the Q-value as the minimum of the two lagged critics\n act = act_batch.act\n return torch.min(\n self.critic_old(obs_batch.obs, act),\n self.critic2_old(obs_batch.obs, act),\n )\n\n\nclass TD3(\n ActorDualCriticsOffPolicyAlgorithm[ContinuousDeterministicPolicy, ActStateBatchProtocol],\n):\n \"\"\"Implementation of TD3, arXiv:1802.09477.\"\"\"\n\n def __init__(\n self,\n *,\n policy: ContinuousDeterministicPolicy,\n policy_optim: OptimizerFactory,\n critic: torch.nn.Module,\n critic_optim: OptimizerFactory,\n critic2: torch.nn.Module | None = None,\n critic2_optim: OptimizerFactory | None = None,\n tau: float = 0.005,\n gamma: float = 0.99,\n policy_noise: float = 0.2,\n update_actor_freq: int = 2,\n noise_clip: float = 0.5,\n n_step_return_horizon: int = 1,\n ) -> None:\n \"\"\"\n :param policy: the policy\n :param policy_optim: the optimizer factory for the policy's model.\n :param critic: the first critic network. (s, a -> Q(s, a))\n :param critic_optim: the optimizer factory for the first critic network.\n :param critic2: the second critic network. (s, a -> Q(s, a)).\n If None, copy the first critic (via deepcopy).\n :param critic2_optim: the optimizer factory for the second critic network.\n If None, use the first critic's factory.\n :param tau: the soft update coefficient for target networks, controlling the rate at which\n target networks track the learned networks.\n When the parameters of the target network are updated with the current (source) network's\n parameters, a weighted average is used: target = tau * source + (1 - tau) * target.\n Smaller values (closer to 0) create more stable but slower learning as target networks\n change more gradually. Higher values (closer to 1) allow faster learning but may reduce\n stability.\n Typically set to a small value (0.001 to 0.01) for most reinforcement learning tasks.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param policy_noise: scaling factor for the Gaussian noise added to target policy actions.\n This parameter implements target policy smoothing, a regularization technique described in the TD3 paper.\n The noise is sampled from a normal distribution and multiplied by this value before being added to actions.\n Higher values increase exploration in the target policy, helping to address function approximation error.\n The added noise is optionally clipped to a range determined by the noise_clip parameter.\n Typically set between 0.1 and 0.5 relative to the action scale of the environment.\n :param update_actor_freq: the frequency of actor network updates relative to critic network updates\n (the actor network is only updated once for every `update_actor_freq` critic updates).\n This implements the \"delayed\" policy updates from the TD3 algorithm, where the actor is\n updated less frequently than the critics.\n Higher values (e.g., 2-5) help stabilize training by allowing the critic to become more\n accurate before updating the policy.\n The default value of 2 follows the original TD3 paper's recommendation of updating the\n policy at half the rate of the Q-functions.\n :param noise_clip: defines the maximum absolute value of the noise added to target policy actions, i.e. noise values\n are clipped to the range [-noise_clip, noise_clip] (after generating and scaling the noise\n via `policy_noise`).\n This parameter implements bounded target policy smoothing as described in the TD3 paper.\n It prevents extreme noise values from causing unrealistic target values during training.\n Setting it 0.0 (or a negative value) disables clipping entirely.\n It is typically set to about twice the `policy_noise` value (e.g. 0.5 when `policy_noise` is 0.2).\n \"\"\"\n super().__init__(\n policy=policy,\n policy_optim=policy_optim,\n critic=critic,\n critic_optim=critic_optim,\n critic2=critic2,\n critic2_optim=critic2_optim,\n tau=tau,\n gamma=gamma,\n n_step_return_horizon=n_step_return_horizon,\n )\n self.actor_old = self._add_lagged_network(self.policy.actor)\n self.policy_noise = policy_noise\n self.update_actor_freq = update_actor_freq\n self.noise_clip = noise_clip\n self._cnt = 0\n self._last = 0\n\n def _target_q_compute_action(self, obs_batch: Batch) -> ActStateBatchProtocol:\n # compute action using lagged actor\n act_batch = self.policy(obs_batch, model=self.actor_old)\n act_ = act_batch.act\n\n # add noise\n noise = torch.randn(size=act_.shape, device=act_.device) * self.policy_noise\n if self.noise_clip > 0.0:\n noise = noise.clamp(-self.noise_clip, self.noise_clip)\n act_ += noise\n\n act_batch.act = act_\n return act_batch\n\n def _update_with_batch(self, batch: RolloutBatchProtocol) -> TD3TrainingStats:\n # critic 1&2\n td1, critic1_loss = self._minimize_critic_squared_loss(\n batch, self.critic, self.critic_optim\n )\n td2, critic2_loss = self._minimize_critic_squared_loss(\n batch, self.critic2, self.critic2_optim\n )\n batch.weight = (td1 + td2) / 2.0 # prio-buffer\n\n # actor\n if self._cnt % self.update_actor_freq == 0:\n actor_loss = -self.critic(batch.obs, self.policy(batch, eps=0.0).act).mean()\n self._last = actor_loss.item()\n self.policy_optim.step(actor_loss)\n self._update_lagged_network_weights()\n self._cnt += 1\n\n return TD3TrainingStats(\n actor_loss=self._last,\n critic1_loss=critic1_loss.item(),\n critic2_loss=critic2_loss.item(),\n )\n" }, { "path": "tianshou/algorithm/modelfree/trpo.py", "content": "import warnings\nfrom dataclasses import dataclass\n\nimport torch\nimport torch.nn.functional as F\nfrom torch.distributions import kl_divergence\n\nfrom tianshou.algorithm import NPG\nfrom tianshou.algorithm.modelfree.npg import NPGTrainingStats\nfrom tianshou.algorithm.modelfree.reinforce import ProbabilisticActorPolicy\nfrom tianshou.algorithm.optim import OptimizerFactory\nfrom tianshou.data import SequenceSummaryStats\nfrom tianshou.data.types import BatchWithAdvantagesProtocol\nfrom tianshou.utils.net.continuous import ContinuousCritic\nfrom tianshou.utils.net.discrete import DiscreteCritic\n\n\n@dataclass(kw_only=True)\nclass TRPOTrainingStats(NPGTrainingStats):\n step_size: SequenceSummaryStats\n\n\nclass TRPO(NPG):\n \"\"\"Implementation of Trust Region Policy Optimization. arXiv:1502.05477.\"\"\"\n\n def __init__(\n self,\n *,\n policy: ProbabilisticActorPolicy,\n critic: torch.nn.Module | ContinuousCritic | DiscreteCritic,\n optim: OptimizerFactory,\n max_kl: float = 0.01,\n backtrack_coeff: float = 0.8,\n max_backtracks: int = 10,\n optim_critic_iters: int = 5,\n trust_region_size: float = 0.5,\n advantage_normalization: bool = True,\n gae_lambda: float = 0.95,\n max_batchsize: int = 256,\n gamma: float = 0.99,\n return_scaling: bool = False,\n ) -> None:\n \"\"\"\n :param policy: the policy\n :param critic: the critic network. (s -> V(s))\n :param optim: the optimizer factory for the critic network.\n :param max_kl: max kl-divergence used to constrain each actor network update.\n :param backtrack_coeff: Coefficient to be multiplied by step size when\n constraints are not met.\n :param max_backtracks: Max number of backtracking times in linesearch.\n :param optim_critic_iters: the number of optimization steps performed on the critic network\n for each policy (actor) update.\n Controls the learning rate balance between critic and actor.\n Higher values prioritize critic accuracy by training the value function more\n extensively before each policy update, which can improve stability but slow down\n training. Lower values maintain a more even learning pace between policy and value\n function but may lead to less reliable advantage estimates.\n Typically set between 1 and 10, depending on the complexity of the value function.\n :param trust_region_size: the parameter delta - a scalar multiplier for policy updates in the natural gradient direction.\n The mathematical meaning is the trust region size, which is the maximum KL divergence\n allowed between the old and new policy distributions.\n Controls how far the policy parameters move in the calculated direction\n during each update. Higher values allow for faster learning but may cause instability\n or policy deterioration; lower values provide more stable but slower learning. Unlike\n regular policy gradients, natural gradients already account for the local geometry of\n the parameter space, making this step size more robust to different parameterizations.\n Typically set between 0.1 and 1.0 for most reinforcement learning tasks.\n :param advantage_normalization: whether to do per mini-batch advantage\n normalization.\n :param gae_lambda: the lambda parameter in [0, 1] for generalized advantage estimation (GAE).\n Controls the bias-variance tradeoff in advantage estimates, acting as a\n weighting factor for combining different n-step advantage estimators. Higher values\n (closer to 1) reduce bias but increase variance by giving more weight to longer\n trajectories, while lower values (closer to 0) reduce variance but increase bias\n by relying more on the immediate TD error and value function estimates. At λ=0,\n GAE becomes equivalent to the one-step TD error (high bias, low variance); at λ=1,\n it becomes equivalent to Monte Carlo advantage estimation (low bias, high variance).\n Intermediate values create a weighted average of n-step returns, with exponentially\n decaying weights for longer-horizon returns. Typically set between 0.9 and 0.99 for\n most policy gradient methods.\n :param max_batchsize: the maximum number of samples to process at once when computing\n generalized advantage estimation (GAE) and value function predictions.\n Controls memory usage by breaking large batches into smaller chunks processed sequentially.\n Higher values may increase speed but require more GPU/CPU memory; lower values\n reduce memory requirements but may increase computation time. Should be adjusted\n based on available hardware resources and total batch size of your training data.\n :param gamma: the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n :param return_scaling: flag indicating whether to enable scaling of estimated returns by\n dividing them by their running standard deviation without centering the mean.\n This reduces the magnitude variation of advantages across different episodes while\n preserving their signs and relative ordering.\n The use of running statistics (rather than batch-specific scaling) means that early\n training experiences may be scaled differently than later ones as the statistics evolve.\n When enabled, this improves training stability in environments with highly variable\n reward scales and makes the algorithm less sensitive to learning rate settings.\n However, it may reduce the algorithm's ability to distinguish between episodes with\n different absolute return magnitudes.\n Best used in environments where the relative ordering of actions is more important\n than the absolute scale of returns.\n \"\"\"\n super().__init__(\n policy=policy,\n critic=critic,\n optim=optim,\n optim_critic_iters=optim_critic_iters,\n trust_region_size=trust_region_size,\n advantage_normalization=advantage_normalization,\n gae_lambda=gae_lambda,\n max_batchsize=max_batchsize,\n gamma=gamma,\n return_scaling=return_scaling,\n )\n self.max_backtracks = max_backtracks\n self.max_kl = max_kl\n self.backtrack_coeff = backtrack_coeff\n\n def _update_with_batch( # type: ignore[override]\n self,\n batch: BatchWithAdvantagesProtocol,\n batch_size: int | None,\n repeat: int,\n ) -> TRPOTrainingStats:\n actor_losses, vf_losses, step_sizes, kls = [], [], [], []\n split_batch_size = batch_size or -1\n for _ in range(repeat):\n for minibatch in batch.split(split_batch_size, merge_last=True):\n # optimize actor\n # direction: calculate villia gradient\n dist = self.policy(minibatch).dist\n ratio = (dist.log_prob(minibatch.act) - minibatch.logp_old).exp().float()\n ratio = ratio.reshape(ratio.size(0), -1).transpose(0, 1)\n actor_loss = -(ratio * minibatch.adv).mean()\n flat_grads = self._get_flat_grad(\n actor_loss, self.policy.actor, retain_graph=True\n ).detach()\n\n # direction: calculate natural gradient\n with torch.no_grad():\n old_dist = self.policy(minibatch).dist\n\n kl = kl_divergence(old_dist, dist).mean()\n # calculate first order gradient of kl with respect to theta\n flat_kl_grad = self._get_flat_grad(kl, self.policy.actor, create_graph=True)\n search_direction = -self._conjugate_gradients(flat_grads, flat_kl_grad, nsteps=10)\n\n # stepsize: calculate max stepsize constrained by kl bound\n step_size = torch.sqrt(\n 2\n * self.max_kl\n / (search_direction * self._MVP(search_direction, flat_kl_grad)).sum(\n 0,\n keepdim=True,\n ),\n )\n\n # stepsize: linesearch stepsize\n with torch.no_grad():\n flat_params = torch.cat(\n [param.data.view(-1) for param in self.policy.actor.parameters()],\n )\n for i in range(self.max_backtracks):\n new_flat_params = flat_params + step_size * search_direction\n self._set_from_flat_params(self.policy.actor, new_flat_params)\n # calculate kl and if in bound, loss actually down\n new_dist = self.policy(minibatch).dist\n new_dratio = (\n (new_dist.log_prob(minibatch.act) - minibatch.logp_old).exp().float()\n )\n new_dratio = new_dratio.reshape(new_dratio.size(0), -1).transpose(0, 1)\n new_actor_loss = -(new_dratio * minibatch.adv).mean()\n kl = kl_divergence(old_dist, new_dist).mean()\n\n if kl < self.max_kl and new_actor_loss < actor_loss:\n if i > 0:\n warnings.warn(f\"Backtracking to step {i}.\")\n break\n if i < self.max_backtracks - 1:\n step_size = step_size * self.backtrack_coeff\n else:\n self._set_from_flat_params(self.policy.actor, new_flat_params)\n step_size = torch.tensor([0.0])\n warnings.warn(\n \"Line search failed! It seems hyperparamters\"\n \" are poor and need to be changed.\",\n )\n\n # optimize critic\n for _ in range(self.optim_critic_iters):\n value = self.critic(minibatch.obs).flatten()\n vf_loss = F.mse_loss(minibatch.returns, value)\n self.optim.step(vf_loss)\n\n actor_losses.append(actor_loss.item())\n vf_losses.append(vf_loss.item())\n step_sizes.append(step_size.item())\n kls.append(kl.item())\n\n actor_loss_summary_stat = SequenceSummaryStats.from_sequence(actor_losses)\n vf_loss_summary_stat = SequenceSummaryStats.from_sequence(vf_losses)\n kl_summary_stat = SequenceSummaryStats.from_sequence(kls)\n step_size_stat = SequenceSummaryStats.from_sequence(step_sizes)\n\n return TRPOTrainingStats(\n actor_loss=actor_loss_summary_stat,\n vf_loss=vf_loss_summary_stat,\n kl=kl_summary_stat,\n step_size=step_size_stat,\n )\n" }, { "path": "tianshou/algorithm/multiagent/__init__.py", "content": "" }, { "path": "tianshou/algorithm/multiagent/marl.py", "content": "from collections.abc import Callable\nfrom typing import Any, Generic, Literal, Protocol, Self, TypeVar, cast, overload\n\nimport numpy as np\nfrom overrides import override\nfrom sensai.util.helper import mark_used\nfrom torch.nn import ModuleList\n\nfrom tianshou.algorithm import Algorithm\nfrom tianshou.algorithm.algorithm_base import (\n OffPolicyAlgorithm,\n OnPolicyAlgorithm,\n Policy,\n TrainingStats,\n)\nfrom tianshou.data import Batch, ReplayBuffer\nfrom tianshou.data.batch import BatchProtocol, IndexType\nfrom tianshou.data.types import ActBatchProtocol, ObsBatchProtocol, RolloutBatchProtocol\n\ntry:\n from tianshou.env.pettingzoo_env import PettingZooEnv\nexcept ImportError:\n PettingZooEnv = None # type: ignore\n\n\nmark_used(ActBatchProtocol)\n\n\nclass MapTrainingStats(TrainingStats):\n def __init__(\n self,\n agent_id_to_stats: dict[str | int, TrainingStats],\n train_time_aggregator: Literal[\"min\", \"max\", \"mean\"] = \"max\",\n ) -> None:\n self._agent_id_to_stats = agent_id_to_stats\n train_times = [agent_stats.train_time for agent_stats in agent_id_to_stats.values()]\n match train_time_aggregator:\n case \"max\":\n aggr_function = max\n case \"min\":\n aggr_function = min\n case \"mean\":\n aggr_function = np.mean # type: ignore\n case _:\n raise ValueError(\n f\"Unknown {train_time_aggregator=}\",\n )\n self.train_time = aggr_function(train_times)\n self.smoothed_loss = {}\n\n @override\n def get_loss_stats_dict(self) -> dict[str, float]:\n \"\"\"Collects loss_stats_dicts from all agents, prepends agent_id to all keys, and joins results.\"\"\"\n result_dict = {}\n for agent_id, stats in self._agent_id_to_stats.items():\n agent_loss_stats_dict = stats.get_loss_stats_dict()\n for k, v in agent_loss_stats_dict.items():\n result_dict[f\"{agent_id}/\" + k] = v\n return result_dict\n\n\nclass MAPRolloutBatchProtocol(RolloutBatchProtocol, Protocol):\n # TODO: this might not be entirely correct.\n # The whole MAP data processing pipeline needs more documentation and possibly some refactoring\n @overload\n def __getitem__(self, index: str) -> RolloutBatchProtocol: ...\n\n @overload\n def __getitem__(self, index: IndexType) -> Self: ...\n\n def __getitem__(self, index: str | IndexType) -> Any: ...\n\n\nclass MultiAgentPolicy(Policy):\n def __init__(self, policies: dict[str | int, Policy]):\n p0 = next(iter(policies.values()))\n super().__init__(\n action_space=p0.action_space,\n observation_space=p0.observation_space,\n action_scaling=False,\n action_bound_method=None,\n )\n self.policies = policies\n self._submodules = ModuleList(policies.values())\n\n _TArrOrActBatch = TypeVar(\"_TArrOrActBatch\", bound=\"np.ndarray | ActBatchProtocol\")\n\n def add_exploration_noise(\n self,\n act: _TArrOrActBatch,\n batch: ObsBatchProtocol,\n ) -> _TArrOrActBatch:\n \"\"\"Add exploration noise from sub-policy onto act.\"\"\"\n if not isinstance(batch.obs, Batch):\n raise TypeError(\n f\"here only observations of type Batch are permitted, but got {type(batch.obs)}\",\n )\n for agent_id, policy in self.policies.items():\n agent_index = np.nonzero(batch.obs.agent_id == agent_id)[0]\n if len(agent_index) == 0:\n continue\n act[agent_index] = policy.add_exploration_noise(act[agent_index], batch[agent_index])\n return act\n\n def forward( # type: ignore\n self,\n batch: Batch,\n state: dict | Batch | None = None,\n **kwargs: Any,\n ) -> Batch:\n \"\"\"Dispatch batch data from obs.agent_id to every policy's forward.\n\n :param batch: TODO: document what is expected at input and make a BatchProtocol for it\n :param state: if None, it means all agents have no state. If not\n None, it should contain keys of \"agent_1\", \"agent_2\", ...\n\n :return: a Batch with the following contents:\n TODO: establish a BatcProtocol for this\n\n ::\n\n {\n \"act\": actions corresponding to the input\n \"state\": {\n \"agent_1\": output state of agent_1's policy for the state\n \"agent_2\": xxx\n ...\n \"agent_n\": xxx}\n \"out\": {\n \"agent_1\": output of agent_1's policy for the input\n \"agent_2\": xxx\n ...\n \"agent_n\": xxx}\n }\n \"\"\"\n results: list[tuple[bool, np.ndarray, Batch, np.ndarray | Batch, Batch]] = []\n for agent_id, policy in self.policies.items():\n # This part of code is difficult to understand.\n # Let's follow an example with two agents\n # batch.obs.agent_id is [1, 2, 1, 2, 1, 2] (with batch_size == 6)\n # each agent plays for three transitions\n # agent_index for agent 1 is [0, 2, 4]\n # agent_index for agent 2 is [1, 3, 5]\n # we separate the transition of each agent according to agent_id\n agent_index = np.nonzero(batch.obs.agent_id == agent_id)[0]\n if len(agent_index) == 0:\n # (has_data, agent_index, out, act, state)\n results.append((False, np.array([-1]), Batch(), Batch(), Batch()))\n continue\n tmp_batch = batch[agent_index]\n if \"rew\" in tmp_batch.get_keys() and isinstance(tmp_batch.rew, np.ndarray):\n # reward can be empty Batch (after initial reset) or nparray.\n tmp_batch.rew = tmp_batch.rew[:, self.agent_idx[agent_id]]\n if not hasattr(tmp_batch.obs, \"mask\"):\n if hasattr(tmp_batch.obs, \"obs\"):\n tmp_batch.obs = tmp_batch.obs.obs\n if hasattr(tmp_batch.obs_next, \"obs\"):\n tmp_batch.obs_next = tmp_batch.obs_next.obs\n out = policy(\n batch=tmp_batch,\n state=None if state is None else state[agent_id],\n **kwargs,\n )\n act = out.act\n each_state = out.state if (hasattr(out, \"state\") and out.state is not None) else Batch()\n results.append((True, agent_index, out, act, each_state))\n holder: Batch = Batch.cat(\n [{\"act\": act} for (has_data, agent_index, out, act, each_state) in results if has_data],\n )\n state_dict, out_dict = {}, {}\n for (agent_id, _), (has_data, agent_index, out, act, state) in zip(\n self.policies.items(),\n results,\n strict=True,\n ):\n if has_data:\n holder.act[agent_index] = act\n state_dict[agent_id] = state\n out_dict[agent_id] = out\n holder[\"out\"] = out_dict\n holder[\"state\"] = state_dict\n return holder\n\n\nTAlgorithm = TypeVar(\"TAlgorithm\", bound=Algorithm)\n\n\nclass MARLDispatcher(Generic[TAlgorithm]):\n \"\"\"\n Supports multi-agent learning by dispatching calls to the corresponding\n algorithm for each agent.\n \"\"\"\n\n def __init__(self, algorithms: list[TAlgorithm], env: PettingZooEnv):\n agent_ids = env.agents\n assert len(algorithms) == len(agent_ids), \"One policy must be assigned for each agent.\"\n self.algorithms: dict[str | int, TAlgorithm] = dict(zip(agent_ids, algorithms, strict=True))\n \"\"\"maps agent_id to the corresponding algorithm.\"\"\"\n self.agent_idx = env.agent_idx\n \"\"\"maps agent_id to 0-based index.\"\"\"\n\n def create_policy(self) -> MultiAgentPolicy:\n return MultiAgentPolicy({agent_id: a.policy for agent_id, a in self.algorithms.items()})\n\n def dispatch_process_fn(\n self,\n batch: MAPRolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n ) -> MAPRolloutBatchProtocol:\n \"\"\"Dispatch batch data from `obs.agent_id` to every algorithm's processing function.\n\n Save original multi-dimensional rew in \"save_rew\", set rew to the\n reward of each agent during their \"process_fn\", and restore the\n original reward afterwards.\n \"\"\"\n # TODO: maybe only str is actually allowed as agent_id? See MAPRolloutBatchProtocol\n results: dict[str | int, RolloutBatchProtocol] = {}\n assert isinstance(\n batch.obs,\n BatchProtocol,\n ), f\"here only observations of type Batch are permitted, but got {type(batch.obs)}\"\n # reward can be empty Batch (after initial reset) or nparray.\n has_rew = isinstance(buffer.rew, np.ndarray)\n if has_rew: # save the original reward in save_rew\n # Since we do not override buffer.__setattr__, here we use _meta to\n # change buffer.rew, otherwise buffer.rew = Batch() has no effect.\n save_rew, buffer._meta.rew = buffer.rew, Batch() # type: ignore\n for agent, algorithm in self.algorithms.items():\n agent_index = np.nonzero(batch.obs.agent_id == agent)[0]\n if len(agent_index) == 0:\n results[agent] = cast(RolloutBatchProtocol, Batch())\n continue\n tmp_batch, tmp_indice = batch[agent_index], indices[agent_index]\n if has_rew:\n tmp_batch.rew = tmp_batch.rew[:, self.agent_idx[agent]]\n buffer._meta.rew = save_rew[:, self.agent_idx[agent]]\n if not hasattr(tmp_batch.obs, \"mask\"):\n if hasattr(tmp_batch.obs, \"obs\"):\n tmp_batch.obs = tmp_batch.obs.obs\n if hasattr(tmp_batch.obs_next, \"obs\"):\n tmp_batch.obs_next = tmp_batch.obs_next.obs\n results[agent] = algorithm._preprocess_batch(tmp_batch, buffer, tmp_indice)\n if has_rew: # restore from save_rew\n buffer._meta.rew = save_rew\n return cast(MAPRolloutBatchProtocol, Batch(results))\n\n def dispatch_update_with_batch(\n self,\n batch: MAPRolloutBatchProtocol,\n algorithm_update_with_batch_fn: Callable[[TAlgorithm, RolloutBatchProtocol], TrainingStats],\n ) -> MapTrainingStats:\n \"\"\"Dispatch the respective subset of the batch data to each algorithm.\n\n :param batch: must map agent_ids to rollout batches\n :param algorithm_update_with_batch_fn: a function that performs the algorithm-specific\n update with the given agent-specific batch data\n \"\"\"\n agent_id_to_stats = {}\n for agent_id, algorithm in self.algorithms.items():\n data = batch[agent_id]\n if len(data.get_keys()) != 0:\n train_stats = algorithm_update_with_batch_fn(algorithm, data)\n agent_id_to_stats[agent_id] = train_stats\n return MapTrainingStats(agent_id_to_stats)\n\n\nclass MultiAgentOffPolicyAlgorithm(OffPolicyAlgorithm[MultiAgentPolicy]):\n \"\"\"Multi-agent reinforcement learning where each agent uses off-policy learning.\"\"\"\n\n def __init__(\n self,\n *,\n algorithms: list[OffPolicyAlgorithm],\n env: PettingZooEnv,\n ) -> None:\n \"\"\"\n :param algorithms: a list of off-policy algorithms.\n :param env: the multi-agent RL environment\n \"\"\"\n self._dispatcher: MARLDispatcher[OffPolicyAlgorithm] = MARLDispatcher(algorithms, env)\n super().__init__(\n policy=self._dispatcher.create_policy(),\n )\n self._submodules = ModuleList(algorithms)\n\n def get_algorithm(self, agent_id: str | int) -> OffPolicyAlgorithm:\n return self._dispatcher.algorithms[agent_id]\n\n def _preprocess_batch(\n self,\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n ) -> RolloutBatchProtocol:\n batch = cast(MAPRolloutBatchProtocol, batch)\n return self._dispatcher.dispatch_process_fn(batch, buffer, indices)\n\n def _update_with_batch(\n self,\n batch: RolloutBatchProtocol,\n ) -> MapTrainingStats:\n batch = cast(MAPRolloutBatchProtocol, batch)\n\n def update(algorithm: OffPolicyAlgorithm, data: RolloutBatchProtocol) -> TrainingStats:\n return algorithm._update_with_batch(data)\n\n return self._dispatcher.dispatch_update_with_batch(batch, update)\n\n\nclass MultiAgentOnPolicyAlgorithm(OnPolicyAlgorithm[MultiAgentPolicy]):\n \"\"\"Multi-agent reinforcement learning where each agent uses on-policy learning.\"\"\"\n\n def __init__(\n self,\n *,\n algorithms: list[OnPolicyAlgorithm],\n env: PettingZooEnv,\n ) -> None:\n \"\"\"\n :param algorithms: a list of off-policy algorithms.\n :param env: the multi-agent RL environment\n \"\"\"\n self._dispatcher: MARLDispatcher[OnPolicyAlgorithm] = MARLDispatcher(algorithms, env)\n super().__init__(\n policy=self._dispatcher.create_policy(),\n )\n self._submodules = ModuleList(algorithms)\n\n def get_algorithm(self, agent_id: str | int) -> OnPolicyAlgorithm:\n return self._dispatcher.algorithms[agent_id]\n\n def _preprocess_batch(\n self,\n batch: RolloutBatchProtocol,\n buffer: ReplayBuffer,\n indices: np.ndarray,\n ) -> RolloutBatchProtocol:\n batch = cast(MAPRolloutBatchProtocol, batch)\n return self._dispatcher.dispatch_process_fn(batch, buffer, indices)\n\n def _update_with_batch(\n self, batch: RolloutBatchProtocol, batch_size: int | None, repeat: int\n ) -> MapTrainingStats:\n batch = cast(MAPRolloutBatchProtocol, batch)\n\n def update(algorithm: OnPolicyAlgorithm, data: RolloutBatchProtocol) -> TrainingStats:\n return algorithm._update_with_batch(data, batch_size, repeat)\n\n return self._dispatcher.dispatch_update_with_batch(batch, update)\n" }, { "path": "tianshou/algorithm/optim.py", "content": "from abc import ABC, abstractmethod\nfrom collections.abc import Callable, Iterable\nfrom typing import Any, Self, TypeAlias\n\nimport numpy as np\nimport torch\nfrom sensai.util.string import ToStringMixin\nfrom torch.optim import Adam, RMSprop\nfrom torch.optim.lr_scheduler import LambdaLR, LRScheduler\n\nParamsType: TypeAlias = Iterable[torch.Tensor] | Iterable[dict[str, Any]]\n\n\nclass LRSchedulerFactory(ToStringMixin, ABC):\n \"\"\"Factory for the creation of a learning rate scheduler.\"\"\"\n\n @abstractmethod\n def create_scheduler(self, optim: torch.optim.Optimizer) -> LRScheduler:\n pass\n\n\nclass LRSchedulerFactoryLinear(LRSchedulerFactory):\n \"\"\"\n Factory for a learning rate scheduler where the learning rate linearly decays towards\n zero for the given trainer parameters.\n \"\"\"\n\n def __init__(self, max_epochs: int, epoch_num_steps: int, collection_step_num_env_steps: int):\n self.num_epochs = max_epochs\n self.epoch_num_steps = epoch_num_steps\n self.collection_step_num_env_steps = collection_step_num_env_steps\n\n def create_scheduler(self, optim: torch.optim.Optimizer) -> LRScheduler:\n return LambdaLR(optim, lr_lambda=self._LRLambda(self).compute)\n\n class _LRLambda:\n def __init__(self, parent: \"LRSchedulerFactoryLinear\"):\n self.max_update_num = (\n np.ceil(parent.epoch_num_steps / parent.collection_step_num_env_steps)\n * parent.num_epochs\n )\n\n def compute(self, epoch: int) -> float:\n return 1.0 - epoch / self.max_update_num\n\n\nclass OptimizerFactory(ABC, ToStringMixin):\n def __init__(self) -> None:\n self.lr_scheduler_factory: LRSchedulerFactory | None = None\n\n def with_lr_scheduler_factory(self, lr_scheduler_factory: LRSchedulerFactory) -> Self:\n self.lr_scheduler_factory = lr_scheduler_factory\n return self\n\n def create_instances(\n self,\n module: torch.nn.Module,\n ) -> tuple[torch.optim.Optimizer, LRScheduler | None]:\n optimizer = self._create_optimizer_for_params(module.parameters())\n lr_scheduler = None\n if self.lr_scheduler_factory is not None:\n lr_scheduler = self.lr_scheduler_factory.create_scheduler(optimizer)\n return optimizer, lr_scheduler\n\n @abstractmethod\n def _create_optimizer_for_params(self, params: ParamsType) -> torch.optim.Optimizer:\n pass\n\n\nclass TorchOptimizerFactory(OptimizerFactory):\n \"\"\"General factory for arbitrary torch optimizers.\"\"\"\n\n def __init__(self, optim_class: Callable[..., torch.optim.Optimizer], **kwargs: Any):\n \"\"\"\n\n :param optim_class: the optimizer class (e.g. subclass of `torch.optim.Optimizer`),\n which will be passed the module parameters, the learning rate as `lr` and the\n kwargs provided.\n :param kwargs: keyword arguments to provide at optimizer construction\n \"\"\"\n super().__init__()\n self.optim_class = optim_class\n self.kwargs = kwargs\n\n def _create_optimizer_for_params(self, params: ParamsType) -> torch.optim.Optimizer:\n return self.optim_class(params, **self.kwargs)\n\n\nclass AdamOptimizerFactory(OptimizerFactory):\n def __init__(\n self,\n lr: float = 1e-3,\n betas: tuple[float, float] = (0.9, 0.999),\n eps: float = 1e-08,\n weight_decay: float = 0,\n ):\n super().__init__()\n self.lr = lr\n self.weight_decay = weight_decay\n self.eps = eps\n self.betas = betas\n\n def _create_optimizer_for_params(self, params: ParamsType) -> torch.optim.Optimizer:\n return Adam(\n params,\n lr=self.lr,\n betas=self.betas,\n eps=self.eps,\n weight_decay=self.weight_decay,\n )\n\n\nclass RMSpropOptimizerFactory(OptimizerFactory):\n def __init__(\n self,\n lr: float = 1e-2,\n alpha: float = 0.99,\n eps: float = 1e-08,\n weight_decay: float = 0,\n momentum: float = 0,\n centered: bool = False,\n ):\n super().__init__()\n self.lr = lr\n self.alpha = alpha\n self.momentum = momentum\n self.centered = centered\n self.weight_decay = weight_decay\n self.eps = eps\n\n def _create_optimizer_for_params(self, params: ParamsType) -> torch.optim.Optimizer:\n return RMSprop(\n params,\n lr=self.lr,\n alpha=self.alpha,\n eps=self.eps,\n weight_decay=self.weight_decay,\n momentum=self.momentum,\n centered=self.centered,\n )\n" }, { "path": "tianshou/algorithm/random.py", "content": "from typing import cast\n\nimport gymnasium as gym\nimport numpy as np\n\nfrom tianshou.algorithm.algorithm_base import OffPolicyAlgorithm, TrainingStats\nfrom tianshou.algorithm.algorithm_base import Policy as BasePolicy\nfrom tianshou.data import Batch\nfrom tianshou.data.batch import BatchProtocol\nfrom tianshou.data.types import ActBatchProtocol, ObsBatchProtocol, RolloutBatchProtocol\n\n\nclass MARLRandomTrainingStats(TrainingStats):\n pass\n\n\nclass MARLRandomDiscreteMaskedOffPolicyAlgorithm(OffPolicyAlgorithm):\n \"\"\"A random agent used in multi-agent learning.\n\n It randomly chooses an action from the legal actions (according to the given mask).\n \"\"\"\n\n class Policy(BasePolicy):\n \"\"\"A random agent used in multi-agent learning.\n\n It randomly chooses an action from the legal actions.\n \"\"\"\n\n def __init__(self, action_space: gym.spaces.Space) -> None:\n super().__init__(action_space=action_space)\n\n def forward(\n self,\n batch: ObsBatchProtocol,\n state: dict | BatchProtocol | np.ndarray | None = None,\n **kwargs: dict,\n ) -> ActBatchProtocol:\n \"\"\"Compute the random action over the given batch data.\n\n The input should contain a mask in batch.obs, with \"True\" to be\n available and \"False\" to be unavailable. For example,\n ``batch.obs.mask == np.array([[False, True, False]])`` means with batch\n size 1, action \"1\" is available but action \"0\" and \"2\" are unavailable.\n\n :return: A :class:`~tianshou.data.Batch` with \"act\" key, containing\n the random action.\n \"\"\"\n mask = batch.obs.mask # type: ignore\n logits = np.random.rand(*mask.shape)\n logits[~mask] = -np.inf\n result = Batch(act=logits.argmax(axis=-1))\n return cast(ActBatchProtocol, result)\n\n def __init__(self, action_space: gym.spaces.Space) -> None:\n \"\"\":param action_space: the environment's action space.\"\"\"\n super().__init__(policy=self.Policy(action_space))\n\n def _update_with_batch(self, batch: RolloutBatchProtocol) -> MARLRandomTrainingStats: # type: ignore\n \"\"\"Since a random agent learns nothing, it returns an empty dict.\"\"\"\n return MARLRandomTrainingStats()\n" }, { "path": "tianshou/config.py", "content": "ENABLE_VALIDATION = False\n\"\"\"Validation can help catching bugs and issues but it slows down training and collection. Enable it only if needed.\"\"\"\n" }, { "path": "tianshou/data/__init__.py", "content": "\"\"\"Data package.\"\"\"\n# isort:skip_file\n\nfrom tianshou.data.batch import Batch\nfrom tianshou.data.utils.converter import to_numpy, to_torch, to_torch_as\nfrom tianshou.data.utils.segtree import SegmentTree\nfrom tianshou.data.buffer.buffer_base import ReplayBuffer\nfrom tianshou.data.buffer.prio import PrioritizedReplayBuffer\nfrom tianshou.data.buffer.her import HERReplayBuffer\nfrom tianshou.data.buffer.manager import (\n ReplayBufferManager,\n PrioritizedReplayBufferManager,\n HERReplayBufferManager,\n)\nfrom tianshou.data.buffer.vecbuf import (\n HERVectorReplayBuffer,\n PrioritizedVectorReplayBuffer,\n VectorReplayBuffer,\n)\nfrom tianshou.data.buffer.cached import CachedReplayBuffer\nfrom tianshou.data.stats import (\n EpochStats,\n InfoStats,\n SequenceSummaryStats,\n TimingStats,\n)\nfrom tianshou.data.collector import (\n Collector,\n AsyncCollector,\n CollectStats,\n CollectStatsBase,\n BaseCollector,\n)\n\n__all__ = [\n \"AsyncCollector\",\n \"BaseCollector\",\n \"Batch\",\n \"CachedReplayBuffer\",\n \"CollectStats\",\n \"CollectStatsBase\",\n \"Collector\",\n \"EpochStats\",\n \"HERReplayBuffer\",\n \"HERReplayBufferManager\",\n \"HERVectorReplayBuffer\",\n \"InfoStats\",\n \"PrioritizedReplayBuffer\",\n \"PrioritizedReplayBufferManager\",\n \"PrioritizedVectorReplayBuffer\",\n \"ReplayBuffer\",\n \"ReplayBufferManager\",\n \"SegmentTree\",\n \"SequenceSummaryStats\",\n \"TimingStats\",\n \"VectorReplayBuffer\",\n \"to_numpy\",\n \"to_torch\",\n \"to_torch_as\",\n]\n" }, { "path": "tianshou/data/batch.py", "content": "\"\"\"This module implements :class:`Batch`, a flexible data structure for\nhandling heterogeneous data in reinforcement learning algorithms. Such a data structure\nis needed since RL algorithms differ widely in the conceptual fields that they need.\n`Batch` is the main data carrier in Tianshou. It bears some similarities to\n`TensorDict `_\nthat is used for a similar purpose in `pytorch-rl `_.\nThe main differences between the two are that `Batch` can hold arbitrary objects (and not just torch tensors),\nand that Tianshou implements `BatchProtocol` for enabling type checking and autocompletion (more on that below).\n\nThe `Batch` class is designed to store and manipulate collections of data with\nvarying types and structures. It strikes a balance between flexibility and type safety, the latter mainly\nachieved through the use of protocols. One can thing of it as a mixture of a dictionary and an array,\nas it has both key-value pairs and nesting, while also having a shape, being indexable and sliceable.\n\nKey features of the `Batch` class include:\n\n1. Flexible data storage: Can hold numpy arrays, torch tensors, scalars, and nested Batch objects.\n2. Dynamic attribute access: Allows setting and accessing data using attribute notation (e.g., `batch.observation`).\n This allows for type-safe and readable code and enables IDE autocompletion. See comments on `BatchProtocol` below.\n3. Indexing and slicing: Supports numpy-like indexing and slicing operations. The slicing is extended to nested\n Batch objects and torch Distributions.\n4. Batch operations: Provides methods for splitting, shuffling, concatenating and stacking multiple Batch objects.\n5. Data type conversion: Offers methods to convert data between numpy arrays and torch tensors.\n6. Value transformations: Allows applying functions to all values in the Batch recursively.\n7. Analysis utilities: Provides methods for checking for missing values, dropping entries with missing values,\n and others.\n\nSince we want to keep `Batch` flexible and not fix a specific set of fields or their types,\nwe don't have fixed interfaces for actual `Batch` objects that are used throughout\ntianshou (such interfaces could be dataclasses, for example). However, we still want to enable\nIDE autocompletion and type checking for `Batch` objects. To achieve this, we rely on dynamic duck typing\nby using `Protocol`. The :class:`BatchProtocol` defines the interface that all `Batch` objects should adhere to,\nand its various implementations (like :class:`~.types.ActBatchProtocol` or :class:`~.types.RolloutBatchProtocol`) define the specific\nfields that are expected in the respective `Batch` objects. The protocols are then used as type hints\nthroughout the codebase. Protocols can't be instantiated, but we can cast to them.\nFor example, we \"instantiate\" an `ActBatchProtocol` with something like:\n\n>>> act_batch = cast(ActBatchProtocol, Batch(act=my_action))\n\nThe users can decide for themselves how to structure their `Batch` objects, and can opt in to the\n`BatchProtocol` style to enable type checking and autocompletion. Opting out will have no effect on\nthe functionality.\n\"\"\"\n\nimport pprint\nimport warnings\nfrom collections.abc import Callable, Collection, Iterable, Iterator, KeysView, Sequence\nfrom copy import deepcopy\nfrom numbers import Number\nfrom types import EllipsisType\nfrom typing import (\n Any,\n Literal,\n Protocol,\n Self,\n TypeVar,\n Union,\n cast,\n overload,\n runtime_checkable,\n)\n\nimport numpy as np\nimport pandas as pd\nimport torch\nfrom deepdiff import DeepDiff\nfrom sensai.util import logging\nfrom torch.distributions import Categorical, Distribution, Independent, Normal\n\n_SingleIndexType = slice | int | EllipsisType\nIndexType = np.ndarray | _SingleIndexType | Sequence[_SingleIndexType]\nTBatch = TypeVar(\"TBatch\", bound=\"BatchProtocol\")\nTDistribution = TypeVar(\"TDistribution\", bound=Distribution)\nT = TypeVar(\"T\")\nTArr = torch.Tensor | np.ndarray\nTObsArr = torch.Tensor | np.ndarray\n\nlog = logging.getLogger(__name__)\n\n\ndef _is_batch_set(obj: Any) -> bool:\n # Batch set is a list/tuple of dict/Batch objects,\n # or 1-D np.ndarray with object type,\n # where each element is a dict/Batch object\n if isinstance(obj, np.ndarray): # most often case\n # \"for element in obj\" will just unpack the first dimension,\n # but obj.tolist() will flatten ndarray of objects\n # so do not use obj.tolist()\n if obj.shape == ():\n return False\n return obj.dtype == object and all(isinstance(element, dict | Batch) for element in obj)\n return (\n isinstance(obj, list | tuple)\n and len(obj) > 0\n and all(isinstance(element, dict | Batch) for element in obj)\n )\n\n\ndef _is_scalar(value: Any) -> bool:\n # check if the value is a scalar\n # 1. python bool object, number object: isinstance(value, Number)\n # 2. numpy scalar: isinstance(value, np.generic)\n # 3. python object rather than dict / Batch / tensor\n # the check of dict / Batch is omitted because this only checks a value.\n # a dict / Batch will eventually check their values\n if isinstance(value, torch.Tensor):\n return value.numel() == 1 and not value.shape\n # np.asanyarray will cause dead loop in some cases\n return np.isscalar(value)\n\n\ndef _is_number(value: Any) -> bool:\n # isinstance(value, Number) checks 1, 1.0, np.int(1), np.float(1.0), etc.\n # isinstance(value, np.nummber) checks np.int32(1), np.float64(1.0), etc.\n # isinstance(value, np.bool_) checks np.bool_(True), etc.\n # similar to np.isscalar but np.isscalar('st') returns True\n return isinstance(value, Number | np.number | np.bool_)\n\n\ndef _to_array_with_correct_type(obj: Any) -> np.ndarray:\n if isinstance(obj, np.ndarray) and issubclass(obj.dtype.type, np.bool_ | np.number):\n return obj # most often case\n # convert the value to np.ndarray\n # convert to object obj type if neither bool nor number\n # raises an exception if array's elements are tensors themselves\n try:\n obj_array = np.asanyarray(obj)\n except ValueError:\n obj_array = np.asanyarray(obj, dtype=object)\n if not issubclass(obj_array.dtype.type, np.bool_ | np.number):\n obj_array = obj_array.astype(object)\n if obj_array.dtype == object:\n # scalar ndarray with object obj type is very annoying\n # a=np.array([np.array({}, dtype=object), np.array({}, dtype=object)])\n # a is not array([{}, {}], dtype=object), and a[0]={} results in\n # something very strange:\n # array([{}, array({}, dtype=object)], dtype=object)\n if not obj_array.shape:\n obj_array = obj_array.item(0)\n elif all(isinstance(arr, np.ndarray) for arr in obj_array.reshape(-1)):\n return obj_array # various length, np.array([[1], [2, 3], [4, 5, 6]])\n elif any(isinstance(arr, torch.Tensor) for arr in obj_array.reshape(-1)):\n raise ValueError(\"Numpy arrays of tensors are not supported yet.\")\n return obj_array\n\n\ndef create_value(\n inst: Any,\n size: int,\n stack: bool = True,\n) -> Union[\"Batch\", np.ndarray, torch.Tensor]:\n \"\"\"Create empty place-holders according to inst's shape.\n\n :param stack: whether to stack or to concatenate. E.g. if inst has shape of\n (3, 5), size = 10, stack=True returns an np.array with shape of (10, 3, 5),\n otherwise (10, 5)\n \"\"\"\n has_shape = isinstance(inst, np.ndarray | torch.Tensor)\n is_scalar = _is_scalar(inst)\n if not stack and is_scalar:\n # should never hit since it has already checked in Batch.cat_ , here we do not\n # consider scalar types, following the behavior of numpy which does not support\n # concatenation of zero-dimensional arrays (scalars)\n raise TypeError(f\"cannot concatenate with {inst} which is scalar\")\n if has_shape:\n shape = (size, *inst.shape) if stack else (size, *inst.shape[1:])\n if isinstance(inst, np.ndarray):\n target_type = (\n inst.dtype.type if issubclass(inst.dtype.type, np.bool_ | np.number) else object\n )\n return np.full(shape, fill_value=None if target_type is object else 0, dtype=target_type)\n if isinstance(inst, torch.Tensor):\n return torch.full(shape, fill_value=0, device=inst.device, dtype=inst.dtype)\n if isinstance(inst, dict | Batch):\n zero_batch = Batch()\n for key, val in inst.items():\n zero_batch.__dict__[key] = create_value(val, size, stack=stack)\n return zero_batch\n if is_scalar:\n return create_value(np.asarray(inst), size, stack=stack)\n # fall back to object\n return np.array([None for _ in range(size)], object)\n\n\ndef _assert_type_keys(keys: Iterable[str]) -> None:\n assert all(isinstance(key, str) for key in keys), f\"keys should all be string, but got {keys}\"\n\n\ndef _parse_value(obj: Any) -> Union[\"Batch\", np.ndarray, torch.Tensor] | None:\n if isinstance(obj, Batch): # most often case\n return obj\n if (\n (isinstance(obj, np.ndarray) and issubclass(obj.dtype.type, np.bool_ | np.number))\n or isinstance(obj, torch.Tensor)\n or obj is None\n ): # third often case\n return obj\n if _is_number(obj): # second often case, but it is more time-consuming\n return np.asanyarray(obj)\n if isinstance(obj, dict):\n return Batch(obj)\n if (\n not isinstance(obj, np.ndarray)\n and isinstance(obj, Collection)\n and len(obj) > 0\n and all(isinstance(element, torch.Tensor) for element in obj)\n ):\n try:\n obj = cast(list[torch.Tensor], obj)\n return torch.stack(obj)\n except RuntimeError as exception:\n raise TypeError(\n \"Batch does not support non-stackable iterable\"\n \" of torch.Tensor as unique value yet.\",\n ) from exception\n if _is_batch_set(obj):\n obj = Batch(obj) # list of dict / Batch\n else:\n # None, scalar, normal obj list (main case)\n # or an actual list of objects\n try:\n obj = _to_array_with_correct_type(obj)\n except ValueError as exception:\n raise TypeError(\n \"Batch does not support heterogeneous list/tuple of tensors as unique value yet.\",\n ) from exception\n return obj\n\n\ndef alloc_by_keys_diff(\n meta: \"BatchProtocol\",\n batch: \"BatchProtocol\",\n size: int,\n stack: bool = True,\n) -> None:\n \"\"\"Creates place-holders inside meta for keys that are in batch but not in meta.\n\n This mainly is an internal method, use it only if you know what you are doing.\n \"\"\"\n for key in batch.get_keys():\n if key in meta.get_keys():\n if isinstance(meta[key], Batch) and isinstance(batch[key], Batch):\n alloc_by_keys_diff(meta[key], batch[key], size, stack)\n elif isinstance(meta[key], Batch) and len(meta[key].get_keys()) == 0:\n meta[key] = create_value(batch[key], size, stack)\n else:\n meta[key] = create_value(batch[key], size, stack)\n\n\nclass ProtocolCalledException(Exception):\n \"\"\"The methods of a Protocol should never be called.\n\n Currently, no static type checker actually verifies that a class that inherits\n from a Protocol does in fact provide the correct interface. Thus, it may happen\n that a method of the protocol is called accidentally (this is an\n implementation error). The normal error for that is a somewhat cryptic\n AttributeError, wherefore we instead raise this custom exception in the\n BatchProtocol.\n\n Finally and importantly: using this in BatchProtocol makes mypy verify the fields\n in the various sub-protocols and thus renders is MUCH more useful!\n \"\"\"\n\n\ndef get_sliced_dist(dist: TDistribution, index: IndexType) -> TDistribution:\n \"\"\"Slice a distribution object by the given index.\"\"\"\n if isinstance(dist, Categorical):\n return Categorical(probs=dist.probs[index]) # type: ignore[return-value]\n if isinstance(dist, Normal):\n return Normal(loc=dist.loc[index], scale=dist.scale[index]) # type: ignore[return-value]\n if isinstance(dist, Independent):\n return Independent(\n get_sliced_dist(dist.base_dist, index),\n dist.reinterpreted_batch_ndims,\n ) # type: ignore[return-value]\n else:\n raise NotImplementedError(f\"Unsupported distribution for slicing: {dist}\")\n\n\ndef get_len_of_dist(dist: Distribution) -> int:\n \"\"\"Return the length (typically batch size) of a distribution object.\"\"\"\n if len(dist.batch_shape) == 0:\n raise TypeError(f\"scalar Distribution has no length: {dist=}\")\n return dist.batch_shape[0]\n\n\ndef dist_to_atleast_2d(dist: TDistribution) -> TDistribution:\n \"\"\"Convert a distribution to at least 2D, such that the `batch_shape` attribute has a len of at least 1.\"\"\"\n if len(dist.batch_shape) > 0:\n return dist\n if isinstance(dist, Categorical):\n return Categorical(probs=dist.probs.unsqueeze(0)) # type: ignore[return-value]\n elif isinstance(dist, Normal):\n return Normal(loc=dist.loc.unsqueeze(0), scale=dist.scale.unsqueeze(0)) # type: ignore[return-value]\n elif isinstance(dist, Independent):\n return Independent(\n dist_to_atleast_2d(dist.base_dist),\n dist.reinterpreted_batch_ndims,\n ) # type: ignore[return-value]\n else:\n raise NotImplementedError(f\"Unsupported distribution for conversion to 2D: {type(dist)}\")\n\n\n# Note: This is implemented as a protocol because the interface\n# of Batch is always extended by adding new fields. Having a hierarchy of\n# protocols building off this one allows for type safety and IDE support despite\n# the dynamic nature of Batch\n@runtime_checkable\nclass BatchProtocol(Protocol):\n \"\"\"The internal data structure in Tianshou.\n\n Batch is a kind of supercharged array (of temporal data) stored individually in a\n (recursive) dictionary of objects that can be either numpy arrays, torch tensors, or\n batches themselves. It is designed to make it extremely easily to access, manipulate\n and set partial view of the heterogeneous data conveniently.\n \"\"\"\n\n @property\n def shape(self) -> list[int]:\n raise ProtocolCalledException\n\n # NOTE: even though setattr and getattr are defined for any object, we need\n # to explicitly define them for the BatchProtocol, since otherwise mypy will\n # complain about new fields being added dynamically. For example, things like\n # `batch.new_field = ...` followed by using `batch.new_field` become type errors\n # if getattr and setattr are missing in the BatchProtocol.\n #\n # For the moment, tianshou relies on this kind of dynamic-field-addition\n # in many, many places. In principle, it would be better to construct new\n # objects with new combinations of fields instead of mutating existing ones - the\n # latter is error-prone and can't properly be expressed with types. May be in a\n # future, rather different version of tianshou it would be feasible to have stricter\n # typing. Then the need for Protocols would in fact disappear\n def __setattr__(self, key: str, value: Any) -> None:\n raise ProtocolCalledException\n\n def __getattr__(self, key: str) -> Any:\n raise ProtocolCalledException\n\n def __iter__(self) -> Iterator[Self]:\n raise ProtocolCalledException\n\n @overload\n def __getitem__(self, index: str) -> Any:\n raise ProtocolCalledException\n\n @overload\n def __getitem__(self, index: IndexType) -> Self:\n raise ProtocolCalledException\n\n def __getitem__(self, index: str | IndexType) -> Any:\n raise ProtocolCalledException\n\n def __setitem__(self, index: str | IndexType, value: Any) -> None:\n raise ProtocolCalledException\n\n def __iadd__(self, other: Self | Number | np.number) -> Self:\n raise ProtocolCalledException\n\n def __add__(self, other: Self | Number | np.number) -> Self:\n raise ProtocolCalledException\n\n def __imul__(self, value: Number | np.number) -> Self:\n raise ProtocolCalledException\n\n def __mul__(self, value: Number | np.number) -> Self:\n raise ProtocolCalledException\n\n def __itruediv__(self, value: Number | np.number) -> Self:\n raise ProtocolCalledException\n\n def __truediv__(self, value: Number | np.number) -> Self:\n raise ProtocolCalledException\n\n def __repr__(self) -> str:\n raise ProtocolCalledException\n\n def __eq__(self, other: Any) -> bool:\n raise ProtocolCalledException\n\n def to_numpy(self: Self) -> Self:\n \"\"\"Change all torch.Tensor to numpy.ndarray and return a new Batch.\"\"\"\n raise ProtocolCalledException\n\n def to_numpy_(self) -> None:\n \"\"\"Change all torch.Tensor to numpy.ndarray in-place.\"\"\"\n raise ProtocolCalledException\n\n def to_torch(\n self: Self,\n dtype: torch.dtype | None = None,\n device: str | int | torch.device = \"cpu\",\n ) -> Self:\n \"\"\"Change all numpy.ndarray to torch.Tensor and return a new Batch.\"\"\"\n raise ProtocolCalledException\n\n def to_torch_(\n self,\n dtype: torch.dtype | None = None,\n device: str | int | torch.device = \"cpu\",\n ) -> None:\n \"\"\"Change all numpy.ndarray to torch.Tensor in-place.\"\"\"\n raise ProtocolCalledException\n\n def cat_(self, batches: Self | Sequence[dict | Self]) -> None:\n \"\"\"Concatenate a list of (or one) Batch objects into current batch.\"\"\"\n raise ProtocolCalledException\n\n @staticmethod\n def cat(batches: Sequence[dict | TBatch]) -> TBatch:\n \"\"\"Concatenate a list of Batch object into a single new batch.\n\n For keys that are not shared across all batches, batches that do not\n have these keys will be padded by zeros with appropriate shapes. E.g.\n ::\n\n >>> a = Batch(a=np.zeros([3, 4]), common=Batch(c=np.zeros([3, 5])))\n >>> b = Batch(b=np.zeros([4, 3]), common=Batch(c=np.zeros([4, 5])))\n >>> c = Batch.cat([a, b])\n >>> c.a.shape\n (7, 4)\n >>> c.b.shape\n (7, 3)\n >>> c.common.c.shape\n (7, 5)\n \"\"\"\n raise ProtocolCalledException\n\n def stack_(self, batches: Sequence[dict | Self], axis: int = 0) -> None:\n \"\"\"Stack a list of Batch object into current batch.\"\"\"\n raise ProtocolCalledException\n\n @staticmethod\n def stack(batches: Sequence[dict | TBatch], axis: int = 0) -> TBatch:\n \"\"\"Stack a list of Batch object into a single new batch.\n\n For keys that are not shared across all batches, batches that do not\n have these keys will be padded by zeros. E.g.\n ::\n\n >>> a = Batch(a=np.zeros([4, 4]), common=Batch(c=np.zeros([4, 5])))\n >>> b = Batch(b=np.zeros([4, 6]), common=Batch(c=np.zeros([4, 5])))\n >>> c = Batch.stack([a, b])\n >>> c.a.shape\n (2, 4, 4)\n >>> c.b.shape\n (2, 4, 6)\n >>> c.common.c.shape\n (2, 4, 5)\n\n .. note::\n\n If there are keys that are not shared across all batches, ``stack``\n with ``axis != 0`` is undefined, and will cause an exception.\n \"\"\"\n raise ProtocolCalledException\n\n def empty_(self, index: slice | IndexType | None = None) -> Self:\n \"\"\"Return an empty Batch object with 0 or None filled.\n\n If \"index\" is specified, it will only reset the specific indexed-data.\n ::\n\n >>> data.empty_()\n >>> print(data)\n Batch(\n a: array([[0., 0.],\n [0., 0.]]),\n b: array([None, None], dtype=object),\n )\n >>> b={'c': [2., 'st'], 'd': [1., 0.]}\n >>> data = Batch(a=[False, True], b=b)\n >>> data[0] = Batch.empty(data[1])\n >>> data\n Batch(\n a: array([False, True]),\n b: Batch(\n c: array([None, 'st']),\n d: array([0., 0.]),\n ),\n )\n \"\"\"\n raise ProtocolCalledException\n\n @staticmethod\n def empty(batch: TBatch, index: IndexType | None = None) -> TBatch:\n \"\"\"Return an empty Batch object with 0 or None filled.\n\n The shape is the same as the given Batch.\n \"\"\"\n raise ProtocolCalledException\n\n def update(self, batch: dict | Self | None = None, **kwargs: Any) -> None:\n \"\"\"Update this batch from another dict/Batch.\"\"\"\n raise ProtocolCalledException\n\n def __len__(self) -> int:\n raise ProtocolCalledException\n\n def split(\n self,\n size: int,\n shuffle: bool = True,\n merge_last: bool = False,\n ) -> Iterator[Self]:\n \"\"\"Split whole data into multiple small batches.\n\n :param size: divide the data batch with the given size, but one\n batch if the length of the batch is smaller than \"size\". Size of -1 means\n the whole batch.\n :param shuffle: randomly shuffle the entire data batch if it is\n True, otherwise remain in the same. Default to True.\n :param merge_last: merge the last batch into the previous one.\n Default to False.\n \"\"\"\n raise ProtocolCalledException\n\n def to_dict(self, recurse: bool = True) -> dict[str, Any]:\n raise ProtocolCalledException\n\n def to_list_of_dicts(self) -> list[dict[str, Any]]:\n raise ProtocolCalledException\n\n def get_keys(self) -> KeysView:\n raise ProtocolCalledException\n\n def set_array_at_key(\n self,\n seq: np.ndarray,\n key: str,\n index: IndexType | None = None,\n default_value: float | None = None,\n ) -> None:\n \"\"\"Set a sequence of values at a given key.\n\n If `index` is not passed, the sequence must have the same length as the batch.\n\n :param seq: the array of values to set.\n :param key: the key to set the sequence at.\n :param index: the indices to set the sequence at. If None, the sequence must have\n the same length as the batch and will be set at all indices.\n :param default_value: this only applies if `index` is passed and the key does not exist yet\n in the batch. In that case, entries outside the passed index will be filled\n with this default value.\n Note that the array at the key will be of the same dtype as the passed sequence,\n so `default_value` should be such that numpy can cast it to this dtype.\n \"\"\"\n raise ProtocolCalledException\n\n def isnull(self) -> Self:\n \"\"\"Return a boolean mask of the same shape, indicating missing values.\"\"\"\n raise ProtocolCalledException\n\n def hasnull(self) -> bool:\n \"\"\"Return whether the batch has missing values.\"\"\"\n raise ProtocolCalledException\n\n def dropnull(self) -> Self:\n \"\"\"Return a batch where all items in which any value is null are dropped.\n\n Note that it is not the same as just dropping the entries of the sequence.\n For example, with\n\n >>> b = Batch(a=[None, 2, 3, 4], b=[4, 5, None, 7])\n >>> b.dropnull()\n\n will result in\n\n >>> Batch(a=[2, 4], b=[5, 7])\n\n This logic is applied recursively to all nested batches. The result is\n the same as if the batch was flattened, entries were dropped,\n and then the batch was reshaped back to the original nested structure.\n \"\"\"\n ...\n\n @overload\n def apply_values_transform(\n self,\n values_transform: Callable[[np.ndarray | torch.Tensor], Any],\n ) -> Self: ...\n\n @overload\n def apply_values_transform(\n self,\n values_transform: Callable,\n inplace: Literal[True],\n ) -> None: ...\n\n @overload\n def apply_values_transform(\n self,\n values_transform: Callable[[np.ndarray | torch.Tensor], Any],\n inplace: Literal[False],\n ) -> Self: ...\n\n def apply_values_transform(\n self,\n values_transform: Callable[[np.ndarray | torch.Tensor], Any],\n inplace: bool = False,\n ) -> None | Self:\n \"\"\"Apply a function to all arrays in the batch, including nested ones.\n\n :param values_transform: the function to apply to the arrays.\n :param inplace: whether to apply the function in-place. If False, a new batch is returned,\n otherwise the batch is modified in-place and None is returned.\n \"\"\"\n raise ProtocolCalledException\n\n def get(self, key: str, default: Any | None = None) -> Any:\n raise ProtocolCalledException\n\n def pop(self, key: str, default: Any | None = None) -> Any:\n raise ProtocolCalledException\n\n def to_at_least_2d(self) -> Self:\n \"\"\"Ensures that all arrays and dists in the batch have at least 2 dimensions.\n\n This is useful for ensuring that all arrays in the batch can be concatenated\n along a new axis.\n \"\"\"\n raise ProtocolCalledException\n\n\nclass Batch(BatchProtocol):\n \"\"\"See :class:`~tianshou.data.batch.BatchProtocol`.\"\"\"\n\n __doc__ = BatchProtocol.__doc__\n\n def __init__(\n self,\n batch_dict: dict\n | BatchProtocol\n | Sequence[dict | BatchProtocol]\n | np.ndarray\n | None = None,\n copy: bool = False,\n **kwargs: Any,\n ) -> None:\n if copy:\n batch_dict = deepcopy(batch_dict)\n if batch_dict is not None:\n if isinstance(batch_dict, dict | BatchProtocol):\n _assert_type_keys(batch_dict.keys())\n for batch_key, obj in batch_dict.items():\n self.__dict__[batch_key] = _parse_value(obj)\n elif _is_batch_set(batch_dict):\n batch_dict = cast(Sequence[dict | BatchProtocol], batch_dict)\n self.stack_(batch_dict)\n if len(kwargs) > 0:\n # TODO: that's a rather weird pattern, is it really needed?\n # Feels like kwargs could be just merged into batch_dict in the beginning\n self.__init__(kwargs, copy=copy) # type: ignore\n\n def to_dict(self, recursive: bool = True) -> dict[str, Any]:\n result = {}\n for k, v in self.__dict__.items():\n if recursive and isinstance(v, Batch):\n v = v.to_dict(recursive=recursive)\n result[k] = v\n return result\n\n def get_keys(self) -> KeysView:\n return self.__dict__.keys()\n\n def get(self, key: str, default: Any | None = None) -> Any:\n return self.__dict__.get(key, default)\n\n def pop(self, key: str, default: Any | None = None) -> Any:\n return self.__dict__.pop(key, default)\n\n def to_list_of_dicts(self) -> list[dict[str, Any]]:\n return [entry.to_dict() for entry in self]\n\n def __setattr__(self, key: str, value: Any) -> None:\n \"\"\"Set self.key = value.\"\"\"\n self.__dict__[key] = _parse_value(value)\n\n def __getattr__(self, key: str) -> Any:\n \"\"\"Return self.key. The \"Any\" return type is needed for mypy.\"\"\"\n return getattr(self.__dict__, key)\n\n def __contains__(self, key: str) -> bool:\n \"\"\"Return key in self.\"\"\"\n return key in self.__dict__\n\n def __getstate__(self) -> dict[str, Any]:\n \"\"\"Pickling interface.\n\n Only the actual data are serialized for both efficiency and simplicity.\n \"\"\"\n state = {}\n for batch_key, obj in self.items():\n if isinstance(obj, Batch):\n state[batch_key] = obj.__getstate__()\n else:\n state[batch_key] = obj\n return state\n\n def __setstate__(self, state: dict[str, Any]) -> None:\n \"\"\"Unpickling interface.\n\n At this point, self is an empty Batch instance that has not been\n initialized, so it can safely be initialized by the pickle state.\n \"\"\"\n self.__init__(**state) # type: ignore\n\n @overload\n def __getitem__(self, index: str) -> Any: ...\n\n @overload\n def __getitem__(self, index: IndexType) -> Self: ...\n\n def __getitem__(self, index: str | IndexType) -> Any:\n \"\"\"Returns either the value of a key or a sliced Batch object.\"\"\"\n if isinstance(index, str):\n return self.__dict__[index]\n batch_items = self.items()\n if len(batch_items) > 0:\n new_batch = Batch()\n\n sliced_obj: Any\n for batch_key, obj in batch_items:\n # None and empty Batches as values are added to any slice\n if obj is None:\n sliced_obj = None\n elif isinstance(obj, Batch) and len(obj.get_keys()) == 0:\n sliced_obj = Batch()\n # We attempt slicing of a distribution. This is hacky, but presents an important special case\n elif isinstance(obj, Distribution):\n sliced_obj = get_sliced_dist(obj, index)\n # All other objects are either array-like or Batch-like, so hopefully sliceable\n # A batch should have no scalars\n else:\n sliced_obj = obj[index]\n new_batch.__dict__[batch_key] = sliced_obj\n return new_batch\n raise IndexError(\"Cannot access item from empty Batch object.\")\n\n def __eq__(self, other: Any) -> bool:\n if not isinstance(other, self.__class__):\n return False\n\n this_batch_no_torch_tensor = self.to_numpy()\n other_batch_no_torch_tensor = other.to_numpy()\n # DeepDiff 7.0.1 cannot compare 0-dimensional arrays\n # so, we ensure with this transform that all array values have at least 1 dim\n this_batch_no_torch_tensor.apply_values_transform(\n values_transform=np.atleast_1d,\n inplace=True,\n )\n other_batch_no_torch_tensor.apply_values_transform(\n values_transform=np.atleast_1d,\n inplace=True,\n )\n this_dict = this_batch_no_torch_tensor.to_dict(recursive=True)\n other_dict = other_batch_no_torch_tensor.to_dict(recursive=True)\n\n return not DeepDiff(this_dict, other_dict)\n\n def __iter__(self) -> Iterator[Self]:\n # TODO: empty batch raises an error on len and needs separate treatment, that's probably not a good idea\n if len(self.__dict__) == 0:\n yield from []\n else:\n for i in range(len(self)):\n yield self[i]\n\n def __setitem__(self, index: str | IndexType, value: Any) -> None:\n \"\"\"Assign value to self[index].\"\"\"\n value = _parse_value(value)\n if isinstance(index, str):\n self.__dict__[index] = value\n return\n if not isinstance(value, Batch):\n raise ValueError(\n \"Batch does not supported tensor assignment. \"\n \"Use a compatible Batch or dict instead.\",\n )\n if not set(value.keys()).issubset(self.__dict__.keys()):\n raise ValueError(\"Creating keys is not supported by item assignment.\")\n for key, val in self.items():\n try:\n self.__dict__[key][index] = value[key]\n except KeyError:\n if isinstance(val, Batch):\n self.__dict__[key][index] = Batch()\n elif isinstance(val, torch.Tensor) or (\n isinstance(val, np.ndarray) and issubclass(val.dtype.type, np.bool_ | np.number)\n ):\n self.__dict__[key][index] = 0\n else:\n self.__dict__[key][index] = None\n\n def __iadd__(self, other: Self | Number | np.number) -> Self:\n \"\"\"Algebraic addition with another Batch instance in-place.\"\"\"\n if isinstance(other, Batch):\n for (batch_key, obj), value in zip(\n self.__dict__.items(),\n other.__dict__.values(),\n strict=True,\n ): # TODO are keys consistent?\n if isinstance(obj, Batch) and len(obj.get_keys()) == 0:\n continue\n self.__dict__[batch_key] += value\n return self\n if _is_number(other):\n for batch_key, obj in self.items():\n if isinstance(obj, Batch) and len(obj.get_keys()) == 0:\n continue\n self.__dict__[batch_key] += other\n return self\n raise TypeError(\"Only addition of Batch or number is supported.\")\n\n def __add__(self, other: Self | Number | np.number) -> Self:\n \"\"\"Algebraic addition with another Batch instance out-of-place.\"\"\"\n return deepcopy(self).__iadd__(other)\n\n def __imul__(self, value: Number | np.number) -> Self:\n \"\"\"Algebraic multiplication with a scalar value in-place.\"\"\"\n assert _is_number(value), \"Only multiplication by a number is supported.\"\n for batch_key, obj in self.__dict__.items():\n if isinstance(obj, Batch) and len(obj.get_keys()) == 0:\n continue\n self.__dict__[batch_key] *= value\n return self\n\n def __mul__(self, value: Number | np.number) -> Self:\n \"\"\"Algebraic multiplication with a scalar value out-of-place.\"\"\"\n return deepcopy(self).__imul__(value)\n\n def __itruediv__(self, value: Number | np.number) -> Self:\n \"\"\"Algebraic division with a scalar value in-place.\"\"\"\n assert _is_number(value), \"Only division by a number is supported.\"\n for batch_key, obj in self.__dict__.items():\n if isinstance(obj, Batch) and len(obj.get_keys()) == 0:\n continue\n self.__dict__[batch_key] /= value\n return self\n\n def __truediv__(self, value: Number | np.number) -> Self:\n \"\"\"Algebraic division with a scalar value out-of-place.\"\"\"\n return deepcopy(self).__itruediv__(value)\n\n def __repr__(self) -> str:\n \"\"\"Return str(self).\"\"\"\n self_str = self.__class__.__name__ + \"(\\n\"\n flag = False\n for batch_key, obj in self.__dict__.items():\n rpl = \"\\n\" + \" \" * (6 + len(batch_key))\n obj_name = pprint.pformat(obj).replace(\"\\n\", rpl)\n self_str += f\" {batch_key}: {obj_name},\\n\"\n flag = True\n if flag:\n self_str += \")\"\n else:\n self_str = self.__class__.__name__ + \"()\"\n return self_str\n\n def to_numpy(self: Self) -> Self:\n result = deepcopy(self)\n result.to_numpy_()\n return result\n\n def to_numpy_(self) -> None:\n def arr_to_numpy(arr: TArr) -> TArr:\n if isinstance(arr, torch.Tensor):\n return arr.detach().cpu().numpy()\n return arr\n\n self.apply_values_transform(arr_to_numpy, inplace=True)\n\n def to_torch(\n self: Self,\n dtype: torch.dtype | None = None,\n device: str | int | torch.device = \"cpu\",\n ) -> Self:\n result = deepcopy(self)\n result.to_torch_(dtype=dtype, device=device)\n return result\n\n def to_torch_(\n self,\n dtype: torch.dtype | None = None,\n device: str | int | torch.device = \"cpu\",\n ) -> None:\n if not isinstance(device, torch.device):\n device = torch.device(device)\n\n def arr_to_torch(arr: TArr) -> TArr:\n if isinstance(arr, np.ndarray):\n return torch.from_numpy(arr).to(device)\n\n # TODO: simplify\n if (\n (dtype is not None and arr.dtype != dtype)\n or arr.device.type != device.type\n or device.index != arr.device.index\n ):\n if dtype is not None:\n arr = arr.type(dtype)\n return arr.to(device)\n return arr\n\n self.apply_values_transform(arr_to_torch, inplace=True)\n\n def __cat(self, batches: Sequence[dict | Self], lens: list[int]) -> None:\n \"\"\"Private method for Batch.cat_.\n\n ::\n\n >>> a = Batch(a=np.random.randn(3, 4))\n >>> x = Batch(a=a, b=np.random.randn(4, 4))\n >>> y = Batch(a=Batch(a=Batch()), b=np.random.randn(4, 4))\n\n If we want to concatenate x and y, we want to pad y.a.a with zeros.\n Without ``lens`` as a hint, when we concatenate x.a and y.a, we would\n not be able to know how to pad y.a. So ``Batch.cat_`` should compute\n the ``lens`` to give ``Batch.__cat`` a hint.\n ::\n\n >>> ans = Batch.cat([x, y])\n >>> # this is equivalent to the following line\n >>> ans = Batch(); ans.__cat([x, y], lens=[3, 4])\n >>> # this lens is equal to [len(a), len(b)]\n \"\"\"\n # partial keys will be padded by zeros\n # with the shape of [len, rest_shape]\n sum_lens = [0]\n for len_ in lens:\n sum_lens.append(sum_lens[-1] + len_)\n # collect non-empty keys\n keys_map = [\n {\n batch_key\n for batch_key, obj in batch.items()\n if not (isinstance(obj, Batch) and len(obj.get_keys()) == 0)\n }\n for batch in batches\n ]\n keys_shared = set.intersection(*keys_map)\n values_shared = [[batch[key] for batch in batches] for key in keys_shared]\n for key, shared_value in zip(keys_shared, values_shared, strict=True):\n if all(isinstance(element, dict | Batch) for element in shared_value):\n batch_holder = Batch()\n batch_holder.__cat(shared_value, lens=lens)\n self.__dict__[key] = batch_holder\n elif all(isinstance(element, torch.Tensor) for element in shared_value):\n self.__dict__[key] = torch.cat(shared_value)\n else:\n # cat Batch(a=np.zeros((3, 4))) and Batch(a=Batch(b=Batch()))\n # will fail here\n self.__dict__[key] = _to_array_with_correct_type(np.concatenate(shared_value))\n keys_total = set.union(*[set(batch.keys()) for batch in batches])\n keys_reserve_or_partial = set.difference(keys_total, keys_shared)\n # keys that are reserved in all batches\n keys_reserve = set.difference(keys_total, set.union(*keys_map))\n # keys that occur only in some batches, but not all\n keys_partial = keys_reserve_or_partial.difference(keys_reserve)\n for key in keys_reserve:\n # reserved keys\n self.__dict__[key] = Batch()\n for key in keys_partial:\n for i, batch in enumerate(batches):\n if key not in batch.__dict__:\n continue\n value = batch.get(key)\n if isinstance(value, Batch) and len(value.get_keys()) == 0:\n continue\n try:\n self.__dict__[key][sum_lens[i] : sum_lens[i + 1]] = value\n except KeyError:\n self.__dict__[key] = create_value(value, sum_lens[-1], stack=False)\n self.__dict__[key][sum_lens[i] : sum_lens[i + 1]] = value\n\n def cat_(self, batches: BatchProtocol | Sequence[dict | BatchProtocol]) -> None:\n if isinstance(batches, Batch | dict):\n batches = [batches]\n # check input format\n batch_list = []\n\n original_keys_only_batch = None\n \"\"\"A batch with all values removed, just keys left. Can be considered a sort of schema.\n Will be either the schema of self, or of the first non-empty batch in the sequence.\n \"\"\"\n if len(self) > 0:\n original_keys_only_batch = self.apply_values_transform(lambda x: None)\n original_keys_only_batch.replace_empty_batches_by_none()\n\n for batch in batches:\n if isinstance(batch, dict):\n batch = Batch(batch)\n if not isinstance(batch, Batch):\n raise ValueError(f\"Cannot concatenate {type(batch)} in Batch.cat_\")\n if len(batch.get_keys()) == 0:\n continue\n if original_keys_only_batch is None:\n original_keys_only_batch = batch.apply_values_transform(lambda x: None)\n original_keys_only_batch.replace_empty_batches_by_none()\n batch_list.append(batch)\n continue\n\n cur_keys_only_batch = batch.apply_values_transform(lambda x: None)\n cur_keys_only_batch.replace_empty_batches_by_none()\n if original_keys_only_batch != cur_keys_only_batch:\n raise ValueError(\n f\"Batch.cat_ only supports concatenation of batches with the same structure but got \"\n f\"structures: \\n{original_keys_only_batch}\\n and\\n{cur_keys_only_batch}.\",\n )\n batch_list.append(batch)\n if len(batch_list) == 0:\n return\n\n batches = batch_list\n\n # TODO: lot's of the remaining logic is devoted to filling up remaining keys with zeros\n # this should be removed, and also the check above should be extended to nested keys\n try:\n # len(batch) here means batch is a nested empty batch\n # like Batch(a=Batch), and we have to treat it as length zero and\n # keep it.\n lens = [0 if len(batch) == 0 else len(batch) for batch in batches]\n except TypeError as exception:\n raise ValueError(\n \"Batch.cat_ meets an exception. Maybe because there is any \"\n f\"scalar in {batches} but Batch.cat_ does not support the \"\n \"concatenation of scalar.\",\n ) from exception\n if len(self.get_keys()) != 0:\n batches = [self, *list(batches)]\n # len of zero means that that item is Batch() and should be ignored\n lens = [0 if len(self) == 0 else len(self), *lens]\n self.__cat(batches, lens)\n\n @staticmethod\n def cat(batches: Sequence[dict | TBatch]) -> TBatch:\n batch = Batch()\n batch.cat_(batches)\n return batch # type: ignore\n\n def stack_(self, batches: Sequence[dict | BatchProtocol], axis: int = 0) -> None:\n # check input format\n batch_list = []\n for batch in batches:\n if isinstance(batch, dict):\n if len(batch) > 0:\n batch_list.append(Batch(batch))\n elif isinstance(batch, Batch):\n if len(batch.get_keys()) != 0:\n batch_list.append(batch)\n else:\n raise ValueError(f\"Cannot concatenate {type(batch)} in Batch.stack_\")\n if len(batch_list) == 0:\n return\n batches = batch_list\n if len(self.get_keys()) != 0:\n batches = [self, *batches]\n # collect non-empty keys\n keys_map = [\n {\n batch_key\n for batch_key, obj in batch.items()\n if not (isinstance(obj, Batch) and len(obj.get_keys()) == 0)\n }\n for batch in batches\n ]\n keys_shared = set.intersection(*keys_map)\n values_shared = [[batch[key] for batch in batches] for key in keys_shared]\n for shared_key, value in zip(keys_shared, values_shared, strict=True):\n # second often\n if all(isinstance(element, torch.Tensor) for element in value):\n self.__dict__[shared_key] = torch.stack(value, axis)\n # third often\n elif all(isinstance(element, Batch | dict) for element in value):\n self.__dict__[shared_key] = Batch.stack(value, axis)\n else: # most often case is np.ndarray\n try:\n self.__dict__[shared_key] = _to_array_with_correct_type(np.stack(value, axis))\n except ValueError:\n warnings.warn(\n \"You are using tensors with different shape,\"\n \" fallback to dtype=object by default.\",\n )\n self.__dict__[shared_key] = np.array(value, dtype=object)\n # all the keys\n keys_total = set.union(*[set(batch.keys()) for batch in batches])\n # keys that are reserved in all batches\n keys_reserve = set.difference(keys_total, set.union(*keys_map))\n # keys that are either partial or reserved\n keys_reserve_or_partial = set.difference(keys_total, keys_shared)\n # keys that occur only in some batches, but not all\n keys_partial = keys_reserve_or_partial.difference(keys_reserve)\n if keys_partial and axis != 0:\n raise ValueError(\n f\"Stack of Batch with non-shared keys {keys_partial} is only \"\n f\"supported with axis=0, but got axis={axis}!\",\n )\n for key in keys_reserve:\n # reserved keys\n self.__dict__[key] = Batch()\n for key in keys_partial:\n for i, batch in enumerate(batches):\n if key not in batch.__dict__:\n continue\n value = batch.get(key)\n # TODO: fix code/annotations s.t. the ignores can be removed\n if (\n isinstance(value, Batch) # type: ignore\n and len(value.get_keys()) == 0 # type: ignore\n ):\n continue # type: ignore\n try:\n self.__dict__[key][i] = value\n except KeyError:\n self.__dict__[key] = create_value(value, len(batches))\n self.__dict__[key][i] = value\n\n @staticmethod\n def stack(batches: Sequence[dict | TBatch], axis: int = 0) -> TBatch:\n batch = Batch()\n batch.stack_(batches, axis)\n # can't cast to a generic type, so we have to ignore the type here\n return batch # type: ignore\n\n def empty_(self, index: slice | IndexType | None = None) -> Self:\n for batch_key, obj in self.items():\n if isinstance(obj, torch.Tensor): # most often case\n self.__dict__[batch_key][index] = 0\n elif obj is None:\n continue\n elif isinstance(obj, np.ndarray):\n if obj.dtype == object:\n self.__dict__[batch_key][index] = None\n else:\n self.__dict__[batch_key][index] = 0\n elif isinstance(obj, Batch):\n self.__dict__[batch_key].empty_(index=index)\n else: # scalar value\n warnings.warn(\n \"You are calling Batch.empty on a NumPy scalar, \"\n \"which may cause undefined behaviors.\",\n )\n if _is_number(obj):\n self.__dict__[batch_key] = obj.__class__(0)\n else:\n self.__dict__[batch_key] = None\n return self\n\n @staticmethod\n def empty(batch: TBatch, index: IndexType | None = None) -> TBatch:\n return deepcopy(batch).empty_(index)\n\n def update(self, batch: dict | Self | None = None, **kwargs: Any) -> None:\n if batch is None:\n self.update(kwargs)\n return\n for batch_key, obj in batch.items():\n self.__dict__[batch_key] = _parse_value(obj)\n if kwargs:\n self.update(kwargs)\n\n def __len__(self) -> int:\n \"\"\"Raises `TypeError` if any value in the batch has no len(), typically meaning it's a batch of scalars.\"\"\"\n lens = []\n for key, obj in self.__dict__.items():\n # TODO: causes inconsistent behavior to batch with empty batches\n # and batch with empty sequences of other type. Remove, but only after\n # Buffer and Collectors have been improved to no longer rely on this\n if isinstance(obj, Batch) and len(obj) == 0:\n continue\n if obj is None:\n continue\n if hasattr(obj, \"__len__\") and (isinstance(obj, Batch) or obj.ndim > 0):\n lens.append(len(obj))\n continue\n if isinstance(obj, Distribution):\n lens.append(get_len_of_dist(obj))\n continue\n raise TypeError(f\"Entry for {key} in {self} is {obj} has no len()\")\n if not lens:\n return 0\n return min(lens)\n\n @property\n def shape(self) -> list[int]:\n \"\"\"Return self.shape.\"\"\"\n if len(self.get_keys()) == 0:\n return []\n data_shape = []\n for obj in self.__dict__.values():\n try:\n data_shape.append(list(obj.shape))\n except AttributeError:\n data_shape.append([])\n return (\n list(map(min, zip(*data_shape, strict=False))) if len(data_shape) > 1 else data_shape[0]\n )\n\n def split(\n self,\n size: int,\n shuffle: bool = True,\n merge_last: bool = False,\n ) -> Iterator[Self]:\n length = len(self)\n if size == -1:\n size = length\n assert size >= 1 # size can be greater than length, return whole batch\n indices = np.random.permutation(length) if shuffle else np.arange(length)\n merge_last = merge_last and length % size > 0\n for idx in range(0, length, size):\n if merge_last and idx + size + size >= length:\n yield self[indices[idx:]]\n break\n yield self[indices[idx : idx + size]]\n\n @overload\n def apply_values_transform(\n self,\n values_transform: Callable,\n ) -> Self: ...\n\n @overload\n def apply_values_transform(\n self,\n values_transform: Callable,\n inplace: Literal[True],\n ) -> None: ...\n\n @overload\n def apply_values_transform(\n self,\n values_transform: Callable,\n inplace: Literal[False],\n ) -> Self: ...\n\n def apply_values_transform(\n self,\n values_transform: Callable,\n inplace: bool = False,\n ) -> None | Self:\n \"\"\"Applies a function to all non-batch-values in the batch, including\n values in nested batches.\n\n A batch with keys pointing to either batches or to non-batch values can\n be thought of as a tree of Batch nodes. This function traverses the tree\n and applies the function to all leaf nodes (i.e. values that are not\n batches themselves).\n\n The values are usually arrays, but can also be scalar values of an\n arbitrary type since retrieving a single entry from a Batch a la\n `batch[0]` will return a batch with scalar values.\n \"\"\"\n return _apply_batch_values_func_recursively(self, values_transform, inplace=inplace)\n\n def set_array_at_key(\n self,\n arr: np.ndarray,\n key: str,\n index: IndexType | None = None,\n default_value: float | None = None,\n ) -> None:\n if index is not None:\n if key not in self.get_keys():\n log.info(\n f\"Key {key} not found in batch, \"\n f\"creating a sequence of len {len(self)} with {default_value=} for it.\",\n )\n try:\n self[key] = np.array([default_value] * len(self), dtype=arr.dtype)\n except TypeError as exception:\n raise TypeError(\n f\"Cannot create a sequence of dtype {arr.dtype} with default value {default_value}. \"\n f\"You can fix this either by passing an array with the correct dtype or by passing \"\n f\"a different default value that can be cast to the array's dtype (or both).\",\n ) from exception\n else:\n existing_entry = self[key]\n if isinstance(existing_entry, Batch):\n raise ValueError(\n f\"Cannot set sequence at key {key} because it is a nested batch, \"\n f\"can only set a subsequence of an array.\",\n )\n self[key][index] = arr\n else:\n if len(arr) != len(self):\n raise ValueError(\n f\"Sequence length {len(arr)} does not match \"\n f\"batch length {len(self)}. For setting a subsequence with missing \"\n f\"entries filled up by default values, consider passing an index.\",\n )\n self[key] = arr\n\n def isnull(self) -> Self:\n return self.apply_values_transform(pd.isnull, inplace=False)\n\n def hasnull(self) -> bool:\n isnan_batch = self.isnull()\n is_any_null_batch = isnan_batch.apply_values_transform(np.any, inplace=False)\n\n def is_any_true(boolean_batch: BatchProtocol) -> bool:\n for val in boolean_batch.values():\n if isinstance(val, Batch):\n if is_any_true(val):\n return True\n else:\n assert val.size == 1, \"This shouldn't have happened, it's a bug!\"\n # an unsized array with a boolean, e.g. np.array(False). behaves like the boolean itself\n if val:\n return True\n return False\n\n return is_any_true(is_any_null_batch)\n\n def dropnull(self) -> Self:\n # we need to use dicts since a batch retrieved for a single index has no length and cat fails\n # TODO: make cat work with batches containing scalars?\n sub_batches = []\n for b in self:\n if b.hasnull():\n continue\n # needed for cat to work\n b = b.apply_values_transform(np.atleast_1d)\n sub_batches.append(b)\n return Batch.cat(sub_batches)\n\n def replace_empty_batches_by_none(self) -> None:\n \"\"\"Goes through the batch-tree\" recursively and replaces empty batches by None.\n\n This is useful for extracting the structure of a batch without the actual data,\n especially in combination with `apply_values_transform` with a\n transform function a la `lambda x: None`.\n \"\"\"\n empty_batch = Batch()\n for key, val in self.items():\n if isinstance(val, Batch):\n if val == empty_batch:\n self[key] = None\n else:\n val.replace_empty_batches_by_none()\n\n def to_at_least_2d(self) -> Self:\n \"\"\"Ensures that all arrays and dists in the batch have at least 2 dimensions.\n\n This is useful for ensuring that all arrays in the batch can be concatenated\n along a new axis.\n \"\"\"\n result = self.apply_values_transform(np.atleast_2d, inplace=False)\n for key, val in self.items():\n if isinstance(val, Distribution):\n result[key] = dist_to_atleast_2d(val)\n return result\n\n\ndef _apply_batch_values_func_recursively(\n batch: TBatch,\n values_transform: Callable,\n inplace: bool = False,\n) -> TBatch | None:\n \"\"\"Applies the desired function on all values of the batch recursively.\n\n See docstring of the corresponding method in the Batch class for more details.\n \"\"\"\n result = batch if inplace else deepcopy(batch)\n for key, val in batch.__dict__.items():\n if isinstance(val, Batch):\n result[key] = _apply_batch_values_func_recursively(val, values_transform, inplace=False)\n else:\n result[key] = values_transform(val)\n if not inplace:\n return result\n return None\n" }, { "path": "tianshou/data/buffer/__init__.py", "content": "def _backward_compatibility() -> None:\n import sys\n\n from . import buffer_base\n\n # backward compatibility with persisted buffers from v1 for determinism tests\n sys.modules[\"tianshou.data.buffer.base\"] = buffer_base\n\n\n_backward_compatibility()\n" }, { "path": "tianshou/data/buffer/buffer_base.py", "content": "from collections.abc import Sequence\nfrom typing import Any, ClassVar, Self, TypeVar, cast\n\nimport h5py\nimport numpy as np\nfrom sensai.util.pickle import setstate\n\nfrom tianshou.data import Batch\nfrom tianshou.data.batch import (\n IndexType,\n alloc_by_keys_diff,\n create_value,\n log,\n)\nfrom tianshou.data.types import RolloutBatchProtocol\nfrom tianshou.data.utils.converter import from_hdf5, to_hdf5\n\nTBuffer = TypeVar(\"TBuffer\", bound=\"ReplayBuffer\")\n\n\nclass MalformedBufferError(RuntimeError):\n pass\n\n\nclass ReplayBuffer:\n \"\"\":class:`~tianshou.data.ReplayBuffer` stores data generated from interaction between the policy and environment.\n\n ReplayBuffer can be considered as a specialized form (or management) of Batch. It\n stores all the data in a batch with circular-queue style.\n\n :param size: the maximum size of replay buffer.\n :param stack_num: the frame-stack sampling argument, should be greater than or\n equal to 1. Default to 1 (no stacking).\n :param ignore_obs_next: whether to not store obs_next.\n :param save_only_last_obs: only save the last obs/obs_next when it has a shape\n of (timestep, ...) because of temporal stacking.\n :param sample_avail: whether to sample only available indices\n when using the frame-stack sampling method.\n \"\"\"\n\n _reserved_keys = (\n \"obs\",\n \"act\",\n \"rew\",\n \"terminated\",\n \"truncated\",\n \"done\",\n \"obs_next\",\n \"info\",\n \"policy\",\n )\n _input_keys = (\n \"obs\",\n \"act\",\n \"rew\",\n \"terminated\",\n \"truncated\",\n \"obs_next\",\n \"info\",\n \"policy\",\n )\n _required_keys_for_add: ClassVar[set[str]] = {\n \"obs\",\n \"act\",\n \"rew\",\n \"terminated\",\n \"truncated\",\n \"done\",\n }\n\n def __init__(\n self,\n size: int,\n stack_num: int = 1,\n ignore_obs_next: bool = False,\n save_only_last_obs: bool = False,\n sample_avail: bool = False,\n random_seed: int = 42,\n **kwargs: Any, # otherwise PrioritizedVectorReplayBuffer will cause TypeError\n ) -> None:\n # TODO: why do we need this? Just for readout?\n self.options: dict[str, Any] = {\n \"stack_num\": stack_num,\n \"ignore_obs_next\": ignore_obs_next,\n \"save_only_last_obs\": save_only_last_obs,\n \"sample_avail\": sample_avail,\n }\n super().__init__()\n self.maxsize = int(size)\n assert stack_num > 0, \"stack_num should be greater than 0\"\n self.stack_num = stack_num\n self._indices = np.arange(size)\n # TODO: remove double negation and different name\n self._save_obs_next = not ignore_obs_next\n self._save_only_last_obs = save_only_last_obs\n self._sample_avail = sample_avail\n self._meta = cast(RolloutBatchProtocol, Batch())\n self._random_state = np.random.RandomState(random_seed)\n\n # Keep in sync with reset!\n self.last_index = np.array([0])\n self._insertion_idx = self._size = 0\n self._ep_return, self._ep_len, self._ep_start_idx = 0.0, 0, 0\n\n def __setstate__(self, state: dict[str, Any]) -> None:\n setstate(\n ReplayBuffer,\n self,\n state,\n new_default_properties={\"_random_state\": np.random.RandomState(42)},\n )\n\n @property\n def subbuffer_edges(self) -> np.ndarray:\n \"\"\"Edges of contained buffers, mostly needed as part of the VectorReplayBuffer interface.\n\n For the standard ReplayBuffer it is always [0, maxsize]. Transitions can be added\n to the buffer indefinitely, and one episode can \"go over the edge\". Having the edges\n available is useful for fishing out whole episodes from the buffer and for input validation.\n \"\"\"\n return np.array([0, self.maxsize], dtype=int)\n\n def _get_start_stop_tuples_for_edge_crossing_interval(\n self,\n start: int,\n stop: int,\n ) -> tuple[tuple[int, int], tuple[int, int]]:\n \"\"\"Assumes that stop < start and retrieves tuples corresponding to the two\n slices that determine the interval within the buffer.\n\n Example:\n -------\n >>> list(self.subbuffer_edges) == [0, 5, 10]\n >>> start = 4\n >>> stop = 2\n >>> self._get_start_stop_tuples_for_edge_crossing_interval(start, stop)\n ((4, 5), (0, 2))\n\n The buffer sliced from 4 to 5 and then from 0 to 2 will contain the transitions\n corresponding to the provided start and stop values.\n\n \"\"\"\n if stop >= start:\n raise ValueError(\n f\"Expected stop < start, but got {start=}, {stop=}. \"\n f\"For stop larger-equal than start this method should never be called. This can occur either due to an implementation error, \"\n f\"or due a bad configuration of the buffer that resulted in a single episode being so long that \"\n f\"it completely filled a subbuffer (of size len(buffer)/degree_of_vectorization). \"\n f\"Consider either shortening the episode, increasing the size of the buffer, or decreasing the \"\n f\"degree of vectorization.\",\n )\n subbuffer_edges = cast(Sequence[int], self.subbuffer_edges)\n\n edge_after_start_idx = int(np.searchsorted(subbuffer_edges, start, side=\"left\"))\n \"\"\"This is the crossed edge\"\"\"\n\n if edge_after_start_idx == 0:\n raise ValueError(\n f\"The start value should be larger than the first edge, but got {start=}, {subbuffer_edges[1]=}.\",\n )\n edge_after_start = subbuffer_edges[edge_after_start_idx]\n edge_before_stop = subbuffer_edges[edge_after_start_idx - 1]\n \"\"\"It's the edge before the crossed edge\"\"\"\n\n if edge_before_stop >= stop:\n raise ValueError(\n f\"The edge before the crossed edge should be smaller than the stop, but got {edge_before_stop=}, {stop=}.\",\n )\n return (start, edge_after_start), (edge_before_stop, stop)\n\n def get_buffer_indices(self, start: int, stop: int) -> np.ndarray:\n \"\"\"Get the indices of the transitions in the buffer between start and stop.\n\n The special thing about this is that stop may actually be smaller than start,\n since one often is interested in a sequence of transitions that goes over a subbuffer edge.\n\n The main use case for this method is to retrieve an episode from the buffer, in which case\n start is the index of the first transition in the episode and stop is the index where `done` is True + 1.\n This can be done with the following code:\n\n .. code-block:: python\n\n episode_indices = buffer.get_buffer_indices(episode_start_index, episode_done_index + 1)\n episode = buffer[episode_indices]\n\n Even when `start` is smaller than `stop`, it will be validated that they are in the same subbuffer.\n\n Example:\n --------\n >>> list(buffer.subbuffer_edges) == [0, 5, 10]\n >>> buffer.get_buffer_indices(start=2, stop=4)\n [2, 3]\n >>> buffer.get_buffer_indices(start=4, stop=2)\n [4, 0, 1]\n >>> buffer.get_buffer_indices(start=8, stop=7)\n [8, 9, 5, 6]\n >>> buffer.get_buffer_indices(start=1, stop=6)\n ValueError: Start and stop indices must be within the same subbuffer.\n >>> buffer.get_buffer_indices(start=8, stop=1)\n ValueError: Start and stop indices must be within the same subbuffer.\n\n :param start: The start index of the interval.\n :param stop: The stop index of the interval.\n :return: The indices of the transitions in the buffer between start and stop.\n\n \"\"\"\n start_left_edge = np.searchsorted(self.subbuffer_edges, start, side=\"right\") - 1\n stop_left_edge = np.searchsorted(self.subbuffer_edges, stop - 1, side=\"right\") - 1\n if start_left_edge != stop_left_edge:\n raise ValueError(\n f\"Start and stop indices must be within the same subbuffer. \"\n f\"Got {start=} in subbuffer edge {start_left_edge} and {stop=} in subbuffer edge {stop_left_edge}.\",\n )\n if stop >= start:\n return np.arange(start, stop, dtype=int)\n else:\n (\n (start, upper_edge),\n (\n lower_edge,\n stop,\n ),\n ) = self._get_start_stop_tuples_for_edge_crossing_interval(\n start,\n stop,\n )\n log.debug(f\"{start=}, {upper_edge=}, {lower_edge=}, {stop=}\")\n return np.concatenate(\n (\n np.arange(start, upper_edge, dtype=int),\n np.arange(lower_edge, stop, dtype=int),\n ),\n )\n\n def __len__(self) -> int:\n return self._size\n\n def __repr__(self) -> str:\n wrapped_batch_repr = self._meta.__repr__()[len(self._meta.__class__.__name__) :]\n return self.__class__.__name__ + wrapped_batch_repr\n\n def __getattr__(self, key: str) -> Any:\n try:\n return self._meta[key]\n except KeyError as exception:\n raise AttributeError from exception\n\n def __setattr__(self, key: str, value: Any) -> None:\n assert key not in self._reserved_keys, f\"key '{key}' is reserved and cannot be assigned\"\n super().__setattr__(key, value)\n\n def save_hdf5(self, path: str, compression: str | None = None) -> None:\n \"\"\"Save replay buffer to HDF5 file.\"\"\"\n with h5py.File(path, \"w\") as f:\n to_hdf5(self.__dict__, f, compression=compression)\n\n @classmethod\n def load_hdf5(cls, path: str, device: str | None = None) -> Self:\n \"\"\"Load replay buffer from HDF5 file.\"\"\"\n with h5py.File(path, \"r\") as f:\n buf = cls.__new__(cls)\n buf.__setstate__(from_hdf5(f, device=device)) # type: ignore\n return buf\n\n @classmethod\n def from_data(\n cls,\n obs: h5py.Dataset,\n act: h5py.Dataset,\n rew: h5py.Dataset,\n terminated: h5py.Dataset,\n truncated: h5py.Dataset,\n done: h5py.Dataset,\n obs_next: h5py.Dataset,\n ) -> Self:\n size = len(obs)\n assert all(\n len(dset) == size for dset in [obs, act, rew, terminated, truncated, done, obs_next]\n ), \"Lengths of all hdf5 datasets need to be equal.\"\n buf = cls(size)\n if size == 0:\n return buf\n batch = Batch(\n obs=obs,\n act=act,\n rew=rew,\n terminated=terminated,\n truncated=truncated,\n done=done,\n obs_next=obs_next,\n )\n batch = cast(RolloutBatchProtocol, batch)\n buf.set_batch(batch)\n buf._size = size\n return buf\n\n def reset(self, keep_statistics: bool = False) -> None:\n \"\"\"Clear all the data in replay buffer and episode statistics.\"\"\"\n # Keep in sync with init!\n self.last_index = np.array([0])\n self._insertion_idx = self._size = self._ep_start_idx = 0\n if not keep_statistics:\n self._ep_return, self._ep_len = 0.0, 0\n\n # TODO: is this method really necessary? It's kinda dangerous, can accidentally\n # remove all references to collected data\n def set_batch(self, batch: RolloutBatchProtocol) -> None:\n \"\"\"Manually choose the batch you want the ReplayBuffer to manage.\"\"\"\n assert len(batch) == self.maxsize and set(batch.get_keys()).issubset(\n self._reserved_keys,\n ), \"Input batch doesn't meet ReplayBuffer's data form requirement.\"\n self._meta = batch\n\n def unfinished_index(self) -> np.ndarray:\n \"\"\"Return the index of unfinished episode.\"\"\"\n last = (self._insertion_idx - 1) % self._size if self._size else 0\n return np.array([last] if not self.done[last] and self._size else [], int)\n\n def prev(self, index: int | np.ndarray) -> np.ndarray:\n \"\"\"Return the index of previous transition.\n\n The index won't be modified if it is the beginning of an episode.\n \"\"\"\n index = (index - 1) % self._size\n end_flag = self.done[index] | (index == self.last_index[0])\n return (index + end_flag) % self._size\n\n def next(self, index: int | np.ndarray) -> np.ndarray:\n \"\"\"Return the index of next transition.\n\n The index won't be modified if it is the end of an episode.\n \"\"\"\n end_flag = self.done[index] | (index == self.last_index[0])\n return (index + (1 - end_flag)) % self._size\n\n def update(self, buffer: \"ReplayBuffer\") -> np.ndarray:\n \"\"\"Move the data from the given buffer to current buffer.\n\n Return the updated indices. If update fails, return an empty array.\n \"\"\"\n if len(buffer) == 0 or self.maxsize == 0:\n return np.array([], int)\n stack_num, buffer.stack_num = buffer.stack_num, 1\n from_indices = buffer.sample_indices(0) # get all available indices\n buffer.stack_num = stack_num\n if len(from_indices) == 0:\n return np.array([], int)\n updated_indices = []\n for _ in range(len(from_indices)):\n updated_indices.append(self._insertion_idx)\n self.last_index[0] = self._insertion_idx\n self._insertion_idx = (self._insertion_idx + 1) % self.maxsize\n self._size = min(self._size + 1, self.maxsize)\n updated_indices = np.array(updated_indices)\n if len(self._meta.get_keys()) == 0:\n self._meta = create_value(buffer._meta, self.maxsize, stack=False) # type: ignore\n self._meta[updated_indices] = buffer._meta[from_indices]\n return updated_indices\n\n def _update_state_pre_add(\n self,\n rew: float | np.ndarray,\n done: bool,\n ) -> tuple[int, float, int, int]:\n \"\"\"Update the buffer's state before adding one data batch.\n\n Updates the `_size` and `_insertion_idx`, adds the reward and len\n internally maintained `_ep_len` and `_ep_return`. If `done` is `True`,\n will reset `_ep_len` and `_ep_return` to zero, and set `_ep_start_idx` to\n `_insertion_idx`\n\n Returns a tuple with:\n 0. the index at which to insert the next transition,\n 1. the episode len (if done=True, otherwise 0)\n 2. the episode return (if done=True, otherwise 0)\n 3. the episode start index.\n \"\"\"\n self.last_index[0] = cur_insertion_idx = self._insertion_idx\n self._size = min(self._size + 1, self.maxsize)\n self._insertion_idx = (self._insertion_idx + 1) % self.maxsize\n\n self._ep_return += rew # type: ignore\n self._ep_len += 1\n\n if self._ep_start_idx > len(self):\n raise MalformedBufferError(\n f\"Encountered a starting index {self._ep_start_idx} that is outside \"\n f\"the currently available samples {len(self)=}. \"\n f\"The buffer is malformed. This might be caused by a bug or by manual modifications of the buffer \"\n f\"by users.\",\n )\n\n # return 0 for unfinished episodes\n if done:\n ep_return = self._ep_return\n ep_len = self._ep_len\n else:\n if isinstance(self._ep_return, np.ndarray): # type: ignore[unreachable]\n # TODO: [original remark by MischaPanch] Check whether the entire else case is really correct/necessary.\n # ep_return should be a scalar but is a numpy array.\n # This doesn't make sense for a ReplayBuffer, but currently tests of CachedReplayBuffer require\n # this behavior for some reason; it also occurs in the MARL notebook, for example.\n # Will return an array of zeros instead of a scalar zero.\n pass\n ep_return = np.zeros_like(self._ep_return) # type: ignore\n ep_len = 0\n\n result = cur_insertion_idx, ep_return, ep_len, self._ep_start_idx\n\n if done:\n # prepare for next episode collection\n # set return and len to zero, set start idx to next insertion idx\n self._ep_return, self._ep_len, self._ep_start_idx = (\n 0.0,\n 0,\n self._insertion_idx,\n )\n return result\n\n def add(\n self,\n batch: RolloutBatchProtocol,\n buffer_ids: np.ndarray | list[int] | None = None,\n ) -> tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]:\n \"\"\"Add a batch of data into replay buffer.\n\n :param batch: the input data batch. \"obs\", \"act\", \"rew\",\n \"terminated\", \"truncated\" are required keys.\n :param buffer_ids: id's of subbuffers, allowed here to be consistent with classes similar to\n :class:`~tianshou.data.buffer.vecbuf.VectorReplayBuffer`. Since the `ReplayBuffer`\n has a single subbuffer, if this is not None, it must be a single element with value 0.\n In that case, the batch is expected to have the shape (1, len(data)).\n Failure to adhere to this will result in a `ValueError`.\n\n Return `(current_index, episode_return, episode_length, episode_start_index)`. If\n the episode is not finished, the return value of episode_length and\n episode_reward is 0.\n \"\"\"\n # preprocess and copy batch into a new Batch object to avoid mutating the input\n # TODO: can't we just copy? Why do we need to rely on setting inside __dict__?\n new_batch = Batch()\n for key in batch.get_keys():\n new_batch.__dict__[key] = batch[key]\n batch = new_batch\n batch.__dict__[\"done\"] = np.logical_or(batch.terminated, batch.truncated)\n\n # has to be done after preprocess batch\n if not self._required_keys_for_add.issubset(\n batch.get_keys(),\n ):\n raise ValueError(\n f\"Input batch must have the following keys: {self._required_keys_for_add}\",\n )\n\n batch_is_stacked = False\n \"\"\"True when instead of passing a batch of shape (len(data)), a batch of shape (1, len(data)) is passed.\"\"\"\n\n if buffer_ids is not None:\n if len(buffer_ids) != 1 and buffer_ids[0] != 0:\n raise ValueError(\n \"If `buffer_ids` is not None, it must be a single element with value 0 for the non-vectorized `ReplayBuffer`. \"\n f\"Got {buffer_ids=}.\",\n )\n if len(batch) != 1:\n raise ValueError(\n f\"If `buffer_ids` is not None, the batch must have the shape (1, len(data)) but got {len(batch)=}.\",\n )\n batch_is_stacked = True\n\n # block dealing with exotic options that are currently only used for atari, see various TODOs about that\n # These options have interactions with the case when buffer_ids is not None\n if self._save_only_last_obs:\n batch.obs = batch.obs[:, -1] if batch_is_stacked else batch.obs[-1]\n if not self._save_obs_next:\n batch.pop(\"obs_next\", None)\n elif self._save_only_last_obs:\n batch.obs_next = batch.obs_next[:, -1] if batch_is_stacked else batch.obs_next[-1]\n\n if batch_is_stacked:\n rew, done = batch.rew[0], batch.done[0]\n else:\n rew, done = batch.rew, batch.done\n insertion_idx, ep_return, ep_len, ep_start_idx = (\n np.array([x]) for x in self._update_state_pre_add(rew, done)\n )\n\n # TODO: improve this, don'r rely on try-except, instead process the batch if needed\n try:\n self._meta[insertion_idx] = batch\n except ValueError:\n stack = not batch_is_stacked\n batch.rew = batch.rew.astype(float)\n batch.done = batch.done.astype(bool)\n batch.terminated = batch.terminated.astype(bool)\n batch.truncated = batch.truncated.astype(bool)\n if len(self._meta.get_keys()) == 0:\n self._meta = create_value(batch, self.maxsize, stack) # type: ignore\n else: # dynamic key pops up in batch\n alloc_by_keys_diff(self._meta, batch, self.maxsize, stack)\n self._meta[insertion_idx] = batch\n return insertion_idx, ep_return, ep_len, ep_start_idx\n\n def sample_indices(self, batch_size: int | None) -> np.ndarray:\n \"\"\"Get a random sample of index with size = batch_size.\n\n Return all available indices in the buffer if batch_size is 0; return an empty\n numpy array if batch_size < 0 or no available index can be sampled.\n\n :param batch_size: the number of indices to be sampled. If None, it will be set\n to the length of the buffer (i.e. return all available indices in a\n random order).\n \"\"\"\n if batch_size is None:\n batch_size = len(self)\n if self.stack_num == 1 or not self._sample_avail: # most often case\n if batch_size > 0:\n return self._random_state.choice(self._size, batch_size)\n # TODO: is this behavior really desired?\n if batch_size == 0: # construct current available indices\n return np.concatenate(\n [\n np.arange(self._insertion_idx, self._size),\n np.arange(self._insertion_idx),\n ],\n )\n return np.array([], int)\n # TODO: raise error on negative batch_size instead?\n if batch_size < 0:\n return np.array([], int)\n # TODO: simplify this code - shouldn't have such a large if-else\n # with many returns for handling different stack nums.\n # It is also not clear whether this is really necessary - frame stacking usually is handled\n # by environment wrappers (e.g. FrameStack) and not by the replay buffer.\n all_indices = prev_indices = np.concatenate(\n [\n np.arange(self._insertion_idx, self._size),\n np.arange(self._insertion_idx),\n ],\n )\n for _ in range(self.stack_num - 2):\n prev_indices = self.prev(prev_indices)\n all_indices = all_indices[prev_indices != self.prev(prev_indices)]\n if batch_size > 0:\n return self._random_state.choice(all_indices, batch_size)\n return all_indices\n\n def sample(self, batch_size: int | None) -> tuple[RolloutBatchProtocol, np.ndarray]:\n \"\"\"Get a random sample from buffer with size = batch_size.\n\n Return all the data in the buffer if batch_size is 0.\n\n :return: Sample data and its corresponding index inside the buffer.\n \"\"\"\n indices = self.sample_indices(batch_size)\n return self[indices], indices\n\n def get(\n self,\n index: int | list[int] | np.ndarray,\n key: str,\n default_value: Any = None,\n # TODO 1: this is only here because of atari, it should never be needed (can be solved with index)\n # and should be removed\n # TODO 2: does something entirely different from getitem\n # TODO 3: key should not be required\n stack_num: int | None = None,\n ) -> Batch | np.ndarray:\n \"\"\"Return the stacked result.\n\n E.g., if you set ``key = \"obs\", stack_num = 4, index = t``, it returns the\n stacked result as ``[obs[t-3], obs[t-2], obs[t-1], obs[t]]``.\n\n :param index: the index for getting stacked data.\n :param key: the key to get, should be one of the reserved_keys.\n :param default_value: if the given key's data is not found and default_value is\n set, return this default_value.\n :param stack_num: Default to self.stack_num.\n \"\"\"\n if key not in self._meta.get_keys() and default_value is not None:\n return default_value\n val = self._meta[key]\n if stack_num is None:\n stack_num = self.stack_num\n try:\n if stack_num == 1: # the most common case\n return val[index]\n\n stack = list[Any]()\n indices = np.array(index) if isinstance(index, list) else index\n # NOTE: stack_num > 1, so the range is not empty and indices is turned into\n # np.ndarray by self.prev\n for _ in range(stack_num):\n stack = [val[indices], *stack]\n indices = self.prev(indices)\n indices = cast(np.ndarray, indices)\n if isinstance(val, Batch):\n return Batch.stack(stack, axis=indices.ndim)\n return np.stack(stack, axis=indices.ndim)\n\n except IndexError as exception:\n if not (isinstance(val, Batch) and len(val.keys()) == 0):\n raise exception # val != Batch()\n return Batch()\n\n def __getitem__(self, index: IndexType) -> RolloutBatchProtocol:\n \"\"\"Return a data batch: self[index].\n\n If stack_num is larger than 1, return the stacked obs and obs_next with shape\n (batch, len, ...).\n \"\"\"\n # TODO: this is a seriously problematic hack leading to\n # buffer[slice] != buffer[np.arange(slice.start, slice.stop)]\n # Fix asap, high priority!!!\n if isinstance(index, slice): # change slice to np array\n # buffer[:] will get all available data\n indices = (\n self.sample_indices(0)\n if index == slice(None)\n else self._indices[: len(self)][index]\n )\n else:\n indices = index # type: ignore\n # raise KeyError first instead of AttributeError,\n # to support np.array([ReplayBuffer()])\n obs = self.get(indices, \"obs\")\n if self._save_obs_next:\n obs_next = self.get(indices, \"obs_next\", Batch())\n else:\n obs_next_indices = self.next(indices)\n obs_next = self.get(obs_next_indices, \"obs\", Batch())\n # TODO: don't do this\n batch_dict = {\n \"obs\": obs,\n \"act\": self.act[indices],\n \"rew\": self.rew[indices],\n \"terminated\": self.terminated[indices],\n \"truncated\": self.truncated[indices],\n \"done\": self.done[indices],\n \"obs_next\": obs_next,\n \"info\": self.get(indices, \"info\", Batch()),\n # TODO: what's the use of this key?\n \"policy\": self.get(indices, \"policy\", Batch()),\n }\n # TODO: don't do this, reduce complexity. Why such a big difference between what is returned\n # and sub-batches of self._meta?\n missing_keys = set(self._meta.get_keys()) - set(self._input_keys)\n for key in missing_keys:\n batch_dict[key] = self._meta[key][indices]\n return cast(RolloutBatchProtocol, Batch(batch_dict))\n\n def set_array_at_key(\n self,\n seq: np.ndarray,\n key: str,\n index: IndexType | None = None,\n default_value: float | None = None,\n ) -> None:\n self._meta.set_array_at_key(seq, key, index, default_value)\n\n def hasnull(self) -> bool:\n return self[:].hasnull()\n\n def isnull(self) -> RolloutBatchProtocol:\n return self[:].isnull()\n\n def dropnull(self) -> None:\n # TODO: may fail, needs more testing with VectorBuffers\n self._meta = self._meta.dropnull()\n self._size = len(self._meta)\n self._insertion_idx = len(self._meta)\n" }, { "path": "tianshou/data/buffer/cached.py", "content": "import numpy as np\n\nfrom tianshou.data import ReplayBuffer, ReplayBufferManager\nfrom tianshou.data.types import RolloutBatchProtocol\n\n\nclass CachedReplayBuffer(ReplayBufferManager):\n \"\"\"CachedReplayBuffer contains a given main buffer and n cached buffers, ``cached_buffer_num * ReplayBuffer(size=max_episode_length)``.\n\n The memory layout is: ``| main_buffer | cached_buffers[0] | cached_buffers[1] | ...\n | cached_buffers[cached_buffer_num - 1] |``.\n\n The data is first stored in cached buffers. When an episode is terminated, the data\n will move to the main buffer and the corresponding cached buffer will be reset.\n\n :param main_buffer: the main buffer whose ``.update()`` function\n behaves normally.\n :param cached_buffer_num: number of ReplayBuffer needs to be created for cached\n buffer.\n :param max_episode_length: the maximum length of one episode, used in each\n cached buffer's maxsize.\n\n .. seealso::\n\n Please refer to :class:`~tianshou.data.ReplayBuffer` for other APIs' usage.\n \"\"\"\n\n def __init__(\n self,\n main_buffer: ReplayBuffer,\n cached_buffer_num: int,\n max_episode_length: int,\n ) -> None:\n assert cached_buffer_num > 0\n assert max_episode_length > 0\n assert isinstance(main_buffer, ReplayBuffer)\n kwargs = main_buffer.options\n buffers = [main_buffer] + [\n ReplayBuffer(max_episode_length, **kwargs) for _ in range(cached_buffer_num)\n ]\n super().__init__(buffer_list=buffers)\n self.main_buffer = self.buffers[0]\n self.cached_buffers = self.buffers[1:]\n self.cached_buffer_num = cached_buffer_num\n\n def add(\n self,\n batch: RolloutBatchProtocol,\n buffer_ids: np.ndarray | list[int] | None = None,\n ) -> tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]:\n \"\"\"Add a batch of data into CachedReplayBuffer.\n\n Each of the data's length (first dimension) must equal to the length of\n buffer_ids. By default the buffer_ids is [0, 1, ..., cached_buffer_num - 1].\n\n Return (current_index, episode_reward, episode_length, episode_start_index)\n with each of the shape (len(buffer_ids), ...), where (current_index[i],\n episode_reward[i], episode_length[i], episode_start_index[i]) refers to the\n cached_buffer_ids[i]th cached buffer's corresponding episode result.\n \"\"\"\n if buffer_ids is None:\n cached_buffer_ids = np.arange(1, 1 + self.cached_buffer_num)\n else: # make sure it is np.ndarray, +1 means it's never the main buffer\n cached_buffer_ids = np.asarray(buffer_ids) + 1\n insertion_idx, ep_return, ep_len, ep_start_idx = super().add(\n batch,\n buffer_ids=cached_buffer_ids,\n )\n # find the terminated episode, move data from cached buf to main buf\n updated_insertion_idx, updated_ep_start_idx = [], []\n done = np.logical_or(batch.terminated, batch.truncated)\n for buffer_idx in cached_buffer_ids[done]:\n index = self.main_buffer.update(self.buffers[buffer_idx])\n if len(index) == 0: # unsuccessful move, replace with -1\n index = [-1]\n updated_ep_start_idx.append(index[0])\n updated_insertion_idx.append(index[-1])\n self.buffers[buffer_idx].reset()\n self._lengths[0] = len(self.main_buffer)\n self._lengths[buffer_idx] = 0\n self.last_index[0] = index[-1]\n self.last_index[buffer_idx] = self._offset[buffer_idx]\n insertion_idx[done] = updated_insertion_idx\n ep_start_idx[done] = updated_ep_start_idx\n return insertion_idx, ep_return, ep_len, ep_start_idx\n" }, { "path": "tianshou/data/buffer/her.py", "content": "from collections.abc import Callable\nfrom typing import Any, Union, cast\n\nimport numpy as np\n\nfrom tianshou.data import Batch, ReplayBuffer\nfrom tianshou.data.batch import BatchProtocol\nfrom tianshou.data.types import RolloutBatchProtocol\n\n\nclass HERReplayBuffer(ReplayBuffer):\n \"\"\"Implementation of Hindsight Experience Replay. arXiv:1707.01495.\n\n HERReplayBuffer is to be used with goal-based environment where the\n observation is a dictionary with keys ``observation``, ``achieved_goal`` and\n ``desired_goal``. Currently support only HER's future strategy, online sampling.\n\n :param size: the size of the replay buffer.\n :param compute_reward_fn: a function that takes 2 ``np.array`` arguments,\n ``acheived_goal`` and ``desired_goal``, and returns rewards as ``np.array``.\n The two arguments are of shape (batch_size, ...original_shape) and the returned\n rewards must be of shape (batch_size,).\n :param horizon: the maximum number of steps in an episode.\n :param future_k: the 'k' parameter introduced in the paper. In short, there\n will be at most k episodes that are re-written for every 1 unaltered episode\n during the sampling.\n\n .. seealso::\n\n Please refer to :class:`~tianshou.data.ReplayBuffer` for other APIs' usage.\n \"\"\"\n\n def __init__(\n self,\n size: int,\n compute_reward_fn: Callable[[np.ndarray, np.ndarray], np.ndarray],\n horizon: int,\n future_k: float = 8.0,\n **kwargs: Any,\n ) -> None:\n super().__init__(size, **kwargs)\n self.horizon = horizon\n self.future_p = 1 - 1 / future_k\n self.compute_reward_fn = compute_reward_fn\n self._original_meta = Batch()\n self._altered_indices = np.array([])\n\n def _restore_cache(self) -> None:\n \"\"\"Write cached original meta back to `self._meta`.\n\n It's called everytime before 'writing', 'sampling' or 'saving' the buffer.\n \"\"\"\n if not hasattr(self, \"_altered_indices\"):\n return\n\n if self._altered_indices.size == 0:\n return\n self._meta[self._altered_indices] = self._original_meta\n # Clean\n self._original_meta = Batch()\n self._altered_indices = np.array([])\n\n def reset(self, keep_statistics: bool = False) -> None:\n self._restore_cache()\n return super().reset(keep_statistics)\n\n def save_hdf5(self, path: str, compression: str | None = None) -> None:\n self._restore_cache()\n return super().save_hdf5(path, compression)\n\n def set_batch(self, batch: RolloutBatchProtocol) -> None:\n self._restore_cache()\n return super().set_batch(batch)\n\n def update(self, buffer: Union[\"HERReplayBuffer\", \"ReplayBuffer\"]) -> np.ndarray:\n self._restore_cache()\n return super().update(buffer)\n\n def add(\n self,\n batch: RolloutBatchProtocol,\n buffer_ids: np.ndarray | list[int] | None = None,\n ) -> tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]:\n self._restore_cache()\n return super().add(batch, buffer_ids)\n\n def sample_indices(self, batch_size: int | None) -> np.ndarray:\n \"\"\"Get a random sample of index with size = batch_size.\n\n Return all available indices in the buffer if batch_size is 0; return an \\\n empty numpy array if batch_size < 0 or no available index can be sampled. \\\n Additionally, some episodes of the sampled transitions will be re-written \\\n according to HER.\n \"\"\"\n self._restore_cache()\n indices = super().sample_indices(batch_size=batch_size)\n self.rewrite_transitions(indices.copy())\n return indices\n\n def rewrite_transitions(self, indices: np.ndarray) -> None:\n \"\"\"Re-write the goal of some sampled transitions' episodes according to HER.\n\n Currently applies only HER's 'future' strategy. The new goals will be written \\\n directly to the internal batch data temporarily and will be restored right \\\n before the next sampling or when using some of the buffer's method (e.g. \\\n `add`, `save_hdf5`, etc.). This is to make sure that n-step returns \\\n calculation etc., performs correctly without additional alteration.\n \"\"\"\n if indices.size == 0:\n return\n\n # Sort indices keeping chronological order\n indices[indices < self._insertion_idx] += self.maxsize\n indices = np.sort(indices)\n indices[indices >= self.maxsize] -= self.maxsize\n\n # Construct episode trajectories\n indices = [indices]\n for _ in range(self.horizon - 1):\n indices.append(self.next(indices[-1]))\n indices = np.stack(indices)\n\n # Calculate future timestep to use\n current = indices[0]\n terminal = indices[-1]\n episodes_len = (terminal - current + self.maxsize) % self.maxsize\n future_offset = np.random.uniform(size=len(indices[0])) * episodes_len\n future_offset = np.round(future_offset).astype(int)\n future_t = (current + future_offset) % self.maxsize\n\n # Compute indices\n # open indices are used to find longest, unique trajectories among\n # presented episodes\n unique_ep_open_indices = np.sort(np.unique(terminal, return_index=True)[1])\n unique_ep_indices = indices[:, unique_ep_open_indices]\n # close indices are used to find max future_t among presented episodes\n unique_ep_close_indices = np.hstack([(unique_ep_open_indices - 1)[1:], len(terminal) - 1])\n # episode indices that will be altered\n her_ep_indices = np.random.choice(\n len(unique_ep_open_indices),\n size=int(len(unique_ep_open_indices) * self.future_p),\n replace=False,\n )\n\n # Cache original meta\n self._altered_indices = unique_ep_indices.copy()\n self._original_meta = self._meta[self._altered_indices].copy()\n\n # Copy original obs, ep_rew (and obs_next), and obs of future time step\n ep_obs = self[unique_ep_indices].obs\n # to satisfy mypy\n # TODO: add protocol covering these batches\n assert isinstance(ep_obs, Batch)\n ep_rew = self[unique_ep_indices].rew\n if self._save_obs_next:\n ep_obs_next = self[unique_ep_indices].obs_next\n # to satisfy mypy\n assert isinstance(ep_obs_next, Batch)\n future_obs = self[future_t[unique_ep_close_indices]].obs_next\n else:\n future_obs = self[self.next(future_t[unique_ep_close_indices])].obs\n future_obs = cast(BatchProtocol, future_obs)\n\n # Re-assign goals and rewards via broadcast assignment\n ep_obs.desired_goal[:, her_ep_indices] = future_obs.achieved_goal[None, her_ep_indices]\n if self._save_obs_next:\n ep_obs_next = cast(BatchProtocol, ep_obs_next)\n ep_obs_next.desired_goal[:, her_ep_indices] = future_obs.achieved_goal[\n None,\n her_ep_indices,\n ]\n ep_rew[:, her_ep_indices] = self._compute_reward(ep_obs_next)[:, her_ep_indices]\n else:\n tmp_ep_obs_next = self[self.next(unique_ep_indices)].obs\n assert isinstance(tmp_ep_obs_next, Batch)\n ep_rew[:, her_ep_indices] = self._compute_reward(tmp_ep_obs_next)[:, her_ep_indices]\n\n # Sanity check\n assert ep_obs.desired_goal.shape[:2] == unique_ep_indices.shape\n assert ep_obs.achieved_goal.shape[:2] == unique_ep_indices.shape\n assert ep_rew.shape == unique_ep_indices.shape\n\n # Re-write meta\n assert isinstance(self._meta.obs, Batch)\n self._meta.obs[unique_ep_indices] = ep_obs\n if self._save_obs_next:\n self._meta.obs_next[unique_ep_indices] = ep_obs_next # type: ignore\n self._meta.rew[unique_ep_indices] = ep_rew.astype(np.float32)\n\n def _compute_reward(self, obs: BatchProtocol, lead_dims: int = 2) -> np.ndarray:\n lead_shape = obs.observation.shape[:lead_dims]\n g = obs.desired_goal.reshape(-1, *obs.desired_goal.shape[lead_dims:])\n ag = obs.achieved_goal.reshape(-1, *obs.achieved_goal.shape[lead_dims:])\n rewards = self.compute_reward_fn(ag, g)\n return rewards.reshape(*lead_shape, *rewards.shape[1:])\n" }, { "path": "tianshou/data/buffer/manager.py", "content": "from collections.abc import Sequence\nfrom typing import Union, cast\n\nimport numpy as np\nfrom numba import njit\nfrom overrides import override\n\nfrom tianshou.data import Batch, HERReplayBuffer, PrioritizedReplayBuffer, ReplayBuffer\nfrom tianshou.data.batch import alloc_by_keys_diff, create_value\nfrom tianshou.data.types import RolloutBatchProtocol\n\n\nclass ReplayBufferManager(ReplayBuffer):\n \"\"\"ReplayBufferManager contains a list of ReplayBuffer with exactly the same configuration.\n\n These replay buffers have contiguous memory layout, and the storage space each\n buffer has is a shallow copy of the topmost memory.\n\n :param buffer_list: a list of ReplayBuffer needed to be handled.\n\n .. seealso::\n\n Please refer to :class:`~tianshou.data.ReplayBuffer` for other APIs' usage.\n \"\"\"\n\n def __init__(self, buffer_list: list[ReplayBuffer] | list[HERReplayBuffer]) -> None:\n self.buffer_num = len(buffer_list)\n self.buffers = np.array(buffer_list, dtype=object)\n last_index: list[int] = []\n offset, size = [], 0\n buffer_type = type(self.buffers[0])\n kwargs = self.buffers[0].options\n for buf in self.buffers:\n buf = cast(ReplayBuffer, buf)\n assert len(buf._meta.get_keys()) == 0\n assert isinstance(buf, buffer_type)\n assert buf.options == kwargs\n offset.append(size)\n if len(buf.last_index) != 1:\n raise ValueError(\n f\"{self.__class__.__name__} only supports buffers with a single index \"\n f\"(non-vector buffers), but got {last_index=}. \"\n f\"Did you try to use a {self.__class__.__name__} within a {self.__class__.__name__}?\",\n )\n last_index.append(size + buf.last_index[0])\n size += buf.maxsize\n super().__init__(size=size, **kwargs)\n self._offset = np.array(offset)\n self._extend_offset = np.array([*offset, size])\n self._lengths = np.zeros_like(offset)\n self.last_index = np.array(last_index)\n self._compile()\n self._meta: RolloutBatchProtocol\n\n @property\n @override\n def subbuffer_edges(self) -> np.ndarray:\n return self._extend_offset\n\n def _compile(self) -> None:\n lens = last = index = np.array([0])\n offset = np.array([0, 1])\n done = np.array([False, False])\n _prev_index(index, offset, done, last, lens)\n _next_index(index, offset, done, last, lens)\n\n def __len__(self) -> int:\n return int(self._lengths.sum())\n\n def reset(self, keep_statistics: bool = False) -> None:\n # keep in sync with init!\n self.last_index = self._offset.copy()\n self._lengths = np.zeros_like(self._offset)\n for buf in self.buffers:\n buf.reset(keep_statistics=keep_statistics)\n\n def _set_batch_for_children(self) -> None:\n for offset, buf in zip(self._offset, self.buffers, strict=True):\n buf.set_batch(self._meta[offset : offset + buf.maxsize])\n\n def set_batch(self, batch: RolloutBatchProtocol) -> None:\n super().set_batch(batch)\n self._set_batch_for_children()\n\n def unfinished_index(self) -> np.ndarray:\n return np.concatenate(\n [\n buf.unfinished_index() + offset\n for offset, buf in zip(self._offset, self.buffers, strict=True)\n ],\n )\n\n def prev(self, index: int | np.ndarray) -> np.ndarray:\n if isinstance(index, list | np.ndarray):\n return _prev_index(\n np.asarray(index),\n self._extend_offset,\n self.done,\n self.last_index,\n self._lengths,\n )\n return _prev_index(\n np.array([index]),\n self._extend_offset,\n self.done,\n self.last_index,\n self._lengths,\n )[0]\n\n def next(self, index: int | np.ndarray) -> np.ndarray:\n if isinstance(index, list | np.ndarray):\n return _next_index(\n np.asarray(index),\n self._extend_offset,\n self.done,\n self.last_index,\n self._lengths,\n )\n return _next_index(\n np.array([index]),\n self._extend_offset,\n self.done,\n self.last_index,\n self._lengths,\n )[0]\n\n def update(self, buffer: ReplayBuffer) -> np.ndarray:\n \"\"\"The ReplayBufferManager cannot be updated by any buffer.\"\"\"\n raise NotImplementedError\n\n def add(\n self,\n batch: RolloutBatchProtocol,\n buffer_ids: np.ndarray | list[int] | None = None,\n ) -> tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]:\n \"\"\"Add a batch of data into ReplayBufferManager.\n\n Each of the data's length (first dimension) must equal to the length of\n buffer_ids. By default buffer_ids is [0, 1, ..., buffer_num - 1].\n\n Return (current_index, episode_reward, episode_length, episode_start_index). If\n the episode is not finished, the return value of episode_length and\n episode_reward is 0.\n \"\"\"\n # preprocess batch\n new_batch = Batch()\n for key in set(self._reserved_keys).intersection(batch.get_keys()):\n new_batch.__dict__[key] = batch[key]\n batch = new_batch\n batch.__dict__[\"done\"] = np.logical_or(batch.terminated, batch.truncated)\n assert {\"obs\", \"act\", \"rew\", \"terminated\", \"truncated\", \"done\"}.issubset(batch.get_keys())\n if self._save_only_last_obs:\n batch.obs = batch.obs[:, -1]\n if not self._save_obs_next:\n batch.pop(\"obs_next\", None)\n elif self._save_only_last_obs:\n batch.obs_next = batch.obs_next[:, -1]\n # get index\n if buffer_ids is None:\n buffer_ids = np.arange(self.buffer_num)\n insertion_indxS, ep_lens, ep_returns, ep_idxs = [], [], [], []\n for batch_idx, buffer_id in enumerate(buffer_ids):\n # TODO: don't access private method!\n insertion_index, ep_return, ep_len, ep_start_idx = self.buffers[\n buffer_id\n ]._update_state_pre_add(\n batch.rew[batch_idx],\n batch.done[batch_idx],\n )\n offset_insertion_idx = insertion_index + self._offset[buffer_id]\n offset_ep_start_idx = ep_start_idx + self._offset[buffer_id]\n insertion_indxS.append(offset_insertion_idx)\n ep_lens.append(ep_len)\n ep_returns.append(ep_return)\n ep_idxs.append(offset_ep_start_idx)\n self.last_index[buffer_id] = insertion_index + self._offset[buffer_id]\n self._lengths[buffer_id] = len(self.buffers[buffer_id])\n insertion_indxS = np.array(insertion_indxS)\n try:\n self._meta[insertion_indxS] = batch\n # TODO: don't do this!\n except ValueError:\n batch.rew = batch.rew.astype(float)\n batch.done = batch.done.astype(bool)\n batch.terminated = batch.terminated.astype(bool)\n batch.truncated = batch.truncated.astype(bool)\n if len(self._meta.get_keys()) == 0:\n self._meta = create_value(batch, self.maxsize, stack=False) # type: ignore\n else: # dynamic key pops up in batch\n alloc_by_keys_diff(self._meta, batch, self.maxsize, False)\n self._set_batch_for_children()\n self._meta[insertion_indxS] = batch\n return (\n insertion_indxS,\n np.array(ep_returns),\n np.array(ep_lens),\n np.array(ep_idxs),\n )\n\n def sample_indices(self, batch_size: int | None) -> np.ndarray:\n # TODO: simplify this code\n if batch_size is not None and batch_size < 0:\n # TODO: raise error instead?\n return np.array([], int)\n if self._sample_avail and self.stack_num > 1:\n all_indices = np.concatenate(\n [\n buf.sample_indices(0) + offset\n for offset, buf in zip(self._offset, self.buffers, strict=True)\n ],\n )\n if batch_size == 0:\n return all_indices\n if batch_size is None:\n batch_size = len(all_indices)\n return self._random_state.choice(all_indices, batch_size)\n if batch_size == 0 or batch_size is None: # get all available indices\n sample_num = np.zeros(self.buffer_num, int)\n else:\n buffer_idx = self._random_state.choice(\n self.buffer_num,\n batch_size,\n p=self._lengths / self._lengths.sum(),\n )\n sample_num = np.bincount(buffer_idx, minlength=self.buffer_num)\n # avoid batch_size > 0 and sample_num == 0 -> get child's all data\n sample_num[sample_num == 0] = -1\n\n return np.concatenate(\n [\n buf.sample_indices(int(bsz)) + offset\n for offset, buf, bsz in zip(self._offset, self.buffers, sample_num, strict=True)\n ],\n )\n\n\n# TODO: unintuitively, the order of inheritance has to stay this way for tests to pass\n# As also described in the todo below, this is a bad design and should be refactored\nclass PrioritizedReplayBufferManager(PrioritizedReplayBuffer, ReplayBufferManager):\n \"\"\"PrioritizedReplayBufferManager contains a list of PrioritizedReplayBuffer with exactly the same configuration.\n\n These replay buffers have contiguous memory layout, and the storage space each\n buffer has is a shallow copy of the topmost memory.\n\n :param buffer_list: a list of PrioritizedReplayBuffer needed to be handled.\n\n .. seealso::\n\n Please refer to :class:`~tianshou.data.ReplayBuffer` for other APIs' usage.\n \"\"\"\n\n def __init__(self, buffer_list: Sequence[PrioritizedReplayBuffer]) -> None:\n ReplayBufferManager.__init__(self, buffer_list) # type: ignore\n # last_index = copy(self.last_index)\n kwargs = buffer_list[0].options\n last_index_from_buffer_manager = self.last_index\n\n for buf in buffer_list:\n del buf.weight\n PrioritizedReplayBuffer.__init__(self, self.maxsize, **kwargs)\n\n # TODO: the line below is needed since we now set the last_index of the manager in init\n # (previously it was only set in reset), and it clashes with multiple inheritance\n # Initializing the ReplayBufferManager after the PrioritizedReplayBuffer would be a better solution,\n # but it currently leads to infinite recursion. This kind of multiple inheritance with overlapping\n # interfaces is evil and we should get rid of it\n self.last_index = last_index_from_buffer_manager\n\n\nclass HERReplayBufferManager(ReplayBufferManager):\n \"\"\"HERReplayBufferManager contains a list of HERReplayBuffer with exactly the same configuration.\n\n These replay buffers have contiguous memory layout, and the storage space each\n buffer has is a shallow copy of the topmost memory.\n\n :param buffer_list: a list of HERReplayBuffer needed to be handled.\n\n .. seealso::\n\n Please refer to :class:`~tianshou.data.ReplayBuffer` for other APIs' usage.\n \"\"\"\n\n def __init__(self, buffer_list: list[HERReplayBuffer]) -> None:\n super().__init__(buffer_list)\n\n def _restore_cache(self) -> None:\n for buf in self.buffers:\n buf._restore_cache()\n\n def save_hdf5(self, path: str, compression: str | None = None) -> None:\n self._restore_cache()\n return super().save_hdf5(path, compression)\n\n def set_batch(self, batch: RolloutBatchProtocol) -> None:\n self._restore_cache()\n return super().set_batch(batch)\n\n def update(self, buffer: Union[\"HERReplayBuffer\", \"ReplayBuffer\"]) -> np.ndarray:\n self._restore_cache()\n return super().update(buffer)\n\n def add(\n self,\n batch: RolloutBatchProtocol,\n buffer_ids: np.ndarray | list[int] | None = None,\n ) -> tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]:\n self._restore_cache()\n return super().add(batch, buffer_ids)\n\n\n@njit\ndef _prev_index(\n index: np.ndarray,\n offset: np.ndarray,\n done: np.ndarray,\n last_index: np.ndarray,\n lengths: np.ndarray,\n) -> np.ndarray:\n index = index % offset[-1]\n prev_index = np.zeros_like(index)\n # disable B905 until strict=True in zip is implemented in numba\n # https://github.com/numba/numba/issues/8943\n for start, end, cur_len, last in zip( # noqa: B905\n offset[:-1],\n offset[1:],\n lengths,\n last_index,\n ):\n mask = (start <= index) & (index < end)\n correct_cur_len = max(1, cur_len)\n if np.sum(mask) > 0:\n subind = index[mask]\n subind = (subind - start - 1) % correct_cur_len\n end_flag = done[subind + start] | (subind + start == last)\n prev_index[mask] = (subind + end_flag) % correct_cur_len + start\n return prev_index\n\n\n@njit\ndef _next_index(\n index: np.ndarray,\n offset: np.ndarray,\n done: np.ndarray,\n last_index: np.ndarray,\n lengths: np.ndarray,\n) -> np.ndarray:\n index = index % offset[-1]\n next_index = np.zeros_like(index)\n # disable B905 until strict=True in zip is implemented in numba\n # https://github.com/numba/numba/issues/8943\n for start, end, cur_len, last in zip( # noqa: B905\n offset[:-1],\n offset[1:],\n lengths,\n last_index,\n ):\n mask = (start <= index) & (index < end)\n correct_cur_len = max(1, cur_len)\n if np.sum(mask) > 0:\n subind = index[mask]\n end_flag = done[subind] | (subind == last)\n next_index[mask] = (subind - start + 1 - end_flag) % correct_cur_len + start\n return next_index\n" }, { "path": "tianshou/data/buffer/prio.py", "content": "from collections.abc import Sequence\nfrom typing import Any, cast\n\nimport numpy as np\nimport torch\n\nfrom tianshou.data import ReplayBuffer, SegmentTree, to_numpy\nfrom tianshou.data.batch import IndexType\nfrom tianshou.data.types import PrioBatchProtocol, RolloutBatchProtocol\n\n\nclass PrioritizedReplayBuffer(ReplayBuffer):\n \"\"\"Implementation of Prioritized Experience Replay. arXiv:1511.05952.\n\n :param alpha: the prioritization exponent.\n :param beta: the importance sample soft coefficient.\n :param weight_norm: whether to normalize returned weights with the maximum\n weight value within the batch. Default to True.\n\n .. seealso::\n\n Please refer to :class:`~tianshou.data.ReplayBuffer` for other APIs' usage.\n \"\"\"\n\n def __init__(\n self,\n size: int,\n alpha: float,\n beta: float,\n weight_norm: bool = True,\n **kwargs: Any,\n ) -> None:\n # will raise KeyError in PrioritizedVectorReplayBuffer\n # super().__init__(size, **kwargs)\n ReplayBuffer.__init__(self, size, **kwargs)\n assert alpha > 0.0\n assert beta >= 0.0\n self._alpha, self._beta = alpha, beta\n self._max_prio = self._min_prio = 1.0\n # save weight directly in this class instead of self._meta\n self.weight = SegmentTree(size)\n self.__eps = np.finfo(np.float32).eps.item()\n self.options.update(alpha=alpha, beta=beta)\n self._weight_norm = weight_norm\n\n def init_weight(self, index: int | np.ndarray) -> None:\n self.weight[index] = self._max_prio**self._alpha\n\n def update(self, buffer: ReplayBuffer) -> np.ndarray:\n indices = super().update(buffer)\n self.init_weight(indices)\n return indices\n\n def add(\n self,\n batch: RolloutBatchProtocol,\n buffer_ids: np.ndarray | list[int] | None = None,\n ) -> tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]:\n ptr, ep_rew, ep_len, ep_idx = super().add(batch, buffer_ids)\n self.init_weight(ptr)\n return ptr, ep_rew, ep_len, ep_idx\n\n def sample_indices(self, batch_size: int | None) -> np.ndarray:\n if batch_size is not None and batch_size > 0 and len(self) > 0:\n scalar = np.random.rand(batch_size) * self.weight.reduce()\n return self.weight.get_prefix_sum_idx(scalar) # type: ignore\n return super().sample_indices(batch_size)\n\n def get_weight(self, index: int | np.ndarray) -> float | np.ndarray:\n \"\"\"Get the importance sampling weight.\n\n The \"weight\" in the returned Batch is the weight on loss function to debias\n the sampling process (some transition tuples are sampled more often so their\n losses are weighted less).\n \"\"\"\n # important sampling weight calculation\n # original formula: ((p_j/p_sum*N)**(-beta))/((p_min/p_sum*N)**(-beta))\n # simplified formula: (p_j/p_min)**(-beta)\n return (self.weight[index] / self._min_prio) ** (-self._beta)\n\n def update_weight(self, index: np.ndarray, new_weight: np.ndarray | torch.Tensor) -> None:\n \"\"\"Update priority weight by index in this buffer.\n\n :param np.ndarray index: index you want to update weight.\n :param np.ndarray new_weight: new priority weight you want to update.\n \"\"\"\n weight = np.abs(to_numpy(new_weight)) + self.__eps\n self.weight[index] = weight**self._alpha\n self._max_prio = max(self._max_prio, weight.max())\n self._min_prio = min(self._min_prio, weight.min())\n\n def __getitem__(self, index: IndexType) -> PrioBatchProtocol:\n indices: Sequence[int] | np.ndarray\n if isinstance(index, slice): # change slice to np array\n # buffer[:] will get all available data\n indices = (\n self.sample_indices(0)\n if index == slice(None)\n else self._indices[: len(self)][index]\n )\n else:\n indices = cast(np.ndarray, index)\n batch = super().__getitem__(indices)\n weight = self.get_weight(indices)\n # ref: https://github.com/Kaixhin/Rainbow/blob/master/memory.py L154\n batch.weight = weight / np.max(weight) if self._weight_norm else weight\n return cast(PrioBatchProtocol, batch)\n\n def sample(self, batch_size: int | None) -> tuple[PrioBatchProtocol, np.ndarray]:\n return cast(tuple[PrioBatchProtocol, np.ndarray], super().sample(batch_size=batch_size))\n\n def set_beta(self, beta: float) -> None:\n self._beta = beta\n" }, { "path": "tianshou/data/buffer/vecbuf.py", "content": "from typing import Any\n\nimport numpy as np\n\nfrom tianshou.data import (\n HERReplayBuffer,\n HERReplayBufferManager,\n PrioritizedReplayBuffer,\n PrioritizedReplayBufferManager,\n ReplayBuffer,\n ReplayBufferManager,\n)\n\n\nclass VectorReplayBuffer(ReplayBufferManager):\n \"\"\"VectorReplayBuffer contains n ReplayBuffer with the same size.\n\n It is used for storing transition from different environments yet keeping the order\n of time.\n\n :param total_size: the total size of VectorReplayBuffer.\n :param buffer_num: the number of ReplayBuffer it uses, which are under the same\n configuration.\n\n Other input arguments (stack_num/ignore_obs_next/save_only_last_obs/sample_avail)\n are the same as :class:`~tianshou.data.ReplayBuffer`.\n\n .. seealso::\n\n Please refer to :class:`~tianshou.data.ReplayBuffer` for other APIs' usage.\n \"\"\"\n\n def __init__(self, total_size: int, buffer_num: int, **kwargs: Any) -> None:\n assert buffer_num > 0\n size = int(np.ceil(total_size / buffer_num))\n buffer_list = [ReplayBuffer(size, **kwargs) for _ in range(buffer_num)]\n super().__init__(buffer_list)\n\n\nclass PrioritizedVectorReplayBuffer(PrioritizedReplayBufferManager):\n \"\"\"PrioritizedVectorReplayBuffer contains n PrioritizedReplayBuffer with same size.\n\n It is used for storing transition from different environments yet keeping the order\n of time.\n\n :param total_size: the total size of PrioritizedVectorReplayBuffer.\n :param buffer_num: the number of PrioritizedReplayBuffer it uses, which are\n under the same configuration.\n\n Other input arguments (alpha/beta/stack_num/ignore_obs_next/save_only_last_obs/\n sample_avail) are the same as :class:`~tianshou.data.PrioritizedReplayBuffer`.\n\n .. seealso::\n\n Please refer to :class:`~tianshou.data.ReplayBuffer` for other APIs' usage.\n \"\"\"\n\n def __init__(self, total_size: int, buffer_num: int, **kwargs: Any) -> None:\n assert buffer_num > 0\n size = int(np.ceil(total_size / buffer_num))\n buffer_list = [PrioritizedReplayBuffer(size, **kwargs) for _ in range(buffer_num)]\n super().__init__(buffer_list)\n\n def set_beta(self, beta: float) -> None:\n for buffer in self.buffers:\n buffer.set_beta(beta)\n\n\nclass HERVectorReplayBuffer(HERReplayBufferManager):\n \"\"\"HERVectorReplayBuffer contains n HERReplayBuffer with same size.\n\n It is used for storing transition from different environments yet keeping the order\n of time.\n\n :param total_size: the total size of HERVectorReplayBuffer.\n :param buffer_num: the number of HERReplayBuffer it uses, which are\n under the same configuration.\n\n Other input arguments are the same as :class:`~tianshou.data.HERReplayBuffer`.\n\n .. seealso::\n Please refer to :class:`~tianshou.data.ReplayBuffer` for other APIs' usage.\n \"\"\"\n\n def __init__(self, total_size: int, buffer_num: int, **kwargs: Any) -> None:\n assert buffer_num > 0\n size = int(np.ceil(total_size / buffer_num))\n buffer_list = [HERReplayBuffer(size, **kwargs) for _ in range(buffer_num)]\n super().__init__(buffer_list)\n" }, { "path": "tianshou/data/collector.py", "content": "import logging\nimport time\nimport warnings\nfrom abc import ABC, abstractmethod\nfrom copy import copy\nfrom dataclasses import dataclass, field\nfrom typing import Any, Generic, Optional, Protocol, Self, TypedDict, TypeVar, cast\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\nfrom overrides import override\nfrom torch.distributions import Categorical, Distribution\n\nfrom tianshou.algorithm import Algorithm\nfrom tianshou.algorithm.algorithm_base import Policy, episode_mc_return_to_go\nfrom tianshou.config import ENABLE_VALIDATION\nfrom tianshou.data import (\n Batch,\n CachedReplayBuffer,\n ReplayBuffer,\n ReplayBufferManager,\n SequenceSummaryStats,\n VectorReplayBuffer,\n to_numpy,\n)\nfrom tianshou.data.buffer.buffer_base import MalformedBufferError\nfrom tianshou.data.stats import compute_dim_to_summary_stats\nfrom tianshou.data.types import (\n ActBatchProtocol,\n DistBatchProtocol,\n ObsBatchProtocol,\n RolloutBatchProtocol,\n)\nfrom tianshou.env import BaseVectorEnv, DummyVectorEnv\nfrom tianshou.utils.determinism import TraceLogger\nfrom tianshou.utils.print import DataclassPPrintMixin\nfrom tianshou.utils.torch_utils import torch_train_mode\n\nlog = logging.getLogger(__name__)\n\nDEFAULT_BUFFER_MAXSIZE = int(1e4)\n\n_TArrLike = TypeVar(\"_TArrLike\", bound=\"np.ndarray | torch.Tensor | Batch | None\")\n\nTScalarArrayShape = TypeVar(\"TScalarArrayShape\")\n\n\nclass CollectActionBatchProtocol(Protocol):\n \"\"\"A protocol for results of computing actions from a batch of observations within a single collect step.\n\n All fields all have length R (the dist is a Distribution of batch size R),\n where R is the number of ready envs.\n \"\"\"\n\n act: np.ndarray | torch.Tensor\n act_normalized: np.ndarray | torch.Tensor\n policy_entry: Batch\n dist: Distribution | None\n hidden_state: np.ndarray | torch.Tensor | Batch | None\n\n\nclass CollectStepBatchProtocol(RolloutBatchProtocol):\n \"\"\"A batch of steps collected from a single collect step from multiple envs in parallel.\n\n All fields have length R (the dist is a Distribution of batch size R), where R is the number of ready envs.\n This is essentially the response of the vectorized environment to making a step\n with :class:`CollectActionBatchProtocol`.\n \"\"\"\n\n dist: Distribution | None\n\n\nclass EpisodeBatchProtocol(RolloutBatchProtocol):\n \"\"\"Marker interface for a batch containing a single episode.\n\n Instances are created by retrieving an episode from the buffer when the :class:`Collector` encounters\n `done=True`.\n \"\"\"\n\n\ndef get_stddev_from_dist(dist: Distribution) -> torch.Tensor:\n \"\"\"Return the standard deviation of the given distribution.\n\n Same as `dist.stddev` for all distributions except `Categorical`, where it is computed\n by assuming that the output values 0, ..., K have the corresponding numerical meaning.\n See `here `_\n for a discussion on `stddev` and `mean` of `Categorical`.\n \"\"\"\n if isinstance(dist, Categorical):\n # torch doesn't implement stddev for Categorical, so we compute it ourselves\n probs = torch.atleast_2d(dist.probs)\n n_actions = probs.shape[-1]\n possible_actions = torch.arange(n_actions, device=dist.probs.device).float()\n\n mean = torch.sum(probs * possible_actions, dim=1)\n var = torch.sum(probs * (possible_actions - mean.unsqueeze(1)) ** 2, dim=1)\n stddev = torch.sqrt(var)\n if len(dist.batch_shape) == 0:\n return stddev\n return torch.atleast_2d(stddev).T\n\n return dist.stddev if dist is not None else torch.tensor([])\n\n\n@dataclass(kw_only=True)\nclass CollectStatsBase(DataclassPPrintMixin):\n \"\"\"The most basic stats, often used for offline learning.\"\"\"\n\n n_collected_episodes: int = 0\n \"\"\"The number of collected episodes.\"\"\"\n n_collected_steps: int = 0\n \"\"\"The number of collected steps.\"\"\"\n\n\n@dataclass(kw_only=True)\nclass CollectStats(CollectStatsBase):\n \"\"\"A data structure for storing the statistics of rollouts.\n\n Custom stats collection logic can be implemented by subclassing this class and\n overriding the `update_` methods.\n\n Ideally, it is instantiated once with correct values and then never modified.\n However, during the collection process instances of modified\n using the `update_` methods. Then the arrays and their corresponding `_stats` fields\n may become out of sync (we don't update the stats after each update for performance reasons,\n only at the end of the collection). The same for the `collect_time` and `collect_speed`.\n In the `Collector`, :meth:`refresh_sequence_stats` and :meth:`set_collect_time` are\n is called at the end of the collection to update the stats. But for other use cases,\n the users should keep in mind to call this method manually if using the `update_`\n methods.\n \"\"\"\n\n collect_time: float = 0.0\n \"\"\"The time for collecting transitions.\"\"\"\n collect_speed: float = 0.0\n \"\"\"The speed of collecting (env_step per second).\"\"\"\n returns: np.ndarray = field(default_factory=lambda: np.array([], dtype=float))\n \"\"\"The collected episode returns.\"\"\"\n returns_stat: SequenceSummaryStats | None = None\n \"\"\"Stats of the collected returns.\"\"\"\n lens: np.ndarray = field(default_factory=lambda: np.array([], dtype=int))\n \"\"\"The collected episode lengths.\"\"\"\n lens_stat: SequenceSummaryStats | None = None\n \"\"\"Stats of the collected episode lengths.\"\"\"\n pred_dist_std_array: np.ndarray | None = None\n \"\"\"The standard deviations of the predicted distributions.\"\"\"\n pred_dist_std_array_stat: dict[int, SequenceSummaryStats] | None = None\n \"\"\"Stats of the standard deviations of the predicted distributions (maps action dim to stats)\"\"\"\n\n @classmethod\n def with_autogenerated_stats(\n cls,\n returns: np.ndarray,\n lens: np.ndarray,\n n_collected_episodes: int = 0,\n n_collected_steps: int = 0,\n collect_time: float = 0.0,\n collect_speed: float = 0.0,\n ) -> Self:\n \"\"\"Return a new instance with the stats autogenerated from the given lists.\"\"\"\n returns_stat = SequenceSummaryStats.from_sequence(returns) if returns.size > 0 else None\n lens_stat = SequenceSummaryStats.from_sequence(lens) if lens.size > 0 else None\n return cls(\n n_collected_episodes=n_collected_episodes,\n n_collected_steps=n_collected_steps,\n collect_time=collect_time,\n collect_speed=collect_speed,\n returns=returns,\n returns_stat=returns_stat,\n lens=np.array(lens, int),\n lens_stat=lens_stat,\n )\n\n def update_at_step_batch(\n self,\n step_batch: CollectStepBatchProtocol,\n refresh_sequence_stats: bool = False,\n ) -> None:\n self.n_collected_steps += len(step_batch)\n dist = step_batch.dist\n action_std: torch.Tensor | None = None\n\n if dist is not None:\n action_std = np.atleast_2d(to_numpy(get_stddev_from_dist(dist)))\n\n if self.pred_dist_std_array is None:\n self.pred_dist_std_array = np.atleast_2d(to_numpy(action_std))\n else:\n self.pred_dist_std_array = np.concatenate(\n (self.pred_dist_std_array, np.atleast_2d(to_numpy(action_std))),\n )\n if refresh_sequence_stats:\n self.refresh_std_array_stats()\n\n def update_at_episode_done(\n self,\n episode_batch: EpisodeBatchProtocol,\n # NOTE: in the MARL setting this is not actually a float but rather an array or list, see todo below\n episode_return: float,\n refresh_sequence_stats: bool = False,\n ) -> None:\n self.lens = np.concatenate((self.lens, [len(episode_batch)]), dtype=int) # type: ignore\n self.n_collected_episodes += 1\n if self.returns.size == 0:\n # TODO: needed for non-1dim arrays returns that happen in the MARL setting\n # There are multiple places that assume the returns to be 1dim, so this is a hack\n # Since MARL support is currently not a priority, we should either raise an error or\n # implement proper support for it. At the moment tests like `test_collector_with_multi_agent` fail\n # when assuming 1d returns\n self.returns = np.array([episode_return], dtype=float)\n else:\n self.returns = np.concatenate((self.returns, [episode_return]), dtype=float) # type: ignore\n if refresh_sequence_stats:\n self.refresh_return_stats()\n self.refresh_len_stats()\n\n def set_collect_time(self, collect_time: float, update_collect_speed: bool = True) -> None:\n if collect_time < 0:\n raise ValueError(f\"Collect time should be non-negative, but got {collect_time=}.\")\n\n self.collect_time = collect_time\n if update_collect_speed:\n if collect_time == 0:\n log.error(\n \"Collect time is 0, setting collect speed to 0. Did you make a rounding error?\",\n )\n self.collect_speed = 0.0\n else:\n self.collect_speed = self.n_collected_steps / collect_time\n\n def refresh_return_stats(self) -> None:\n if self.returns.size > 0:\n self.returns_stat = SequenceSummaryStats.from_sequence(self.returns)\n else:\n self.returns_stat = None\n\n def refresh_len_stats(self) -> None:\n if self.lens.size > 0:\n self.lens_stat = SequenceSummaryStats.from_sequence(self.lens)\n else:\n self.lens_stat = None\n\n def refresh_std_array_stats(self) -> None:\n if self.pred_dist_std_array is not None and self.pred_dist_std_array.size > 0:\n # need to use .T because action dim supposed to be the first axis in compute_dim_to_summary_stats\n self.pred_dist_std_array_stat = compute_dim_to_summary_stats(self.pred_dist_std_array.T)\n else:\n self.pred_dist_std_array_stat = None\n\n def refresh_all_sequence_stats(self) -> None:\n self.refresh_return_stats()\n self.refresh_len_stats()\n self.refresh_std_array_stats()\n\n\nTCollectStats = TypeVar(\"TCollectStats\", bound=CollectStats)\n\n\ndef _nullable_slice(obj: _TArrLike, indices: np.ndarray) -> _TArrLike:\n \"\"\"Return None, or the values at the given indices if the object is not None.\"\"\"\n if obj is not None:\n return obj[indices] # type: ignore[index, return-value]\n return None # type: ignore[unreachable]\n\n\ndef _dict_of_arr_to_arr_of_dicts(\n dict_of_arr: dict[str, np.ndarray | dict],\n) -> np.ndarray:\n return np.array(Batch(dict_of_arr).to_list_of_dicts())\n\n\ndef _HACKY_create_info_batch(info_array: np.ndarray) -> Batch:\n \"\"\"TODO: this exists because of multiple bugs in Batch and to restore backwards compatibility.\n Batch should be fixed and this function should be removed asap!.\n \"\"\"\n if info_array.dtype != np.dtype(\"O\"):\n raise ValueError(\n f\"Expected info_array to have dtype=object, but got {info_array.dtype}.\",\n )\n\n truthy_info_indices = info_array.nonzero()[0]\n falsy_info_indices = set(range(len(info_array))) - set(truthy_info_indices)\n falsy_info_indices = np.array(list(falsy_info_indices), dtype=int)\n\n if len(falsy_info_indices) == len(info_array):\n return Batch()\n\n some_nonempty_info = None\n for info in info_array:\n if info:\n some_nonempty_info = info\n break\n\n info_array = copy(info_array)\n info_array[falsy_info_indices] = some_nonempty_info\n result_batch_parent = Batch(info=info_array)\n result_batch_parent.info[falsy_info_indices] = {}\n return result_batch_parent.info\n\n\nclass BaseCollector(Generic[TCollectStats], ABC):\n \"\"\"Used to collect data from a vector environment into a buffer using a given policy.\n\n .. note::\n\n Please make sure the given environment has a time limitation if using `n_episode`\n collect option.\n\n .. note::\n\n In past versions of Tianshou, the replay buffer passed to `__init__`\n was automatically reset. This is not done in the current implementation.\n \"\"\"\n\n def __init__(\n self,\n policy: Policy | Algorithm,\n env: BaseVectorEnv | gym.Env,\n buffer: ReplayBuffer | None = None,\n exploration_noise: bool = False,\n # The typing is correct, there's a bug in mypy, see https://github.com/python/mypy/issues/3737\n collect_stats_class: type[TCollectStats] = CollectStats, # type: ignore[assignment]\n raise_on_nan_in_buffer: bool = ENABLE_VALIDATION,\n ) -> None:\n \"\"\"\n :param policy: a tianshou policy, each :class:`BasePolicy` is capable of computing a batch\n of actions from a batch of observations.\n :param env: a ``gymnasium.Env`` environment or a vectorized instance of the\n :class:`~tianshou.env.BaseVectorEnv` class. The latter is strongly recommended, as with\n a gymnasium env the collection will not happen in parallel (a `DummyVectorEnv`\n will be constructed internally from the passed env)\n :param buffer: an instance of the :class:`~tianshou.data.ReplayBuffer` class.\n If set to None, will instantiate a :class:`~tianshou.data.VectorReplayBuffer`\n of size :data:`DEFAULT_BUFFER_MAXSIZE` * (number of envs)\n as the default buffer.\n :param exploration_noise: determine whether the action needs to be modified\n with the corresponding policy's exploration noise. If so, \"policy.\n exploration_noise(act, batch)\" will be called automatically to add the\n exploration noise into action.\n the rollout batch with this hook also modifies the data that is collected to the buffer!\n :param raise_on_nan_in_buffer: whether to raise a `RuntimeError` if NaNs are found in the buffer after\n a collection step. Especially useful when episode-level hooks are passed for making\n sure that nothing is broken during the collection. Consider setting to False if\n the NaN-check becomes a bottleneck.\n :param collect_stats_class: the class to use for collecting statistics. Allows customizing\n the stats collection logic by passing a subclass of :class:`CollectStats`. Changing\n this is rarely necessary and is mainly done by \"power users\".\n \"\"\"\n if isinstance(env, gym.Env) and not hasattr(env, \"__len__\"):\n warnings.warn(\"Single environment detected, wrap to DummyVectorEnv.\")\n # Unfortunately, mypy seems to ignore the isinstance in lambda, maybe a bug in mypy\n env = DummyVectorEnv([lambda: env]) # type: ignore\n\n if buffer is None:\n buffer = VectorReplayBuffer(DEFAULT_BUFFER_MAXSIZE * len(env), len(env))\n\n self.buffer: ReplayBuffer | ReplayBufferManager = buffer\n self.raise_on_nan_in_buffer = raise_on_nan_in_buffer\n self.policy = policy.policy if isinstance(policy, Algorithm) else policy\n self.env = cast(BaseVectorEnv, env)\n self.exploration_noise = exploration_noise\n self.collect_step, self.collect_episode, self.collect_time = 0, 0, 0.0\n\n self._action_space = self.env.action_space\n self._is_closed = False\n\n self._validate_buffer()\n self.collect_stats_class = collect_stats_class\n\n def _validate_buffer(self) -> None:\n buf = self.buffer\n # TODO: a bit weird but true - all VectorReplayBuffers inherit from ReplayBufferManager.\n # We should probably rename the manager\n if isinstance(buf, ReplayBufferManager) and buf.buffer_num < self.env_num:\n raise ValueError(\n f\"Buffer has only {buf.buffer_num} buffers, but at least {self.env_num=} are needed.\",\n )\n if isinstance(buf, CachedReplayBuffer) and buf.cached_buffer_num < self.env_num:\n raise ValueError(\n f\"Buffer has only {buf.cached_buffer_num} cached buffers, but at least {self.env_num=} are needed.\",\n )\n # Non-VectorReplayBuffer. TODO: probably shouldn't rely on isinstance\n if not isinstance(buf, ReplayBufferManager):\n if buf.maxsize == 0:\n raise ValueError(\"Buffer maxsize should be greater than 0.\")\n if self.env_num > 1:\n raise ValueError(\n f\"Cannot use {type(buf).__name__} to collect from multiple envs ({self.env_num=}). \"\n f\"Please use the corresponding VectorReplayBuffer instead.\",\n )\n\n @property\n def env_num(self) -> int:\n return len(self.env)\n\n @property\n def action_space(self) -> gym.spaces.Space:\n return self._action_space\n\n def close(self) -> None:\n \"\"\"Close the collector and the environment.\"\"\"\n self.env.close()\n self._is_closed = True\n\n def reset(\n self,\n reset_buffer: bool = True,\n reset_stats: bool = True,\n gym_reset_kwargs: dict[str, Any] | None = None,\n ) -> tuple[np.ndarray, np.ndarray]:\n \"\"\"Reset the environment, statistics, and data needed to start the collection.\n\n :param reset_buffer: if true, reset the replay buffer attached\n to the collector.\n :param reset_stats: if true, reset the statistics attached to the collector.\n :param gym_reset_kwargs: extra keyword arguments to pass into the environment's\n reset function. Defaults to None (extra keyword arguments)\n :return: The initial observation and info from the environment.\n \"\"\"\n obs_NO, info_N = self.reset_env(gym_reset_kwargs=gym_reset_kwargs)\n if reset_buffer:\n self.reset_buffer()\n if reset_stats:\n self.reset_stat()\n self._is_closed = False\n return obs_NO, info_N\n\n def reset_stat(self) -> None:\n \"\"\"Reset the statistic variables.\"\"\"\n self.collect_step, self.collect_episode, self.collect_time = 0, 0, 0.0\n\n def reset_buffer(self, keep_statistics: bool = False) -> None:\n \"\"\"Reset the data buffer.\"\"\"\n self.buffer.reset(keep_statistics=keep_statistics)\n\n def reset_env(\n self,\n gym_reset_kwargs: dict[str, Any] | None = None,\n ) -> tuple[np.ndarray, np.ndarray]:\n \"\"\"Reset the environments and the initial obs, info, and hidden state of the collector.\"\"\"\n gym_reset_kwargs = gym_reset_kwargs or {}\n obs_NO, info_N = self.env.reset(**gym_reset_kwargs)\n # TODO: hack, wrap envpool envs such that they don't return a dict\n if isinstance(info_N, dict): # type: ignore[unreachable]\n # this can happen if the env is an envpool env. Then the thing returned by reset is a dict\n # with array entries instead of an array of dicts\n # We use Batch to turn it into an array of dicts\n info_N = _dict_of_arr_to_arr_of_dicts(info_N) # type: ignore[unreachable]\n return obs_NO, info_N\n\n @abstractmethod\n def _collect(\n self,\n n_step: int | None = None,\n n_episode: int | None = None,\n random: bool = False,\n render: float | None = None,\n gym_reset_kwargs: dict[str, Any] | None = None,\n ) -> TCollectStats:\n pass\n\n def collect(\n self,\n n_step: int | None = None,\n n_episode: int | None = None,\n random: bool = False,\n render: float | None = None,\n reset_before_collect: bool = False,\n gym_reset_kwargs: dict[str, Any] | None = None,\n ) -> TCollectStats:\n \"\"\"Collect the specified number of steps or episodes to the buffer.\n\n .. note::\n\n One and only one collection specification is permitted, either\n ``n_step`` or ``n_episode``.\n\n To ensure an unbiased sampling result with the `n_episode` option, this function will\n first collect ``n_episode - env_num`` episodes, then for the last ``env_num``\n episodes, they will be collected evenly from each env.\n\n :param n_step: how many steps to collect.\n :param n_episode: how many episodes to collect.\n :param random: whether to sample randomly from the action space instead of using the policy for collecting data.\n :param render: the sleep time between rendering consecutive frames.\n :param reset_before_collect: whether to reset the environment before collecting data.\n (The collector needs the initial `obs` and `info` to function properly.)\n :param gym_reset_kwargs: extra keyword arguments to pass into the environment's\n reset function. Only used if reset_before_collect is True.\n\n\n :return: The collected stats\n \"\"\"\n # check that exactly one of n_step or n_episode is set and that the other is larger than 0\n self._validate_n_step_n_episode(n_episode, n_step)\n\n if reset_before_collect:\n self.reset(reset_buffer=False, gym_reset_kwargs=gym_reset_kwargs)\n\n pre_collect_time = time.time()\n with torch_train_mode(self.policy, enabled=False):\n collect_stats = self._collect(\n n_step=n_step,\n n_episode=n_episode,\n random=random,\n render=render,\n gym_reset_kwargs=gym_reset_kwargs,\n )\n collect_time = time.time() - pre_collect_time\n collect_stats.set_collect_time(collect_time, update_collect_speed=True)\n collect_stats.refresh_all_sequence_stats()\n\n if self.raise_on_nan_in_buffer and self.buffer.hasnull():\n nan_batch = self.buffer.isnull().apply_values_transform(np.sum)\n\n raise MalformedBufferError(\n \"NaN detected in the buffer. You can drop them with `buffer.dropnull()`. \"\n f\"This error is most often caused by an incorrect use of {EpisodeRolloutHook.__name__}\"\n \"together with the `n_steps` (instead of `n_episodes`) option, or by \"\n f\"an incorrect implementation of {StepHook.__name__}.\"\n \"Here an overview of the number of NaNs per field: \\n\"\n f\"{nan_batch}\",\n )\n\n return collect_stats\n\n def _validate_n_step_n_episode(self, n_episode: int | None, n_step: int | None) -> None:\n if not n_step and not n_episode:\n raise ValueError(\n f\"Only one of n_step and n_episode should be set to a value larger than zero \"\n f\"but got {n_step=}, {n_episode=}.\",\n )\n if n_step is None and n_episode is None:\n raise ValueError(\n \"Exactly one of n_step and n_episode should be set but got None for both.\",\n )\n if n_step and n_step % self.env_num != 0:\n warnings.warn(\n f\"{n_step=} is not a multiple of ({self.env_num=}), \"\n \"which may cause extra transitions being collected into the buffer.\",\n )\n if n_episode and self.env_num > n_episode:\n warnings.warn(\n f\"{n_episode=} should be larger than {self.env_num=} to \"\n f\"collect at least one trajectory in each environment.\",\n )\n\n\nclass Collector(BaseCollector[TCollectStats], Generic[TCollectStats]):\n \"\"\"Collects transitions from a vectorized env by computing and applying actions batch-wise.\"\"\"\n\n # NAMING CONVENTION (mostly suffixes):\n # episode - An episode means a rollout until done (terminated or truncated). After an episode is completed,\n # the corresponding env is either reset or removed from the ready envs.\n # N - number of envs, always fixed and >= R.\n # R - number ready env ids. Note that this might change when envs get idle.\n # This can only happen in n_episode case, see explanation in the corresponding block.\n # For n_step, we always use all envs to collect the data, while for n_episode,\n # R will be at most n_episode at the beginning, but can decrease during the collection.\n # O - dimension(s) of observations\n # A - dimension(s) of actions\n # H - dimension(s) of hidden state\n # D - number of envs that reached done in the current collect iteration. Only relevant in n_episode case.\n # S - number of surplus envs, i.e., envs that are ready but won't be used in the next iteration.\n # Only used in n_episode case. Then, R becomes R-S.\n # local_index - selecting from the locally available environments. In more details:\n # Each env is associated to a number in [0,..., N-1]. At any moment there are R ready envs,\n # but they are not necessarily equal to [0, ..., R-1]. Let the R corresponding indices be\n # [r_0, ..., r_(R-1)] (each r_i is in [0, ... N-1]). If the local index is\n # [0, 1, 2], it means that we want to select envs [r_0, r_1, r_2].\n # We will usually select from the ready envs by slicing like `ready_env_idx_R[local_index]`\n # global_index - the index in [0, ..., N-1]. Slicing a `_R` index by a local_index produces the\n # corresponding global index. In the example above:\n # 1. _R index is [r_0, ..., r_(R-1)]\n # 2. local_index is [0, 1, 2]\n # 3. global_index is [r_0, r_1, r_2] and can be used to select from an array of length N\n #\n def __init__(\n self,\n policy: Policy | Algorithm,\n env: gym.Env | BaseVectorEnv,\n buffer: ReplayBuffer | None = None,\n exploration_noise: bool = False,\n on_episode_done_hook: Optional[\"EpisodeRolloutHookProtocol\"] = None,\n on_step_hook: Optional[\"StepHookProtocol\"] = None,\n raise_on_nan_in_buffer: bool = ENABLE_VALIDATION,\n collect_stats_class: type[TCollectStats] = CollectStats, # type: ignore[assignment]\n ) -> None:\n \"\"\"\n :param policy: a tianshou policy or algorithm\n :param env: a ``gymnasium.Env`` environment or a vectorized instance of the\n :class:`~tianshou.env.BaseVectorEnv` class. The latter is strongly recommended, as with\n a gymnasium env the collection will not happen in parallel (a `DummyVectorEnv`\n will be constructed internally from the passed env)\n :param buffer: an instance of the :class:`~tianshou.data.ReplayBuffer` class.\n If set to None, will instantiate a :class:`~tianshou.data.VectorReplayBuffer`\n of size :data:`DEFAULT_BUFFER_MAXSIZE` * (number of envs)\n as the default buffer.\n :param exploration_noise: determine whether the action needs to be modified\n with the corresponding policy's exploration noise. If so, \"policy.\n exploration_noise(act, batch)\" will be called automatically to add the\n exploration noise into action.\n :param on_episode_done_hook: if passed will be executed when an episode is done.\n The input to the hook will be a `RolloutBatch` that contains the entire episode (and nothing else).\n If a dict is returned by the hook it will be used to add new entries to the buffer\n for the episode that just ended. The values of the dict should be arrays with floats\n of the same length as the input rollout batch.\n Note that multiple hooks can be combined using :class:`EpisodeRolloutHookMerged`.\n A typical example of a hook is :class:`EpisodeRolloutHookMCReturn` which adds the Monte Carlo return\n as a field to the buffer.\n\n Care must be taken when using such hook, as for unfinished episodes one can easily end\n up with NaNs in the buffer. It is recommended to use the hooks only with the `n_episode` option\n in `collect`, or to strip the buffer of NaNs after the collection.\n :param on_step_hook: if passed will be executed after each step of the collection but before the\n resulting rollout batch is added to the buffer. The inputs to the hook will be\n the action distributions computed from the previous observations (following the\n :class:`CollectActionBatchProtocol`) using the policy, and the resulting\n rollout batch (following the :class:`RolloutBatchProtocol`). **Note** that modifying\n the rollout batch with this hook also modifies the data that is collected to the buffer!\n :param raise_on_nan_in_buffer: whether to raise a `RuntimeError` if NaNs are found in the buffer after\n a collection step. Especially useful when episode-level hooks are passed for making\n sure that nothing is broken during the collection. Consider setting to False if\n the NaN-check becomes a bottleneck.\n :param collect_stats_class: the class to use for collecting statistics. Allows customizing\n the stats collection logic by passing a subclass of :class:`CollectStats`. Changing\n this is rarely necessary and is mainly done by \"power users\".\n \"\"\"\n super().__init__(\n policy,\n env,\n buffer,\n exploration_noise=exploration_noise,\n collect_stats_class=collect_stats_class,\n raise_on_nan_in_buffer=raise_on_nan_in_buffer,\n )\n\n self._pre_collect_obs_RO: np.ndarray | None = None\n self._pre_collect_info_R: np.ndarray | None = None\n self._pre_collect_hidden_state_RH: np.ndarray | torch.Tensor | Batch | None = None\n\n self._is_closed = False\n self._on_episode_done_hook = on_episode_done_hook\n self._on_step_hook = on_step_hook\n self.collect_step, self.collect_episode, self.collect_time = 0, 0, 0.0\n\n def set_on_episode_done_hook(self, hook: Optional[\"EpisodeRolloutHookProtocol\"]) -> None:\n self._on_episode_done_hook = hook\n\n def set_on_step_hook(self, hook: Optional[\"StepHookProtocol\"]) -> None:\n self._on_step_hook = hook\n\n def get_on_episode_done_hook(self) -> Optional[\"EpisodeRolloutHookProtocol\"]:\n return self._on_episode_done_hook\n\n def get_on_step_hook(self) -> Optional[\"StepHookProtocol\"]:\n return self._on_step_hook\n\n def close(self) -> None:\n super().close()\n self._pre_collect_obs_RO = None\n self._pre_collect_info_R = None\n\n def run_on_episode_done(\n self,\n episode_batch: EpisodeBatchProtocol,\n ) -> dict[str, np.ndarray] | None:\n \"\"\"Executes the `on_episode_done_hook` that was passed on init.\n\n One of the main uses of this public method is to allow users to override it in custom\n subclasses of :class:`Collector`. This way, they can override the init to no longer accept\n the `on_episode_done` provider.\n \"\"\"\n if self._on_episode_done_hook is not None:\n return self._on_episode_done_hook(episode_batch)\n return None\n\n def run_on_step_hook(\n self,\n action_batch: CollectActionBatchProtocol,\n rollout_batch: RolloutBatchProtocol,\n ) -> None:\n \"\"\"Executes the instance's `on_step_hook`.\n\n One of the main uses of this public method is to allow users to override it in custom\n subclasses of the `Collector`. This way, they can override the init to no longer accept\n the `on_step_hook` provider.\n \"\"\"\n if self._on_step_hook is not None:\n self._on_step_hook(action_batch, rollout_batch)\n\n def reset_env(\n self,\n gym_reset_kwargs: dict[str, Any] | None = None,\n ) -> tuple[np.ndarray, np.ndarray]:\n \"\"\"Reset the environments and the initial obs, info, and hidden state of the collector.\"\"\"\n obs_NO, info_N = super().reset_env(gym_reset_kwargs=gym_reset_kwargs)\n # We assume that R = N when reset is called.\n # TODO: there is currently no mechanism that ensures this and it's a public method!\n self._pre_collect_obs_RO = obs_NO\n self._pre_collect_info_R = info_N\n self._pre_collect_hidden_state_RH = None\n return obs_NO, info_N\n\n def _compute_action_policy_hidden(\n self,\n random: bool,\n ready_env_ids_R: np.ndarray,\n last_obs_RO: np.ndarray,\n last_info_R: np.ndarray,\n last_hidden_state_RH: np.ndarray | torch.Tensor | Batch | None = None,\n ) -> CollectActionBatchProtocol:\n \"\"\"Returns the action, the normalized action, a \"policy\" entry, and the hidden state.\"\"\"\n if random:\n try:\n act_normalized_RA = np.array(\n [self._action_space[i].sample() for i in ready_env_ids_R],\n )\n # TODO: test whether envpool env explicitly\n except TypeError: # envpool's action space is not for per-env\n act_normalized_RA = np.array([self._action_space.sample() for _ in ready_env_ids_R])\n act_RA = self.policy.map_action_inverse(np.array(act_normalized_RA))\n policy_R = Batch()\n hidden_state_RH = None\n # TODO: instead use a (uniform) Distribution instance that corresponds to sampling from action_space\n action_dist_R = None\n\n else:\n info_batch = _HACKY_create_info_batch(last_info_R)\n obs_batch_R = cast(ObsBatchProtocol, Batch(obs=last_obs_RO, info=info_batch))\n\n act_batch_RA: ActBatchProtocol | DistBatchProtocol = self.policy(\n obs_batch_R,\n last_hidden_state_RH,\n )\n\n act_RA = to_numpy(act_batch_RA.act)\n if self.exploration_noise:\n act_RA = self.policy.add_exploration_noise(act_RA, obs_batch_R)\n act_normalized_RA = self.policy.map_action(act_RA)\n\n # TODO: cleanup the whole policy in batch thing\n # todo policy_R can also be none, check\n policy_R = act_batch_RA.get(\"policy\", Batch())\n if not isinstance(policy_R, Batch):\n raise RuntimeError(\n f\"The policy result should be a {Batch}, but got {type(policy_R)}\",\n )\n\n hidden_state_RH = act_batch_RA.get(\"state\", None)\n # TODO: do we need the conditional? Would be better to just add hidden_state which could be None\n if hidden_state_RH is not None:\n policy_R.hidden_state = (\n hidden_state_RH # save state into buffer through policy attr\n )\n # can't use act_batch_RA.dist directly as act_batch_RA might not have that attribute\n action_dist_R = act_batch_RA.get(\"dist\")\n\n return cast(\n CollectActionBatchProtocol,\n Batch(\n act=act_RA,\n act_normalized=act_normalized_RA,\n policy_entry=policy_R,\n dist=action_dist_R,\n hidden_state=hidden_state_RH,\n ),\n )\n\n # TODO: reduce complexity, remove the noqa\n def _collect( # noqa: C901\n self,\n n_step: int | None = None,\n n_episode: int | None = None,\n random: bool = False,\n render: float | None = None,\n gym_reset_kwargs: dict[str, Any] | None = None,\n ) -> TCollectStats:\n \"\"\"This method is currently very complex, but it's difficult to break it down into smaller chunks.\n\n Please read the block-comment of the class to understand the notation\n in the implementation.\n\n It does the collection by executing the following logic:\n\n 0. Keep track of n_step and n_episode for being able to stop the collection.\n 1. Create a CollectStats instance to store the statistics of the collection.\n 2. Compute actions (with policy or sampling from action space) for the R currently active envs.\n 3. Perform a step in these R envs.\n 4. Perform on-step hook on the result\n 5. Update the CollectStats (using `update_at_step_batch`) and the internal counters after the step\n 6. Add the resulting R transitions to the buffer\n 7. Find the D envs that reached done in the current iteration\n 8. Reset the envs that reached done\n 9. Extract episodes for the envs that reached done from the buffer\n 10. Perform on-episode-done hook. If it has a return, modify the transitions belonging to the episodes inside the buffer inplace\n 11. Update the CollectStats instance with the episodes from 9. by using `update_on_episode_done`\n 12. Prepare next step in while loop by saving the last observations and infos\n 13. Remove S surplus envs from collection mechanism, thereby reducing R to R-S, to increase performance\n 14. Update instance-level collection counters (contrary to counters with a lifetime of the collect execution)\n 15. Prepare for the next call of collect (save last observations and info to collector state)\n\n You can search for Step to find where it happens\n \"\"\"\n # TODO: can't do it init since AsyncCollector is currently a subclass of Collector\n if self.env.is_async:\n raise ValueError(\n f\"Please use AsyncCollector for asynchronous environments. \"\n f\"Env class: {self.env.__class__.__name__}.\",\n )\n\n ready_env_ids_R: np.ndarray[Any, np.dtype[np.signedinteger]]\n \"\"\"provides a mapping from local indices (indexing within `1, ..., R` where `R` is the number of ready envs)\n to global ones (indexing within `1, ..., num_envs`). So the entry i in this array is the global index of the i-th ready env.\"\"\"\n if n_step is not None:\n ready_env_ids_R = np.arange(self.env_num)\n elif n_episode is not None:\n if self.env_num > n_episode:\n log.warning(\n f\"Number of episodes ({n_episode}) is smaller than the number of environments \"\n f\"({self.env_num}). This means that {self.env_num - n_episode} \"\n f\"environments (or, equivalently, parallel workers) will not be used!\",\n )\n ready_env_ids_R = np.arange(min(self.env_num, n_episode))\n else:\n raise RuntimeError(\"Input validation failed, this is a bug and shouldn't have happened\")\n\n if self._pre_collect_obs_RO is None or self._pre_collect_info_R is None:\n raise ValueError(\n \"Initial obs and info should not be None. \"\n \"Either reset the collector (using reset or reset_env) or pass reset_before_collect=True to collect.\",\n )\n\n # Step 0\n # get the first obs to be the current obs in the n_step case as\n # episodes as a new call to collect does not restart trajectories\n # (which we also really don't want)\n step_count = 0\n num_collected_episodes = 0\n episode_returns: list[float] = []\n episode_lens: list[int] = []\n episode_start_indices: list[int] = []\n\n # Step 1\n collect_stats = self.collect_stats_class()\n\n # in case we select fewer episodes than envs, we run only some of them\n last_obs_RO = _nullable_slice(self._pre_collect_obs_RO, ready_env_ids_R)\n last_info_R = _nullable_slice(self._pre_collect_info_R, ready_env_ids_R)\n last_hidden_state_RH = _nullable_slice(\n self._pre_collect_hidden_state_RH,\n ready_env_ids_R,\n )\n\n while True:\n # todo check if we need this when using cur_rollout_batch\n # if len(cur_rollout_batch) != len(ready_env_ids):\n # raise RuntimeError(\n # f\"The length of the collected_rollout_batch {len(cur_rollout_batch)}) is not equal to the length of ready_env_ids\"\n # f\"{len(ready_env_ids)}. This should not happen and could be a bug!\",\n # )\n # restore the state: if the last state is None, it won't store\n\n # Step 2\n # get the next action and related stats from the previous observation\n collect_action_computation_batch_R = self._compute_action_policy_hidden(\n random=random,\n ready_env_ids_R=ready_env_ids_R,\n last_obs_RO=last_obs_RO,\n last_info_R=last_info_R,\n last_hidden_state_RH=last_hidden_state_RH,\n )\n TraceLogger.log(log, lambda: f\"Action: {collect_action_computation_batch_R.act}\")\n\n # Step 3\n obs_next_RO, rew_R, terminated_R, truncated_R, info_R = self.env.step(\n collect_action_computation_batch_R.act_normalized,\n ready_env_ids_R,\n )\n if isinstance(info_R, dict): # type: ignore[unreachable]\n # This can happen if the env is an envpool env. Then the info returned by step is a dict\n info_R = _dict_of_arr_to_arr_of_dicts(info_R) # type: ignore[unreachable]\n done_R = np.logical_or(terminated_R, truncated_R)\n\n current_step_batch_R = cast(\n CollectStepBatchProtocol,\n Batch(\n obs=last_obs_RO,\n dist=collect_action_computation_batch_R.dist,\n act=collect_action_computation_batch_R.act,\n policy=collect_action_computation_batch_R.policy_entry,\n obs_next=obs_next_RO,\n rew=rew_R,\n terminated=terminated_R,\n truncated=truncated_R,\n done=done_R,\n info=info_R,\n ),\n )\n\n # TODO: only makes sense if render_mode is human.\n # Also, doubtful whether it makes sense at all for true vectorized envs\n if render:\n self.env.render()\n if not np.isclose(render, 0):\n time.sleep(render)\n\n # Step 4\n self.run_on_step_hook(\n collect_action_computation_batch_R,\n current_step_batch_R,\n )\n\n # Step 5, collect statistics\n collect_stats.update_at_step_batch(current_step_batch_R)\n num_episodes_done_this_iter = np.sum(done_R)\n num_collected_episodes += num_episodes_done_this_iter\n step_count += len(ready_env_ids_R)\n\n # Step 6\n # add data into the buffer. Since the buffer is essentially an array, we don't want\n # to add the dist. One should not have arrays of dists but rather a single, batch-wise dist.\n # Tianshou already implements slicing of dists, but we don't yet implement merging multiple\n # dists into one, which would be necessary to make a buffer with dists work properly\n batch_to_add_R = copy(current_step_batch_R)\n batch_to_add_R.pop(\"dist\")\n batch_to_add_R = cast(RolloutBatchProtocol, batch_to_add_R)\n insertion_idx_R, ep_return_R, ep_len_R, ep_start_idx_R = self.buffer.add(\n batch_to_add_R,\n buffer_ids=ready_env_ids_R,\n )\n\n # preparing for the next iteration\n # obs_next, info and hidden_state will be modified inplace in the code below,\n # so we copy to not affect the data in the buffer\n last_obs_RO = copy(obs_next_RO)\n last_info_R = copy(info_R)\n last_hidden_state_RH = copy(collect_action_computation_batch_R.hidden_state)\n\n # Preparing last_obs_RO, last_info_R, last_hidden_state_RH for the next while-loop iteration\n # Resetting envs that reached done, or removing some of them from the collection if needed (see below)\n if num_episodes_done_this_iter > 0:\n # TODO: adjust the whole index story, don't use np.where, just slice with boolean arrays\n # D - number of envs that reached done in the rollout above\n # local_idx - see block comment on class level\n # Step 7\n env_done_local_idx_D = np.where(done_R)[0]\n \"\"\"Indexes which episodes are done within the ready envs, so it can be used for selecting from `..._R` arrays.\n Stands in contrast to the \"global\" index, which counts within all envs and is unsuitable for selecting from `..._R` arrays.\"\"\"\n episode_lens_D = ep_len_R[env_done_local_idx_D]\n episode_returns_D = ep_return_R[env_done_local_idx_D]\n episode_start_indices_D = ep_start_idx_R[env_done_local_idx_D]\n\n episode_lens.extend(episode_lens_D)\n episode_returns.extend(episode_returns_D)\n episode_start_indices.extend(episode_start_indices_D)\n\n # Step 8\n # now we copy obs_next to obs, but since there might be\n # finished episodes, we have to reset finished envs first.\n gym_reset_kwargs = gym_reset_kwargs or {}\n # The index env_done_idx_D was based on 0, ..., R\n # However, each env has an index in the context of the vectorized env and buffer. So the env 0 being done means\n # that some env of the corresponding \"global\" index was done. The mapping between \"local\" index in\n # 0,...,R and this global index is maintained by the ready_env_ids_R array.\n # See the class block comment for more details\n env_done_global_idx_D = ready_env_ids_R[env_done_local_idx_D]\n \"\"\"Indexes which episodes are done within all envs, i.e., within the index `1, ..., num_envs`. It can be\n used to communicate with the vector env, where env ids are selected from this \"global\" index.\n Is not suited for selecting from the ready envs (`..._R` arrays), use the local counterpart instead.\n \"\"\"\n obs_reset_DO, info_reset_D = self.env.reset(\n env_id=env_done_global_idx_D,\n **gym_reset_kwargs,\n )\n\n # Set the hidden state to zero or None for the envs that reached done\n # TODO: does it have to be so complicated? We should have a single clear type for hidden_state instead of\n # this complex logic\n self._reset_hidden_state_based_on_type(env_done_local_idx_D, last_hidden_state_RH)\n\n # Step 9\n # execute episode hooks for those envs which emitted 'done'\n for local_done_idx, cur_ep_return in zip(\n env_done_local_idx_D,\n episode_returns_D,\n strict=True,\n ):\n # retrieve the episode batch from the buffer using the episode start and stop indices\n ep_start_idx, ep_stop_idx = (\n int(ep_start_idx_R[local_done_idx]),\n int(insertion_idx_R[local_done_idx] + 1),\n )\n\n ep_index_array = self.buffer.get_buffer_indices(ep_start_idx, ep_stop_idx)\n ep_batch = cast(EpisodeBatchProtocol, self.buffer[ep_index_array])\n\n # Step 10\n episode_hook_additions = self.run_on_episode_done(ep_batch)\n if episode_hook_additions is not None:\n if n_episode is None:\n raise ValueError(\n \"An on_episode_done_hook with non-empty returns is not supported for n_step collection.\"\n \"Such hooks should only be used when collecting full episodes. Got a on_episode_done_hook \"\n f\"that would add the following fields to the buffer: {list(episode_hook_additions)}.\",\n )\n\n for key, episode_addition in episode_hook_additions.items():\n self.buffer.set_array_at_key(\n episode_addition,\n key,\n index=ep_index_array,\n )\n # executing the same logic in the episode-batch since stats computation\n # may depend on the presence of additional fields\n ep_batch.set_array_at_key(\n episode_addition,\n key,\n )\n # Step 11\n # Finally, update the stats\n collect_stats.update_at_episode_done(\n episode_batch=ep_batch,\n episode_return=cur_ep_return,\n )\n\n # Step 12\n # preparing for the next iteration\n last_obs_RO[env_done_local_idx_D] = obs_reset_DO\n last_info_R[env_done_local_idx_D] = info_reset_D\n\n # Step 13\n # Handling the case when we have more ready envs than desired and are not done yet\n #\n # This can only happen if we are collecting a fixed number of episodes\n # If we have more ready envs than there are remaining episodes to collect,\n # we will remove some of them for the next rollout\n # One effect of this is the following: only envs that have completed an episode\n # in the last step can ever be removed from the ready envs.\n # Thus, this guarantees that each env will contribute at least one episode to the\n # collected data (the buffer). This effect was previous called \"avoiding bias in selecting environments\"\n # However, it is not at all clear whether this is actually useful or necessary.\n # Additional naming convention:\n # S - number of surplus envs\n # TODO: can the whole block be removed? If we have too many episodes, we could just strip the last ones.\n # Changing R to R-S highly increases the complexity of the code.\n if n_episode:\n remaining_episodes_to_collect = n_episode - num_collected_episodes\n surplus_env_num = len(ready_env_ids_R) - remaining_episodes_to_collect\n if surplus_env_num > 0:\n # R becomes R-S here, preparing for the next iteration in while loop\n # Everything that was of length R needs to be filtered and become of length R-S.\n # Note that this won't be the last iteration, as one iteration equals one\n # step and we still need to collect the remaining episodes to reach the breaking condition.\n\n # creating the mask\n env_to_be_ignored_ind_local_S = env_done_local_idx_D[:surplus_env_num]\n env_should_remain_R = np.ones_like(ready_env_ids_R, dtype=bool)\n env_should_remain_R[env_to_be_ignored_ind_local_S] = False\n # stripping the \"idle\" indices, shortening the relevant quantities from R to R-S\n ready_env_ids_R = ready_env_ids_R[env_should_remain_R]\n last_obs_RO = last_obs_RO[env_should_remain_R]\n last_info_R = last_info_R[env_should_remain_R]\n if collect_action_computation_batch_R.hidden_state is not None:\n last_hidden_state_RH = last_hidden_state_RH[env_should_remain_R] # type: ignore[index]\n\n if (n_step and step_count >= n_step) or (\n n_episode and num_collected_episodes >= n_episode\n ):\n break\n\n # Check if we screwed up somewhere\n if self.raise_on_nan_in_buffer and self.buffer.hasnull():\n nan_batch = self.buffer.isnull().apply_values_transform(np.sum)\n\n raise MalformedBufferError(\n \"NaN detected in the buffer. You can drop them with `buffer.dropnull()`. \"\n \"This error is most often caused by an incorrect use of `EpisodeRolloutHooks`\"\n \"together with the `n_steps` (instead of `n_episodes`) option, or by \"\n \"an incorrect implementation of `StepHook`.\"\n \"Here an overview of the number of NaNs per field: \\n\"\n f\"{nan_batch}\",\n )\n\n # Step 14\n # update instance-lifetime counters, different from collect_stats\n self.collect_step += step_count\n self.collect_episode += num_collected_episodes\n\n # Step 15\n if n_step:\n # persist for future collect iterations\n self._pre_collect_obs_RO = last_obs_RO\n self._pre_collect_info_R = last_info_R\n self._pre_collect_hidden_state_RH = last_hidden_state_RH\n elif n_episode:\n # reset envs and the _pre_collect fields\n self.reset_env(gym_reset_kwargs) # todo still necessary?\n return collect_stats\n\n @staticmethod\n def _reset_hidden_state_based_on_type(\n env_ind_local_D: np.ndarray,\n last_hidden_state_RH: np.ndarray | torch.Tensor | Batch | None,\n ) -> None:\n if isinstance(last_hidden_state_RH, torch.Tensor):\n last_hidden_state_RH[env_ind_local_D].zero_() # type: ignore[index]\n elif isinstance(last_hidden_state_RH, np.ndarray):\n last_hidden_state_RH[env_ind_local_D] = (\n None if last_hidden_state_RH.dtype == object else 0\n )\n elif isinstance(last_hidden_state_RH, Batch):\n last_hidden_state_RH.empty_(env_ind_local_D)\n # todo is this inplace magic and just working?\n\n\nclass AsyncCollector(Collector[CollectStats]):\n \"\"\"Async Collector handles async vector environment.\n\n Please refer to :class:`~tianshou.data.Collector` for a more detailed explanation.\n \"\"\"\n\n def __init__(\n self,\n policy: Policy | Algorithm,\n env: BaseVectorEnv,\n buffer: ReplayBuffer | None = None,\n exploration_noise: bool = False,\n raise_on_nan_in_buffer: bool = True,\n ) -> None:\n if not env.is_async:\n # TODO: raise an exception?\n log.error(\n f\"Please use {Collector.__name__} if not using async venv. \"\n f\"Env class: {env.__class__.__name__}\",\n )\n # assert env.is_async\n warnings.warn(\"Using async setting may collect extra transitions into buffer.\")\n super().__init__(\n policy,\n env,\n buffer,\n exploration_noise,\n collect_stats_class=CollectStats,\n raise_on_nan_in_buffer=raise_on_nan_in_buffer,\n )\n # E denotes the number of parallel environments: self.env_num\n # At init, E=R but during collection R <= E\n # Keep in sync with reset!\n self._ready_env_ids_R: np.ndarray = np.arange(self.env_num)\n self._current_obs_in_all_envs_EO: np.ndarray | None = copy(self._pre_collect_obs_RO)\n self._current_info_in_all_envs_E: np.ndarray | None = copy(self._pre_collect_info_R)\n self._current_hidden_state_in_all_envs_EH: np.ndarray | torch.Tensor | Batch | None = copy(\n self._pre_collect_hidden_state_RH,\n )\n self._current_action_in_all_envs_EA: np.ndarray = np.empty(self.env_num)\n self._current_policy_in_all_envs_E: Batch | None = None\n\n def reset(\n self,\n reset_buffer: bool = True,\n reset_stats: bool = True,\n gym_reset_kwargs: dict[str, Any] | None = None,\n ) -> tuple[np.ndarray, np.ndarray]:\n \"\"\"Reset the environment, statistics, and data needed to start the collection.\n\n :param reset_buffer: if true, reset the replay buffer attached\n to the collector.\n :param reset_stats: if true, reset the statistics attached to the collector.\n :param gym_reset_kwargs: extra keyword arguments to pass into the environment's\n reset function. Defaults to None (extra keyword arguments)\n :return: The initial observation and info from the environment.\n \"\"\"\n # This sets the _pre_collect attrs\n result = super().reset(\n reset_buffer=reset_buffer,\n reset_stats=reset_stats,\n gym_reset_kwargs=gym_reset_kwargs,\n )\n # Keep in sync with init!\n self._ready_env_ids_R = np.arange(self.env_num)\n # E denotes the number of parallel environments self.env_num\n self._current_obs_in_all_envs_EO = copy(self._pre_collect_obs_RO)\n self._current_info_in_all_envs_E = copy(self._pre_collect_info_R)\n self._current_hidden_state_in_all_envs_EH = copy(self._pre_collect_hidden_state_RH)\n self._current_action_in_all_envs_EA = np.empty(self.env_num)\n self._current_policy_in_all_envs_E = None\n return result\n\n @override\n def reset_env(\n self,\n gym_reset_kwargs: dict[str, Any] | None = None,\n ) -> tuple[np.ndarray, np.ndarray]:\n # we need to step through the envs and wait until they are ready to be able to interact with them\n if self.env.waiting_id:\n self.env.step(None, id=self.env.waiting_id)\n return super().reset_env(gym_reset_kwargs=gym_reset_kwargs)\n\n @override\n def _collect(\n self,\n n_step: int | None = None,\n n_episode: int | None = None,\n random: bool = False,\n render: float | None = None,\n gym_reset_kwargs: dict[str, Any] | None = None,\n ) -> CollectStats:\n start_time = time.time()\n\n step_count = 0\n num_collected_episodes = 0\n episode_returns: list[float] = []\n episode_lens: list[int] = []\n episode_start_indices: list[int] = []\n\n ready_env_ids_R = self._ready_env_ids_R\n # last_obs_RO= self._current_obs_in_all_envs_EO[ready_env_ids_R] # type: ignore[index]\n # last_info_R = self._current_info_in_all_envs_E[ready_env_ids_R] # type: ignore[index]\n # last_hidden_state_RH = self._current_hidden_state_in_all_envs_EH[ready_env_ids_R] # type: ignore[index]\n # last_obs_RO = self._pre_collect_obs_RO\n # last_info_R = self._pre_collect_info_R\n # last_hidden_state_RH = self._pre_collect_hidden_state_RH\n if self._current_obs_in_all_envs_EO is None or self._current_info_in_all_envs_E is None:\n raise RuntimeError(\n \"Current obs or info array is None, did you call reset or pass reset_at_collect=True?\",\n )\n\n last_obs_RO = self._current_obs_in_all_envs_EO[ready_env_ids_R]\n last_info_R = self._current_info_in_all_envs_E[ready_env_ids_R]\n last_hidden_state_RH = _nullable_slice(\n self._current_hidden_state_in_all_envs_EH,\n ready_env_ids_R,\n )\n # Each iteration of the AsyncCollector is only stepping a subset of the\n # envs. The last observation/ hidden state of the ones not included in\n # the current iteration has to be retained. This is done by copying the\n while True:\n # todo do we need this?\n # todo extend to all current attributes but some could be None at init\n if self._current_obs_in_all_envs_EO is None:\n raise RuntimeError(\n \"Current obs is None, did you call reset or pass reset_at_collect=True?\",\n )\n if (\n not len(self._current_obs_in_all_envs_EO)\n == len(self._current_action_in_all_envs_EA)\n == self.env_num\n ): # major difference\n raise RuntimeError(\n f\"{len(self._current_obs_in_all_envs_EO)=} and\"\n f\"{len(self._current_action_in_all_envs_EA)=} have to equal\"\n f\" {self.env_num=} as it tracks the current transition\"\n f\"in all envs\",\n )\n\n # get the next action\n collect_batch_R = self._compute_action_policy_hidden(\n random=random,\n ready_env_ids_R=ready_env_ids_R,\n last_obs_RO=last_obs_RO,\n last_info_R=last_info_R,\n last_hidden_state_RH=last_hidden_state_RH,\n )\n\n # save act_RA/policy_R/ hidden_state_RH before env.step\n self._current_action_in_all_envs_EA[ready_env_ids_R] = collect_batch_R.act\n if self._current_policy_in_all_envs_E:\n self._current_policy_in_all_envs_E[ready_env_ids_R] = collect_batch_R.policy_entry\n else:\n self._current_policy_in_all_envs_E = collect_batch_R.policy_entry # first iteration\n if collect_batch_R.hidden_state is not None:\n if self._current_hidden_state_in_all_envs_EH is not None:\n # Need to cast since if it's a Tensor, the assignment might in fact fail if hidden_state_RH is not\n # a tensor as well. This is hard to express with proper typing, even using @overload, so we cheat\n # and hope that if one of the two is a tensor, the other one is as well.\n self._current_hidden_state_in_all_envs_EH = cast(\n np.ndarray | Batch,\n self._current_hidden_state_in_all_envs_EH,\n )\n self._current_hidden_state_in_all_envs_EH[ready_env_ids_R] = (\n collect_batch_R.hidden_state\n )\n else:\n self._current_hidden_state_in_all_envs_EH = collect_batch_R.hidden_state\n\n # step in env\n obs_next_RO, rew_R, terminated_R, truncated_R, info_R = self.env.step(\n collect_batch_R.act_normalized,\n ready_env_ids_R,\n )\n done_R = np.logical_or(terminated_R, truncated_R)\n # Not all environments of the AsyncCollector might have performed a step in this iteration.\n # Change batch_of_envs_with_step_in_this_iteration here to reflect that ready_env_ids_R has changed.\n # This means especially that R is potentially changing every iteration\n try:\n ready_env_ids_R = cast(np.ndarray, info_R[\"env_id\"])\n # TODO: don't use bare Exception!\n except Exception:\n ready_env_ids_R = np.array([i[\"env_id\"] for i in info_R])\n\n current_iteration_batch = cast(\n RolloutBatchProtocol,\n Batch(\n obs=self._current_obs_in_all_envs_EO[ready_env_ids_R],\n act=self._current_action_in_all_envs_EA[ready_env_ids_R],\n policy=self._current_policy_in_all_envs_E[ready_env_ids_R],\n obs_next=obs_next_RO,\n rew=rew_R,\n terminated=terminated_R,\n truncated=truncated_R,\n done=done_R,\n info=info_R,\n ),\n )\n\n if render:\n self.env.render()\n if render > 0 and not np.isclose(render, 0):\n time.sleep(render)\n\n # add data into the buffer\n ptr_R, ep_rew_R, ep_len_R, ep_idx_R = self.buffer.add(\n current_iteration_batch,\n buffer_ids=ready_env_ids_R,\n )\n\n # collect statistics\n num_episodes_done_this_iter = np.sum(done_R)\n step_count += len(ready_env_ids_R)\n num_collected_episodes += num_episodes_done_this_iter\n\n # preparing for the next iteration\n # todo seem we can get rid of this last_sth stuff altogether\n last_obs_RO = copy(obs_next_RO)\n last_info_R = copy(info_R)\n last_hidden_state_RH = copy(\n self._current_hidden_state_in_all_envs_EH[ready_env_ids_R], # type: ignore[index]\n )\n if num_episodes_done_this_iter:\n env_ind_local_D = np.where(done_R)[0]\n env_ind_global_D = ready_env_ids_R[env_ind_local_D]\n episode_lens.extend(ep_len_R[env_ind_local_D])\n episode_returns.extend(ep_rew_R[env_ind_local_D])\n episode_start_indices.extend(ep_idx_R[env_ind_local_D])\n\n # now we copy obs_next_RO to obs, but since there might be\n # finished episodes, we have to reset finished envs first.\n gym_reset_kwargs = gym_reset_kwargs or {}\n obs_reset_DO, info_reset_D = self.env.reset(\n env_id=env_ind_global_D,\n **gym_reset_kwargs,\n )\n last_obs_RO[env_ind_local_D] = obs_reset_DO\n last_info_R[env_ind_local_D] = info_reset_D\n\n self._reset_hidden_state_based_on_type(env_ind_local_D, last_hidden_state_RH)\n\n # update based on the current transition in all envs\n self._current_obs_in_all_envs_EO[ready_env_ids_R] = last_obs_RO\n # this is a list, so loop over\n for idx, ready_env_id in enumerate(ready_env_ids_R):\n self._current_info_in_all_envs_E[ready_env_id] = last_info_R[idx]\n if self._current_hidden_state_in_all_envs_EH is not None:\n # Need to cast since if it's a Tensor, the assignment might in fact fail if hidden_state_RH is not\n # a tensor as well. This is hard to express with proper typing, even using @overload, so we cheat\n # and hope that if one of the two is a tensor, the other one is as well.\n self._current_hidden_state_in_all_envs_EH = cast(\n np.ndarray | Batch,\n self._current_hidden_state_in_all_envs_EH,\n )\n self._current_hidden_state_in_all_envs_EH[ready_env_ids_R] = last_hidden_state_RH\n else:\n self._current_hidden_state_in_all_envs_EH = last_hidden_state_RH\n\n if (n_step and step_count >= n_step) or (\n n_episode and num_collected_episodes >= n_episode\n ):\n break\n\n # generate statistics\n self.collect_step += step_count\n self.collect_episode += num_collected_episodes\n collect_time = max(time.time() - start_time, 1e-9)\n self.collect_time += collect_time\n\n # persist for future collect iterations\n self._ready_env_ids_R = ready_env_ids_R\n\n return CollectStats.with_autogenerated_stats(\n returns=np.array(episode_returns),\n lens=np.array(episode_lens),\n n_collected_episodes=num_collected_episodes,\n n_collected_steps=step_count,\n )\n\n\nclass StepHookProtocol(Protocol):\n \"\"\"A protocol for step hooks.\"\"\"\n\n def __call__(\n self,\n action_batch: CollectActionBatchProtocol,\n rollout_batch: RolloutBatchProtocol,\n ) -> None:\n \"\"\"The function to call when the hook is executed.\"\"\"\n ...\n\n\nclass StepHook(StepHookProtocol, ABC):\n \"\"\"Marker interface for step hooks.\n\n All step hooks in Tianshou will inherit from it, but only the corresponding protocol will be\n used in type hints. This makes it possible to discover all hooks in the codebase by looking up\n the hierarchy of this class (or the protocol itself) while still allowing the user to pass\n something like a lambda function as a hook.\n \"\"\"\n\n @abstractmethod\n def __call__(\n self,\n action_batch: CollectActionBatchProtocol,\n rollout_batch: RolloutBatchProtocol,\n ) -> None: ...\n\n\nclass StepHookAddActionDistribution(StepHook):\n \"\"\"Adds the action distribution to the collected rollout batch under the field \"action_dist\".\n\n The field is also accessible as class variable `ACTION_DIST_KEY`.\n This hook be useful for algorithms that need the previously taken actions for training, like variants of\n imitation learning or DAGGER.\n \"\"\"\n\n ACTION_DIST_KEY = \"action_dist\"\n\n def __call__(\n self,\n action_batch: CollectActionBatchProtocol,\n rollout_batch: RolloutBatchProtocol,\n ) -> None:\n rollout_batch[self.ACTION_DIST_KEY] = action_batch.dist\n\n\nclass EpisodeRolloutHookProtocol(Protocol):\n \"\"\"A protocol for hooks (functions) that act on an entire collected episode.\n\n Can be used to add values to the buffer that are only known after the episode is finished.\n A prime example is something like the MC return to go.\n \"\"\"\n\n def __call__(self, episode_batch: EpisodeBatchProtocol) -> dict[str, np.ndarray] | None:\n \"\"\"Will be called by the collector when an episode is finished.\n\n If a dictionary is returned, the key-value pairs will be interpreted as new entries\n to be added to the episode batch (inside the buffer). In that case,\n the values should be arrays of the same length as the input `rollout_batch`.\n\n :param episode_batch: the batch of transitions that belong to the episode.\n :return: an optional dictionary containing new entries (of same len as `rollout_batch`)\n to be added to the buffer.\n \"\"\"\n ...\n\n\nclass EpisodeRolloutHook(EpisodeRolloutHookProtocol, ABC):\n \"\"\"Marker interface for episode hooks.\n\n All episode hooks in Tianshou will inherit from it, but only the corresponding protocol will be\n used in type hints. This makes it possible to discover all hooks in the codebase by looking up\n the hierarchy of this class (or the protocol itself) while still allowing the user to pass\n something like a lambda function as a hook.\n \"\"\"\n\n @abstractmethod\n def __call__(self, episode_batch: EpisodeBatchProtocol) -> dict[str, np.ndarray] | None: ...\n\n\nclass EpisodeRolloutHookMCReturn(EpisodeRolloutHook):\n \"\"\"Adds the MC return to go as well as the full episode MC return to the transitions in the buffer.\n\n The latter will be constant for all transitions in the same episode and simply corresponds to\n the initial MC return to go. Useful for algorithms that rely on the monte carlo returns during training.\n \"\"\"\n\n MC_RETURN_TO_GO_KEY = \"mc_return_to_go\"\n FULL_EPISODE_MC_RETURN_KEY = \"full_episode_mc_return\"\n\n class OutputDict(TypedDict):\n mc_return_to_go: np.ndarray\n full_episode_mc_return: np.ndarray\n\n def __init__(self, gamma: float = 0.99):\n if not 0 <= gamma <= 1:\n raise ValueError(f\"Expected 0 <= gamma <= 1, but got {gamma=}.\")\n self.gamma = gamma\n\n def __call__( # type: ignore[override]\n self,\n episode_batch: RolloutBatchProtocol,\n ) -> \"EpisodeRolloutHookMCReturn.OutputDict\":\n mc_return_to_go = episode_mc_return_to_go(episode_batch.rew, self.gamma)\n full_episode_mc_return = np.full_like(mc_return_to_go, mc_return_to_go[0])\n\n return self.OutputDict(\n mc_return_to_go=mc_return_to_go,\n full_episode_mc_return=full_episode_mc_return,\n )\n\n\nclass EpisodeRolloutHookMerged(EpisodeRolloutHook):\n \"\"\"Combines multiple episode hooks into a single one.\n\n If all hooks return `None`, this hook will also return `None`.\n \"\"\"\n\n def __init__(\n self,\n *episode_rollout_hooks: EpisodeRolloutHookProtocol,\n check_overlapping_keys: bool = True,\n ):\n \"\"\"\n :param episode_rollout_hooks: the hooks to combine\n :param check_overlapping_keys: whether to check for overlapping keys in the output of the hooks and\n raise a `KeyError` if any are found. Set to `False` to disable this check (can be useful\n if this becomes a performance bottleneck).\n \"\"\"\n self.episode_rollout_hooks = episode_rollout_hooks\n self.check_overlapping_keys = check_overlapping_keys\n\n def __call__(self, episode_batch: EpisodeBatchProtocol) -> dict[str, np.ndarray] | None:\n result: dict[str, np.ndarray] = {}\n for rollout_hook in self.episode_rollout_hooks:\n new_entries = rollout_hook(episode_batch)\n if new_entries is None:\n continue\n\n if self.check_overlapping_keys and (\n duplicated_entries := set(new_entries).difference(result)\n ):\n raise KeyError(\n f\"Combined rollout hook {rollout_hook} leads to previously \"\n f\"computed entries that would be overwritten: {duplicated_entries=}. \"\n f\"Consider combining hooks which will deliver non-overlapping entries to solve this.\",\n )\n result.update(new_entries)\n if not result:\n return None\n return result\n" }, { "path": "tianshou/data/stats.py", "content": "import logging\nfrom collections.abc import Sequence\nfrom dataclasses import dataclass\nfrom typing import TYPE_CHECKING, Optional\n\nimport numpy as np\n\nfrom tianshou.utils.print import DataclassPPrintMixin\n\nif TYPE_CHECKING:\n from tianshou.algorithm.algorithm_base import TrainingStats\n from tianshou.data import CollectStats, CollectStatsBase\n\nlog = logging.getLogger(__name__)\n\n\n@dataclass(kw_only=True)\nclass SequenceSummaryStats(DataclassPPrintMixin):\n \"\"\"A data structure for storing the statistics of a sequence.\"\"\"\n\n mean: float\n std: float\n max: float\n min: float\n\n @classmethod\n def from_sequence(cls, sequence: Sequence[float | int] | np.ndarray) -> \"SequenceSummaryStats\":\n if len(sequence) == 0:\n return cls(mean=0.0, std=0.0, max=0.0, min=0.0)\n\n if hasattr(sequence, \"shape\") and len(sequence.shape) > 1:\n log.warning(\n f\"Sequence has shape {sequence.shape}, but only 1D sequences are supported. \"\n \"Stats will be computed from the flattened sequence. For computing stats \"\n \"for each dimension consider using the function `compute_dim_to_summary_stats`.\",\n )\n\n return cls(\n mean=float(np.mean(sequence)),\n std=float(np.std(sequence)),\n max=float(np.max(sequence)),\n min=float(np.min(sequence)),\n )\n\n @classmethod\n def from_single_value(cls, value: float | int) -> \"SequenceSummaryStats\":\n return cls(mean=value, std=0.0, max=value, min=value)\n\n\ndef compute_dim_to_summary_stats(\n arr: Sequence[Sequence[float]] | np.ndarray,\n) -> dict[int, SequenceSummaryStats]:\n \"\"\"Compute summary statistics for each dimension of a sequence.\n\n :param arr: a 2-dim arr (or sequence of sequences) from which to compute the statistics.\n :return: A dictionary of summary statistics for each dimension.\n \"\"\"\n stats = {}\n for dim, seq in enumerate(arr):\n stats[dim] = SequenceSummaryStats.from_sequence(seq)\n return stats\n\n\n@dataclass(kw_only=True)\nclass TimingStats(DataclassPPrintMixin):\n \"\"\"A data structure for storing timing statistics.\"\"\"\n\n total_time: float = 0.0\n \"\"\"The total time elapsed.\"\"\"\n train_time: float = 0.0\n \"\"\"The total time elapsed for training (collecting samples plus model update).\"\"\"\n train_time_collect: float = 0.0\n \"\"\"The total time elapsed for collecting training transitions.\"\"\"\n train_time_update: float = 0.0\n \"\"\"The total time elapsed for updating models.\"\"\"\n test_time: float = 0.0\n \"\"\"The total time elapsed for testing models.\"\"\"\n update_speed: float = 0.0\n \"\"\"The speed of updating (env_step per second).\"\"\"\n\n\n@dataclass(kw_only=True)\nclass InfoStats(DataclassPPrintMixin):\n \"\"\"A data structure for storing information about the learning process.\"\"\"\n\n update_step: int\n \"\"\"The total number of update steps that have been taken.\"\"\"\n best_score: float\n \"\"\"The best score over the test results. The one with the highest score will be considered the best model.\"\"\"\n best_reward: float\n \"\"\"The best reward over the test results.\"\"\"\n best_reward_std: float\n \"\"\"Standard deviation of the best reward over the test results.\"\"\"\n train_step: int\n \"\"\"The total collected step of training collector.\"\"\"\n train_episode: int\n \"\"\"The total collected episode of training collector.\"\"\"\n test_step: int\n \"\"\"The total collected step of test collector.\"\"\"\n test_episode: int\n \"\"\"The total collected episode of test collector.\"\"\"\n\n timing: TimingStats\n \"\"\"The timing statistics.\"\"\"\n\n\n@dataclass(kw_only=True)\nclass EpochStats(DataclassPPrintMixin):\n \"\"\"A data structure for storing epoch statistics.\"\"\"\n\n epoch: int\n \"\"\"The current epoch.\"\"\"\n\n train_collect_stat: Optional[\"CollectStatsBase\"]\n \"\"\"The statistics of the last call to the training collector.\"\"\"\n test_collect_stat: Optional[\"CollectStats\"]\n \"\"\"The statistics of the last call to the test collector.\"\"\"\n training_stat: Optional[\"TrainingStats\"]\n \"\"\"The statistics of the last model update step.\n Can be None if no model update is performed, typically in the last training iteration.\"\"\"\n info_stat: InfoStats\n \"\"\"The information of the collector.\"\"\"\n" }, { "path": "tianshou/data/types.py", "content": "from typing import Protocol\n\nimport numpy as np\nimport torch\n\nfrom tianshou.data import Batch\nfrom tianshou.data.batch import BatchProtocol, TArr, TObsArr\n\nTObs = TObsArr | BatchProtocol\n\nTNestedDictValue = np.ndarray | dict[str, \"TNestedDictValue\"]\n\n\nclass ObsBatchProtocol(BatchProtocol, Protocol):\n \"\"\"Observations of an environment that a policy can turn into actions.\n\n Typically used inside a policy's forward\n \"\"\"\n\n obs: TObs\n \"\"\"the observations as generated by the environment in `step`.\n If it is a dict env, this will be an instance of Batch, otherwise it will be an array (or tensor if your env returns tensors)\"\"\"\n info: TArr\n \"\"\"array of info dicts generated by the environment in `step`\"\"\"\n\n\nclass RolloutBatchProtocol(ObsBatchProtocol, Protocol):\n \"\"\"Typically, the outcome of sampling from a replay buffer.\"\"\"\n\n obs_next: TObs\n \"\"\"the observations after obs as generated by the environment in `step`.\n If it is a dict env, this will be an instance of Batch, otherwise it will be an array (or tensor if your env returns tensors)\"\"\"\n act: TArr\n rew: np.ndarray\n terminated: TArr\n truncated: TArr\n\n\nclass BatchWithReturnsProtocol(RolloutBatchProtocol, Protocol):\n \"\"\"With added returns, usually computed with GAE.\"\"\"\n\n returns: torch.Tensor\n\n\nclass PrioBatchProtocol(RolloutBatchProtocol, Protocol):\n \"\"\"Contains weights that can be used for prioritized replay.\"\"\"\n\n weight: np.ndarray | torch.Tensor\n \"\"\"can be used for prioritized replay.\"\"\"\n\n\nclass RecurrentStateBatch(BatchProtocol, Protocol):\n \"\"\"Used by RNNs in policies, contains `hidden` and `cell` fields.\"\"\"\n\n hidden: torch.Tensor\n cell: torch.Tensor\n\n\nclass ActBatchProtocol(BatchProtocol, Protocol):\n \"\"\"Simplest batch, just containing the action. Useful e.g., for random policy.\"\"\"\n\n act: TArr\n\n\nclass ActStateBatchProtocol(ActBatchProtocol, Protocol):\n \"\"\"Contains action and state (which can be None), useful for policies that can support RNNs.\"\"\"\n\n state: dict | BatchProtocol | np.ndarray | None\n \"\"\"Hidden state of RNNs, or None if not using RNNs. Used for recurrent policies.\n At the moment support for recurrent is experimental!\"\"\"\n\n\nclass ModelOutputBatchProtocol(ActStateBatchProtocol, Protocol):\n \"\"\"In addition to state and action, contains model output: (logits).\"\"\"\n\n logits: torch.Tensor\n\n\nclass FQFBatchProtocol(ModelOutputBatchProtocol, Protocol):\n \"\"\"Model outputs, fractions and quantiles_tau - specific to the FQF model.\"\"\"\n\n fractions: torch.Tensor\n quantiles_tau: torch.Tensor\n\n\nclass BatchWithAdvantagesProtocol(BatchWithReturnsProtocol, Protocol):\n \"\"\"Contains estimated advantages and values.\n\n Returns are usually computed from GAE of advantages by adding the value.\n \"\"\"\n\n adv: torch.Tensor\n v_s: torch.Tensor\n\n\nclass DistBatchProtocol(ModelOutputBatchProtocol, Protocol):\n \"\"\"Contains dist instances for actions (created by dist_fn).\n\n Usually categorical or normal.\n \"\"\"\n\n dist: torch.distributions.Distribution\n\n\nclass DistLogProbBatchProtocol(DistBatchProtocol, Protocol):\n \"\"\"Contains dist objects that can be sampled from and log_prob of taken action.\"\"\"\n\n log_prob: torch.Tensor\n\n\nclass LogpOldProtocol(BatchWithAdvantagesProtocol, Protocol):\n \"\"\"Contains logp_old, often needed for importance weights, in particular in PPO.\n\n Builds on batches that contain advantages and values.\n \"\"\"\n\n logp_old: torch.Tensor\n\n\nclass QuantileRegressionBatchProtocol(ModelOutputBatchProtocol, Protocol):\n \"\"\"Contains taus for algorithms using quantile regression.\n\n See e.g. https://arxiv.org/abs/1806.06923\n \"\"\"\n\n taus: torch.Tensor\n\n\nclass ImitationBatchProtocol(ModelOutputBatchProtocol, Protocol):\n \"\"\"Similar to other batches, but contains `imitation_logits` and `q_value` fields.\"\"\"\n\n state: dict | Batch | np.ndarray | None\n q_value: torch.Tensor\n imitation_logits: torch.Tensor\n" }, { "path": "tianshou/data/utils/__init__.py", "content": "" }, { "path": "tianshou/data/utils/converter.py", "content": "import pickle\nfrom copy import deepcopy\nfrom numbers import Number\nfrom typing import Any, Union, no_type_check\n\nimport h5py\nimport numpy as np\nimport torch\n\nfrom tianshou.data.batch import Batch, _parse_value\n\n\n# TODO: confusing name, could actually return a batch...\n# Overrides and generic types should be added\n# todo check for ActBatchProtocol\n@no_type_check\ndef to_numpy(x: Any) -> Batch | np.ndarray:\n \"\"\"Return an object without torch.Tensor.\"\"\"\n if isinstance(x, torch.Tensor): # most often case\n return x.detach().cpu().numpy()\n if isinstance(x, np.ndarray): # second often case\n return x\n if isinstance(x, np.number | np.bool_ | Number):\n return np.asanyarray(x)\n if x is None:\n return np.array(None, dtype=object)\n if isinstance(x, dict | Batch):\n x = Batch(x) if isinstance(x, dict) else deepcopy(x)\n x.to_numpy_()\n return x\n if isinstance(x, list | tuple):\n return to_numpy(_parse_value(x))\n # fallback\n return np.asanyarray(x)\n\n\n@no_type_check\ndef to_torch(\n x: Any,\n dtype: torch.dtype | None = None,\n device: str | int | torch.device = \"cpu\",\n) -> Batch | torch.Tensor:\n \"\"\"Return an object without np.ndarray.\"\"\"\n if isinstance(x, np.ndarray) and issubclass(\n x.dtype.type,\n np.bool_ | np.number,\n ): # most often case\n x = torch.from_numpy(x)\n if dtype is not None:\n x = x.type(dtype)\n return x.to(device)\n if isinstance(x, torch.Tensor): # second often case\n if dtype is not None:\n x = x.type(dtype)\n return x.to(device)\n if isinstance(x, np.number | np.bool_ | Number):\n return to_torch(np.asanyarray(x), dtype, device)\n if isinstance(x, dict | Batch):\n x = Batch(x, copy=True) if isinstance(x, dict) else deepcopy(x)\n x.to_torch_(dtype, device)\n return x\n if isinstance(x, list | tuple):\n return to_torch(_parse_value(x), dtype, device)\n # fallback\n raise TypeError(f\"object {x} cannot be converted to torch.\")\n\n\n@no_type_check\ndef to_torch_as(x: Any, y: torch.Tensor) -> Batch | torch.Tensor:\n \"\"\"Return an object without np.ndarray.\n\n Same as ``to_torch(x, dtype=y.dtype, device=y.device)``.\n \"\"\"\n assert isinstance(y, torch.Tensor)\n return to_torch(x, dtype=y.dtype, device=y.device)\n\n\n# Note: object is used as a proxy for objects that can be pickled\n# Note: mypy does not support cyclic definition currently\nHdf5ConvertibleValues = Union[\n int,\n float,\n Batch,\n np.ndarray,\n torch.Tensor,\n object,\n \"Hdf5ConvertibleType\",\n]\n\nHdf5ConvertibleType = dict[str, Hdf5ConvertibleValues]\n\n\ndef to_hdf5(x: Hdf5ConvertibleType, y: h5py.Group, compression: str | None = None) -> None:\n \"\"\"Copy object into HDF5 group.\"\"\"\n\n def to_hdf5_via_pickle(\n x: object,\n y: h5py.Group,\n key: str,\n compression: str | None = None,\n ) -> None:\n \"\"\"Pickle, convert to numpy array and write to HDF5 dataset.\"\"\"\n data = np.frombuffer(pickle.dumps(x), dtype=np.byte)\n y.create_dataset(key, data=data, compression=compression)\n\n for k, v in x.items():\n if isinstance(v, Batch | dict):\n # dicts and batches are both represented by groups\n subgrp = y.create_group(k)\n if isinstance(v, Batch):\n subgrp_data = v.__getstate__()\n subgrp.attrs[\"__data_type__\"] = \"Batch\"\n else:\n subgrp_data = v\n to_hdf5(subgrp_data, subgrp, compression=compression)\n elif isinstance(v, torch.Tensor):\n # PyTorch tensors are written to datasets\n y.create_dataset(k, data=to_numpy(v), compression=compression)\n y[k].attrs[\"__data_type__\"] = \"Tensor\"\n elif isinstance(v, np.ndarray):\n try:\n # NumPy arrays are written to datasets\n y.create_dataset(k, data=v, compression=compression)\n y[k].attrs[\"__data_type__\"] = \"ndarray\"\n except TypeError:\n # If data type is not supported by HDF5 fall back to pickle.\n # This happens if dtype=object (e.g. due to entries being None)\n # and possibly in other cases like structured arrays.\n try:\n to_hdf5_via_pickle(v, y, k, compression=compression)\n except Exception as exception:\n raise RuntimeError(\n f\"Attempted to pickle {v.__class__.__name__} due to \"\n \"data type not supported by HDF5 and failed.\",\n ) from exception\n y[k].attrs[\"__data_type__\"] = \"pickled_ndarray\"\n elif isinstance(v, int | float):\n # ints and floats are stored as attributes of groups\n y.attrs[k] = v\n else: # resort to pickle for any other type of object\n try:\n to_hdf5_via_pickle(v, y, k, compression=compression)\n except Exception as exception:\n raise NotImplementedError(\n f\"No conversion to HDF5 for object of type '{type(v)}' \"\n \"implemented and fallback to pickle failed.\",\n ) from exception\n y[k].attrs[\"__data_type__\"] = v.__class__.__name__\n\n\ndef from_hdf5(x: h5py.Group, device: str | None = None) -> Hdf5ConvertibleValues:\n \"\"\"Restore object from HDF5 group.\"\"\"\n if isinstance(x, h5py.Dataset):\n # handle datasets\n if x.attrs[\"__data_type__\"] == \"ndarray\":\n return np.array(x)\n if x.attrs[\"__data_type__\"] == \"Tensor\":\n return torch.tensor(x, device=device)\n return pickle.loads(x[()])\n # handle groups representing a dict or a Batch\n y = dict(x.attrs.items())\n data_type = y.pop(\"__data_type__\", None)\n for k, v in x.items():\n y[k] = from_hdf5(v, device)\n return Batch(y) if data_type == \"Batch\" else y\n" }, { "path": "tianshou/data/utils/segtree.py", "content": "import numpy as np\nfrom numba import njit\n\n\nclass SegmentTree:\n \"\"\"Implementation of Segment Tree.\n\n The segment tree stores an array ``arr`` with size ``n``. It supports value\n update and fast query of the sum for the interval ``[left, right)`` in\n O(log n) time. The detailed procedure is as follows:\n\n 1. Pad the array to have length of power of 2, so that leaf nodes in the \\\n segment tree have the same depth.\n 2. Store the segment tree in a binary heap.\n\n :param size: the size of segment tree.\n \"\"\"\n\n def __init__(self, size: int) -> None:\n bound = 1\n while bound < size:\n bound *= 2\n self._size = size\n self._bound = bound\n self._value = np.zeros([bound * 2])\n self._compile()\n\n def __len__(self) -> int:\n return self._size\n\n def __getitem__(self, index: int | np.ndarray) -> float | np.ndarray:\n \"\"\"Return self[index].\"\"\"\n return self._value[index + self._bound]\n\n def __setitem__(self, index: int | np.ndarray, value: float | np.ndarray) -> None:\n \"\"\"Update values in segment tree.\n\n Duplicate values in ``index`` are handled by numpy: later index\n overwrites previous ones.\n ::\n\n >>> a = np.array([1, 2, 3, 4])\n >>> a[[0, 1, 0, 1]] = [4, 5, 6, 7]\n >>> print(a)\n [6 7 3 4]\n \"\"\"\n if isinstance(index, int):\n index, value = np.array([index]), np.array([value])\n assert np.all(index >= 0)\n assert np.all(index < self._size)\n _setitem(self._value, index + self._bound, value)\n\n def reduce(self, start: int = 0, end: int | None = None) -> float:\n \"\"\"Return operation(value[start:end]).\"\"\"\n if start == 0 and end is None:\n return self._value[1]\n if end is None:\n end = self._size\n if end < 0:\n end += self._size\n return _reduce(self._value, start + self._bound - 1, end + self._bound)\n\n def get_prefix_sum_idx(self, value: float | np.ndarray) -> int | np.ndarray:\n r\"\"\"Find the index with given value.\n\n Return the minimum index for each ``v`` in ``value`` so that\n :math:`v \\le \\mathrm{sums}_i`, where\n :math:`\\mathrm{sums}_i = \\sum_{j = 0}^{i} \\mathrm{arr}_j`.\n\n .. warning::\n\n Please make sure all of the values inside the segment tree are\n non-negative when using this function.\n \"\"\"\n assert np.all(value >= 0.0)\n assert np.all(value < self._value[1])\n single = False\n if not isinstance(value, np.ndarray):\n value = np.array([value])\n single = True\n index = _get_prefix_sum_idx(value, self._bound, self._value)\n return index.item() if single else index\n\n def _compile(self) -> None:\n f64 = np.array([0, 1], dtype=np.float64)\n f32 = np.array([0, 1], dtype=np.float32)\n i64 = np.array([0, 1], dtype=np.int64)\n _setitem(f64, i64, f64)\n _setitem(f64, i64, f32)\n _reduce(f64, 0, 1)\n _get_prefix_sum_idx(f64, 1, f64)\n _get_prefix_sum_idx(f32, 1, f64)\n\n\n@njit\ndef _setitem(tree: np.ndarray, index: np.ndarray, value: np.ndarray) -> None:\n \"\"\"Numba version, 4x faster: 0.1 -> 0.024.\"\"\"\n tree[index] = value\n while index[0] > 1:\n index //= 2\n tree[index] = tree[index * 2] + tree[index * 2 + 1]\n\n\n@njit\ndef _reduce(tree: np.ndarray, start: int, end: int) -> float:\n \"\"\"Numba version, 2x faster: 0.009 -> 0.005.\"\"\"\n # nodes in (start, end) should be aggregated\n result = 0.0\n while end - start > 1: # (start, end) interval is not empty\n if start % 2 == 0:\n result += tree[start + 1]\n start //= 2\n if end % 2 == 1:\n result += tree[end - 1]\n end //= 2\n return result\n\n\n@njit\ndef _get_prefix_sum_idx(value: np.ndarray, bound: int, sums: np.ndarray) -> np.ndarray:\n \"\"\"Numba version (v0.51), 5x speed up with size=100000 and bsz=64.\n\n vectorized np: 0.0923 (numpy best) -> 0.024 (now)\n for-loop: 0.2914 -> 0.019 (but not so stable)\n \"\"\"\n index = np.ones(value.shape, dtype=np.int64)\n while index[0] < bound:\n index *= 2\n lsons = sums[index]\n direct = lsons < value\n value -= lsons * direct\n index += direct\n index -= bound\n return index\n" }, { "path": "tianshou/env/__init__.py", "content": "\"\"\"Env package.\"\"\"\n\nfrom tianshou.env.gym_wrappers import (\n ContinuousToDiscrete,\n MultiDiscreteToDiscrete,\n TruncatedAsTerminated,\n)\nfrom tianshou.env.pettingzoo_env import PettingZooEnv\nfrom tianshou.env.venv_wrappers import VectorEnvNormObs, VectorEnvWrapper\nfrom tianshou.env.venvs import (\n BaseVectorEnv,\n DummyVectorEnv,\n RayVectorEnv,\n ShmemVectorEnv,\n SubprocVectorEnv,\n)\n\n__all__ = [\n \"BaseVectorEnv\",\n \"ContinuousToDiscrete\",\n \"DummyVectorEnv\",\n \"MultiDiscreteToDiscrete\",\n \"PettingZooEnv\",\n \"RayVectorEnv\",\n \"ShmemVectorEnv\",\n \"SubprocVectorEnv\",\n \"TruncatedAsTerminated\",\n \"VectorEnvNormObs\",\n \"VectorEnvWrapper\",\n]\n" }, { "path": "tianshou/env/atari/atari_network.py", "content": "from collections.abc import Callable, Sequence\nfrom typing import Any, TypeVar\n\nimport numpy as np\nimport torch\nfrom torch import nn\n\nfrom tianshou.algorithm.modelfree.reinforce import TDistFnDiscrOrCont\nfrom tianshou.data import Batch\nfrom tianshou.data.types import TObs\nfrom tianshou.highlevel.env import Environments\nfrom tianshou.highlevel.module.actor import ActorFactory\nfrom tianshou.highlevel.module.core import (\n TDevice,\n)\nfrom tianshou.highlevel.module.intermediate import (\n IntermediateModule,\n IntermediateModuleFactory,\n)\nfrom tianshou.highlevel.params.dist_fn import DistributionFunctionFactoryCategorical\nfrom tianshou.utils.net.common import (\n ActionReprNetWithVectorOutput,\n)\nfrom tianshou.utils.net.discrete import DiscreteActor, NoisyLinear\nfrom tianshou.utils.torch_utils import torch_device\n\n\ndef layer_init(layer: nn.Module, std: float = np.sqrt(2), bias_const: float = 0.0) -> nn.Module:\n \"\"\"TODO.\"\"\"\n torch.nn.init.orthogonal_(layer.weight, std)\n torch.nn.init.constant_(layer.bias, bias_const)\n return layer\n\n\nT = TypeVar(\"T\")\n\n\nclass ScaledObsInputActionReprNet(ActionReprNetWithVectorOutput):\n def __init__(self, module: ActionReprNetWithVectorOutput, denom: float = 255.0) -> None:\n super().__init__(module.get_output_dim())\n self.module = module\n self.denom = denom\n\n def forward(\n self,\n obs: TObs,\n state: T | None = None,\n info: dict[str, T] | None = None,\n ) -> tuple[torch.Tensor | Sequence[torch.Tensor], T | None]:\n if info is None:\n info = {}\n scaler = lambda arr: arr / self.denom\n if isinstance(obs, Batch):\n scaled_obs = obs.apply_values_transform(scaler)\n else:\n scaled_obs = scaler(obs)\n return self.module.forward(scaled_obs, state, info)\n\n\nclass DQNet(ActionReprNetWithVectorOutput[Any]):\n \"\"\"Reference: Human-level control through deep reinforcement learning.\"\"\"\n\n def __init__(\n self,\n c: int,\n h: int,\n w: int,\n action_shape: Sequence[int] | int,\n features_only: bool = False,\n output_dim_added_layer: int | None = None,\n layer_init: Callable[[nn.Module], nn.Module] = lambda x: x,\n ) -> None:\n # TODO: Add docstring\n if not features_only and output_dim_added_layer is not None:\n raise ValueError(\n \"Should not provide explicit output dimension using `output_dim_added_layer` when `features_only` is true.\",\n )\n net = nn.Sequential(\n layer_init(nn.Conv2d(c, 32, kernel_size=8, stride=4)),\n nn.ReLU(inplace=True),\n layer_init(nn.Conv2d(32, 64, kernel_size=4, stride=2)),\n nn.ReLU(inplace=True),\n layer_init(nn.Conv2d(64, 64, kernel_size=3, stride=1)),\n nn.ReLU(inplace=True),\n nn.Flatten(),\n )\n with torch.no_grad():\n base_cnn_output_dim = int(np.prod(net(torch.zeros(1, c, h, w)).shape[1:]))\n if not features_only:\n action_dim = int(np.prod(action_shape))\n net = nn.Sequential(\n net,\n layer_init(nn.Linear(base_cnn_output_dim, 512)),\n nn.ReLU(inplace=True),\n layer_init(nn.Linear(512, action_dim)),\n )\n output_dim = action_dim\n elif output_dim_added_layer is not None:\n net = nn.Sequential(\n net,\n layer_init(nn.Linear(base_cnn_output_dim, output_dim_added_layer)),\n nn.ReLU(inplace=True),\n )\n output_dim = output_dim_added_layer\n else:\n output_dim = base_cnn_output_dim\n super().__init__(output_dim)\n self.net = net\n\n def forward(\n self,\n obs: TObs,\n state: T | None = None,\n info: dict[str, T] | None = None,\n ) -> tuple[torch.Tensor, T | None]:\n r\"\"\"Mapping: s -> Q(s, \\*).\n\n For more info, see docstring of parent.\n \"\"\"\n device = torch_device(self)\n obs = torch.as_tensor(obs, device=device, dtype=torch.float32)\n return self.net(obs), state\n\n\nclass C51Net(DQNet):\n \"\"\"Reference: A distributional perspective on reinforcement learning.\"\"\"\n\n def __init__(\n self,\n *,\n c: int,\n h: int,\n w: int,\n action_shape: Sequence[int],\n num_atoms: int = 51,\n ) -> None:\n self.action_num = int(np.prod(action_shape))\n super().__init__(c=c, h=h, w=w, action_shape=[self.action_num * num_atoms])\n self.num_atoms = num_atoms\n\n def forward(\n self,\n obs: TObs,\n state: T | None = None,\n info: dict[str, T] | None = None,\n ) -> tuple[torch.Tensor, T | None]:\n r\"\"\"Mapping: x -> Z(x, \\*).\"\"\"\n obs, state = super().forward(obs)\n obs = obs.view(-1, self.num_atoms).softmax(dim=-1)\n obs = obs.view(-1, self.action_num, self.num_atoms)\n return obs, state\n\n\nclass RainbowNet(DQNet):\n \"\"\"Reference: Rainbow: Combining Improvements in Deep Reinforcement Learning.\"\"\"\n\n def __init__(\n self,\n *,\n c: int,\n h: int,\n w: int,\n action_shape: Sequence[int],\n num_atoms: int = 51,\n noisy_std: float = 0.5,\n is_dueling: bool = True,\n is_noisy: bool = True,\n ) -> None:\n super().__init__(c=c, h=h, w=w, action_shape=action_shape, features_only=True)\n self.action_num = int(np.prod(action_shape))\n self.num_atoms = num_atoms\n\n def linear(x: int, y: int) -> NoisyLinear | nn.Linear:\n if is_noisy:\n return NoisyLinear(x, y, noisy_std)\n return nn.Linear(x, y)\n\n self.Q = nn.Sequential(\n linear(self.output_dim, 512),\n nn.ReLU(inplace=True),\n linear(512, self.action_num * self.num_atoms),\n )\n self._is_dueling = is_dueling\n if self._is_dueling:\n self.V = nn.Sequential(\n linear(self.output_dim, 512),\n nn.ReLU(inplace=True),\n linear(512, self.num_atoms),\n )\n self.output_dim = self.action_num * self.num_atoms\n\n def forward(\n self,\n obs: TObs,\n state: T | None = None,\n info: dict[str, Any] | None = None,\n ) -> tuple[torch.Tensor, T | None]:\n obs, state = super().forward(obs)\n q = self.Q(obs)\n q = q.view(-1, self.action_num, self.num_atoms)\n if self._is_dueling:\n v = self.V(obs)\n v = v.view(-1, 1, self.num_atoms)\n logits = q - q.mean(dim=1, keepdim=True) + v\n else:\n logits = q\n probs = logits.softmax(dim=2)\n return probs, state\n\n\nclass QRDQNet(DQNet):\n \"\"\"Reference: Distributional Reinforcement Learning with Quantile Regression.\"\"\"\n\n def __init__(\n self,\n *,\n c: int,\n h: int,\n w: int,\n action_shape: Sequence[int] | int,\n num_quantiles: int = 200,\n ) -> None:\n self.action_num = int(np.prod(action_shape))\n super().__init__(c=c, h=h, w=w, action_shape=[self.action_num * num_quantiles])\n self.num_quantiles = num_quantiles\n\n def forward(\n self,\n obs: TObs,\n state: T | None = None,\n info: dict[str, Any] | None = None,\n ) -> tuple[torch.Tensor, T | None]:\n obs, state = super().forward(obs)\n obs = obs.view(-1, self.action_num, self.num_quantiles)\n return obs, state\n\n\nclass ActorFactoryAtariDQN(ActorFactory):\n USE_SOFTMAX_OUTPUT = False\n\n def __init__(\n self,\n scale_obs: bool = True,\n features_only: bool = False,\n output_dim_added_layer: int | None = None,\n ) -> None:\n self.output_dim_added_layer = output_dim_added_layer\n self.scale_obs = scale_obs\n self.features_only = features_only\n\n def create_module(self, envs: Environments, device: TDevice) -> DiscreteActor:\n c, h, w = envs.get_observation_shape() # type: ignore # only right shape is a sequence of length 3\n action_shape = envs.get_action_shape()\n if isinstance(action_shape, np.int64):\n action_shape = int(action_shape)\n net: DQNet | ScaledObsInputActionReprNet\n net = DQNet(\n c=c,\n h=h,\n w=w,\n action_shape=action_shape,\n features_only=self.features_only,\n output_dim_added_layer=self.output_dim_added_layer,\n layer_init=layer_init,\n )\n if self.scale_obs:\n net = ScaledObsInputActionReprNet(net)\n return DiscreteActor(\n preprocess_net=net,\n action_shape=envs.get_action_shape(),\n softmax_output=self.USE_SOFTMAX_OUTPUT,\n ).to(device)\n\n def create_dist_fn(self, envs: Environments) -> TDistFnDiscrOrCont | None:\n return DistributionFunctionFactoryCategorical(\n is_probs_input=self.USE_SOFTMAX_OUTPUT,\n ).create_dist_fn(envs)\n\n\nclass IntermediateModuleFactoryAtariDQN(IntermediateModuleFactory):\n def __init__(self, features_only: bool = False, net_only: bool = False) -> None:\n self.features_only = features_only\n self.net_only = net_only\n\n def create_intermediate_module(self, envs: Environments, device: TDevice) -> IntermediateModule:\n obs_shape = envs.get_observation_shape()\n if isinstance(obs_shape, int):\n obs_shape = [obs_shape]\n assert len(obs_shape) == 3\n c, h, w = obs_shape\n action_shape = envs.get_action_shape()\n if isinstance(action_shape, np.int64):\n action_shape = int(action_shape)\n dqn = DQNet(\n c=c,\n h=h,\n w=w,\n action_shape=action_shape,\n features_only=self.features_only,\n ).to(device)\n module = dqn.net if self.net_only else dqn\n return IntermediateModule(module, dqn.output_dim)\n\n\nclass IntermediateModuleFactoryAtariDQNFeatures(IntermediateModuleFactoryAtariDQN):\n def __init__(self) -> None:\n super().__init__(features_only=True, net_only=True)\n" }, { "path": "tianshou/env/atari/atari_wrapper.py", "content": "# Borrow a lot from openai baselines:\n# https://github.com/openai/baselines/blob/master/baselines/common/atari_wrappers.py\nimport logging\nimport warnings\nfrom collections import deque\nfrom typing import Any, SupportsFloat\n\nimport cv2\nimport gymnasium as gym\nimport numpy as np\nfrom gymnasium import Env\n\nfrom tianshou.env import BaseVectorEnv\nfrom tianshou.highlevel.env import (\n EnvFactoryRegistered,\n EnvMode,\n EnvPoolFactory,\n VectorEnvType,\n)\nfrom tianshou.highlevel.trainer import EpochStopCallback, TrainingContext\n\nenvpool_is_available = True\ntry:\n import envpool\nexcept ImportError:\n envpool_is_available = False\n envpool = None\nlog = logging.getLogger(__name__)\n\n\ndef _parse_reset_result(reset_result: tuple) -> tuple[tuple, dict, bool]:\n contains_info = (\n isinstance(reset_result, tuple)\n and len(reset_result) == 2\n and isinstance(reset_result[1], dict)\n )\n if contains_info:\n return reset_result[0], reset_result[1], contains_info\n return reset_result, {}, contains_info\n\n\ndef get_space_dtype(obs_space: gym.spaces.Box) -> type[np.floating] | type[np.integer]:\n \"\"\"TODO.\"\"\"\n obs_space_dtype: type[np.integer] | type[np.floating]\n if np.issubdtype(obs_space.dtype, np.integer):\n obs_space_dtype = np.integer\n elif np.issubdtype(obs_space.dtype, np.floating):\n obs_space_dtype = np.floating\n else:\n raise TypeError(\n f\"Unsupported observation space dtype: {obs_space.dtype}. \"\n f\"This might be a bug in tianshou or gymnasium, please report it!\",\n )\n return obs_space_dtype\n\n\nclass NoopResetEnv(gym.Wrapper):\n \"\"\"Sample initial states by taking random number of no-ops on reset.\n\n No-op is assumed to be action 0.\n\n :param gym.Env env: the environment to wrap.\n :param int noop_max: the maximum value of no-ops to run.\n \"\"\"\n\n def __init__(self, env: gym.Env, noop_max: int = 30) -> None:\n super().__init__(env)\n self.noop_max = noop_max\n self.noop_action = 0\n assert hasattr(env.unwrapped, \"get_action_meanings\")\n assert env.unwrapped.get_action_meanings()[0] == \"NOOP\"\n\n def reset(self, **kwargs: Any) -> tuple[Any, dict[str, Any]]:\n _, info, return_info = _parse_reset_result(self.env.reset(**kwargs))\n noops = self.unwrapped.np_random.integers(1, self.noop_max + 1)\n for _ in range(noops):\n step_result = self.env.step(self.noop_action)\n if len(step_result) == 4:\n obs, rew, done, info = step_result # type: ignore[unreachable] # mypy doesn't know that Gym version <0.26 has only 4 items (no truncation)\n else:\n obs, rew, term, trunc, info = step_result\n done = term or trunc\n if done:\n obs, info, _ = _parse_reset_result(self.env.reset())\n if return_info:\n return obs, info\n return obs, {}\n\n\nclass MaxAndSkipEnv(gym.Wrapper):\n \"\"\"Return only every `skip`-th frame (frameskipping) using most recent raw observations (for max pooling across time steps).\n\n :param gym.Env env: the environment to wrap.\n :param int skip: number of `skip`-th frame.\n \"\"\"\n\n def __init__(self, env: gym.Env, skip: int = 4) -> None:\n super().__init__(env)\n self._skip = skip\n\n def step(self, action: Any) -> tuple[Any, float, bool, bool, dict[str, Any]]:\n \"\"\"Step the environment with the given action.\n\n Repeat action, sum reward, and max over last observations.\n \"\"\"\n obs_list = []\n total_reward = 0.0\n new_step_api = False\n for _ in range(self._skip):\n step_result = self.env.step(action)\n if len(step_result) == 4:\n obs, reward, done, info = step_result # type: ignore[unreachable] # mypy doesn't know that Gym version <0.26 has only 4 items (no truncation)\n else:\n obs, reward, term, trunc, info = step_result\n done = term or trunc\n new_step_api = True\n obs_list.append(obs)\n total_reward += float(reward)\n if done:\n break\n max_frame = np.max(obs_list[-2:], axis=0)\n if new_step_api:\n return max_frame, total_reward, term, trunc, info\n\n return (\n max_frame,\n total_reward,\n done,\n info.get(\"TimeLimit.truncated\", False),\n info,\n )\n\n\nclass EpisodicLifeEnv(gym.Wrapper):\n \"\"\"Make end-of-life == end-of-episode, but only reset on true game over.\n\n It helps the value estimation.\n\n :param gym.Env env: the environment to wrap.\n \"\"\"\n\n def __init__(self, env: gym.Env) -> None:\n super().__init__(env)\n self.lives = 0\n self.was_real_done = True\n self._return_info = False\n\n def step(self, action: Any) -> tuple[Any, float, bool, bool, dict[str, Any]]:\n step_result = self.env.step(action)\n if len(step_result) == 4:\n obs, reward, done, info = step_result # type: ignore[unreachable] # mypy doesn't know that Gym version <0.26 has only 4 items (no truncation)\n new_step_api = False\n else:\n obs, reward, term, trunc, info = step_result\n done = term or trunc\n new_step_api = True\n reward = float(reward)\n self.was_real_done = done\n # check current lives, make loss of life terminal, then update lives to\n # handle bonus lives\n assert hasattr(self.env.unwrapped, \"ale\")\n lives = self.env.unwrapped.ale.lives()\n if 0 < lives < self.lives:\n # for Qbert sometimes we stay in lives == 0 condition for a few\n # frames, so its important to keep lives > 0, so that we only reset\n # once the environment is actually done.\n done = True\n term = True\n self.lives = lives\n if new_step_api:\n return obs, reward, term, trunc, info\n return obs, reward, done, info.get(\"TimeLimit.truncated\", False), info\n\n def reset(self, **kwargs: Any) -> tuple[Any, dict[str, Any]]:\n \"\"\"Calls the Gym environment reset, only when lives are exhausted.\n\n This way all states are still reachable even though lives are episodic, and\n the learner need not know about any of this behind-the-scenes.\n \"\"\"\n if self.was_real_done:\n obs, info, self._return_info = _parse_reset_result(self.env.reset(**kwargs))\n else:\n # no-op step to advance from terminal/lost life state\n step_result = self.env.step(0)\n obs, info = step_result[0], step_result[-1]\n assert hasattr(self.env.unwrapped, \"ale\")\n self.lives = self.env.unwrapped.ale.lives()\n if self._return_info:\n return obs, info\n return obs, {}\n\n\nclass FireResetEnv(gym.Wrapper):\n \"\"\"Take action on reset for environments that are fixed until firing.\n\n Related discussion: https://github.com/openai/baselines/issues/240.\n\n :param gym.Env env: the environment to wrap.\n \"\"\"\n\n def __init__(self, env: gym.Env) -> None:\n super().__init__(env)\n assert hasattr(env.unwrapped, \"get_action_meanings\")\n assert env.unwrapped.get_action_meanings()[1] == \"FIRE\"\n assert len(env.unwrapped.get_action_meanings()) >= 3\n\n def reset(self, **kwargs: Any) -> tuple[Any, dict]:\n _, _, return_info = _parse_reset_result(self.env.reset(**kwargs))\n obs = self.env.step(1)[0]\n return obs, {}\n\n\nclass WarpFrame(gym.ObservationWrapper):\n \"\"\"Warp frames to 84x84 as done in the Nature paper and later work.\n\n :param gym.Env env: the environment to wrap.\n \"\"\"\n\n def __init__(self, env: gym.Env) -> None:\n super().__init__(env)\n self.size = 84\n obs_space = env.observation_space\n assert isinstance(obs_space, gym.spaces.Box)\n obs_space_dtype = get_space_dtype(obs_space)\n self.observation_space = gym.spaces.Box(\n low=np.min(obs_space.low),\n high=np.max(obs_space.high),\n shape=(self.size, self.size),\n dtype=obs_space_dtype,\n )\n\n def observation(self, frame: np.ndarray) -> np.ndarray:\n \"\"\"Returns the current observation from a frame.\"\"\"\n frame = cv2.cvtColor(frame, cv2.COLOR_RGB2GRAY)\n return cv2.resize(frame, (self.size, self.size), interpolation=cv2.INTER_AREA)\n\n\nclass ScaledFloatFrame(gym.ObservationWrapper):\n \"\"\"Normalize observations to 0~1.\n\n :param gym.Env env: the environment to wrap.\n \"\"\"\n\n def __init__(self, env: gym.Env) -> None:\n super().__init__(env)\n obs_space = env.observation_space\n assert isinstance(obs_space, gym.spaces.Box)\n low = np.min(obs_space.low)\n high = np.max(obs_space.high)\n self.bias = low\n self.scale = high - low\n self.observation_space = gym.spaces.Box(\n low=0.0,\n high=1.0,\n shape=obs_space.shape,\n dtype=np.float32,\n )\n\n def observation(self, observation: np.ndarray) -> np.ndarray:\n return (observation - self.bias) / self.scale\n\n\nclass ClipRewardEnv(gym.RewardWrapper):\n \"\"\"clips the reward to {+1, 0, -1} by its sign.\n\n :param gym.Env env: the environment to wrap.\n \"\"\"\n\n def __init__(self, env: gym.Env) -> None:\n super().__init__(env)\n self.reward_range = (-1, 1)\n\n def reward(self, reward: SupportsFloat) -> int:\n \"\"\"Bin reward to {+1, 0, -1} by its sign. Note: np.sign(0) == 0.\"\"\"\n return np.sign(float(reward))\n\n\nclass FrameStack(gym.Wrapper):\n \"\"\"Stack n_frames last frames.\n\n :param gym.Env env: the environment to wrap.\n :param int n_frames: the number of frames to stack.\n \"\"\"\n\n def __init__(self, env: gym.Env, n_frames: int) -> None:\n super().__init__(env)\n self.n_frames: int = n_frames\n self.frames: deque[tuple[Any, ...]] = deque([], maxlen=n_frames)\n obs_space = env.observation_space\n obs_space_shape = env.observation_space.shape\n assert obs_space_shape is not None\n shape = (n_frames, *obs_space_shape)\n assert isinstance(obs_space, gym.spaces.Box)\n obs_space_dtype = get_space_dtype(obs_space)\n self.observation_space = gym.spaces.Box(\n low=np.min(obs_space.low),\n high=np.max(obs_space.high),\n shape=shape,\n dtype=obs_space_dtype,\n )\n\n def reset(self, **kwargs: Any) -> tuple[np.ndarray, dict]:\n obs, info, return_info = _parse_reset_result(self.env.reset(**kwargs))\n for _ in range(self.n_frames):\n self.frames.append(obs)\n return (self._get_ob(), info) if return_info else (self._get_ob(), {})\n\n def step(self, action: Any) -> tuple[np.ndarray, float, bool, bool, dict[str, Any]]:\n step_result = self.env.step(action)\n done: bool\n if len(step_result) == 4:\n obs, reward, done, info = step_result # type: ignore[unreachable] # mypy doesn't know that Gym version <0.26 has only 4 items (no truncation)\n new_step_api = False\n else:\n obs, reward, term, trunc, info = step_result\n new_step_api = True\n self.frames.append(obs)\n reward = float(reward)\n if new_step_api:\n return self._get_ob(), reward, term, trunc, info\n return (\n self._get_ob(),\n reward,\n done,\n info.get(\"TimeLimit.truncated\", False),\n info,\n )\n\n def _get_ob(self) -> np.ndarray:\n # the original wrapper use `LazyFrames` but since we use np buffer,\n # it has no effect\n return np.stack(self.frames, axis=0)\n\n\ndef wrap_deepmind(\n env: gym.Env,\n episode_life: bool = True,\n clip_rewards: bool = True,\n frame_stack: int = 4,\n scale: bool = False,\n warp_frame: bool = True,\n) -> (\n MaxAndSkipEnv\n | EpisodicLifeEnv\n | FireResetEnv\n | WarpFrame\n | ScaledFloatFrame\n | ClipRewardEnv\n | FrameStack\n):\n \"\"\"Configure environment for DeepMind-style Atari.\n\n The observation is channel-first: (c, h, w) instead of (h, w, c).\n\n :param env: the Atari environment to wrap.\n :param bool episode_life: wrap the episode life wrapper.\n :param bool clip_rewards: wrap the reward clipping wrapper.\n :param int frame_stack: wrap the frame stacking wrapper.\n :param bool scale: wrap the scaling observation wrapper.\n :param bool warp_frame: wrap the grayscale + resize observation wrapper.\n :return: the wrapped atari environment.\n \"\"\"\n env = NoopResetEnv(env, noop_max=30)\n env = MaxAndSkipEnv(env, skip=4)\n assert hasattr(env.unwrapped, \"get_action_meanings\") # for mypy\n\n wrapped_env: (\n MaxAndSkipEnv\n | EpisodicLifeEnv\n | FireResetEnv\n | WarpFrame\n | ScaledFloatFrame\n | ClipRewardEnv\n | FrameStack\n ) = env\n if episode_life:\n wrapped_env = EpisodicLifeEnv(wrapped_env)\n if \"FIRE\" in env.unwrapped.get_action_meanings():\n wrapped_env = FireResetEnv(wrapped_env)\n if warp_frame:\n wrapped_env = WarpFrame(wrapped_env)\n if scale:\n wrapped_env = ScaledFloatFrame(wrapped_env)\n if clip_rewards:\n wrapped_env = ClipRewardEnv(wrapped_env)\n if frame_stack:\n wrapped_env = FrameStack(wrapped_env, frame_stack)\n return wrapped_env\n\n\ndef make_atari_env(\n task: str,\n seed: int,\n num_training_envs: int,\n num_test_envs: int,\n scale: int | bool = False,\n frame_stack: int = 4,\n) -> tuple[Env, BaseVectorEnv, BaseVectorEnv]:\n \"\"\"Wrapper function for Atari env.\n\n If EnvPool is installed, it will automatically switch to EnvPool's Atari env.\n\n :return: a tuple of (single env, training envs, test envs).\n \"\"\"\n env_factory = AtariEnvFactory(task, frame_stack, scale=bool(scale))\n envs = env_factory.create_envs(num_training_envs, num_test_envs, seed=seed)\n return envs.env, envs.training_envs, envs.test_envs\n\n\nclass AtariEnvFactory(EnvFactoryRegistered):\n def __init__(\n self,\n task: str,\n frame_stack: int,\n scale: bool = False,\n use_envpool_if_available: bool = True,\n venv_type: VectorEnvType = VectorEnvType.SUBPROC_SHARED_MEM_AUTO,\n ) -> None:\n assert \"NoFrameskip\" in task\n self.frame_stack = frame_stack\n self.scale = scale\n envpool_factory = None\n if use_envpool_if_available:\n if envpool_is_available:\n envpool_factory = self.EnvPoolFactoryAtari(self)\n log.info(\"Using envpool, because it available\")\n else:\n log.info(\"Not using envpool, because it is not available\")\n super().__init__(\n task=task,\n venv_type=venv_type,\n envpool_factory=envpool_factory,\n )\n\n def _create_env(self, mode: EnvMode) -> gym.Env:\n env = super()._create_env(mode)\n is_train = mode == EnvMode.TRAINING\n return wrap_deepmind(\n env,\n episode_life=is_train,\n clip_rewards=is_train,\n frame_stack=self.frame_stack,\n scale=self.scale,\n )\n\n class EnvPoolFactoryAtari(EnvPoolFactory):\n \"\"\"Atari-specific envpool creation.\n Since envpool internally handles the functions that are implemented through the wrappers in `wrap_deepmind`,\n it sets the creation keyword arguments accordingly.\n \"\"\"\n\n def __init__(self, parent: \"AtariEnvFactory\") -> None:\n self.parent = parent\n if self.parent.scale:\n warnings.warn(\n \"EnvPool does not include ScaledFloatFrame wrapper, \"\n \"please compensate by scaling inside your network's forward function (e.g. `x = x / 255.0` for Atari)\",\n )\n\n def _transform_task(self, task: str) -> str:\n task = super()._transform_task(task)\n # TODO: Maybe warn user, explain why this is needed\n return task.replace(\"NoFrameskip-v4\", \"-v5\")\n\n def _transform_kwargs(self, kwargs: dict, mode: EnvMode) -> dict:\n kwargs = super()._transform_kwargs(kwargs, mode)\n is_train = mode == EnvMode.TRAINING\n kwargs[\"reward_clip\"] = is_train\n kwargs[\"episodic_life\"] = is_train\n kwargs[\"stack_num\"] = self.parent.frame_stack\n return kwargs\n\n\nclass AtariEpochStopCallback(EpochStopCallback):\n def __init__(self, task: str) -> None:\n self.task = task\n\n def should_stop(self, mean_rewards: float, context: TrainingContext) -> bool:\n env = context.envs.env\n if env.spec and env.spec.reward_threshold:\n return mean_rewards >= env.spec.reward_threshold\n if \"Pong\" in self.task:\n return mean_rewards >= 20\n return False\n" }, { "path": "tianshou/env/gym_wrappers.py", "content": "from typing import Any, SupportsFloat\n\nimport gymnasium as gym\nimport numpy as np\nfrom packaging import version\n\n\nclass ContinuousToDiscrete(gym.ActionWrapper):\n \"\"\"Gym environment wrapper to take discrete action in a continuous environment.\n\n :param gym.Env env: gym environment with continuous action space.\n :param action_per_dim: number of discrete actions in each dimension\n of the action space.\n \"\"\"\n\n def __init__(self, env: gym.Env, action_per_dim: int | list[int]) -> None:\n super().__init__(env)\n assert isinstance(env.action_space, gym.spaces.Box)\n low, high = env.action_space.low, env.action_space.high\n if isinstance(action_per_dim, int):\n action_per_dim = [action_per_dim] * env.action_space.shape[0]\n assert len(action_per_dim) == env.action_space.shape[0]\n self.action_space = gym.spaces.MultiDiscrete(action_per_dim)\n self.mesh = np.array(\n [np.linspace(lo, hi, a) for lo, hi, a in zip(low, high, action_per_dim, strict=True)],\n dtype=object,\n )\n\n def action(self, act: np.ndarray) -> np.ndarray:\n # modify act\n assert len(act.shape) <= 2, f\"Unknown action format with shape {act.shape}.\"\n if len(act.shape) == 1:\n return np.array([self.mesh[i][a] for i, a in enumerate(act)])\n return np.array([[self.mesh[i][a] for i, a in enumerate(a_)] for a_ in act])\n\n\nclass MultiDiscreteToDiscrete(gym.ActionWrapper):\n \"\"\"Gym environment wrapper to take discrete action in multidiscrete environment.\n\n :param gym.Env env: gym environment with multidiscrete action space.\n \"\"\"\n\n def __init__(self, env: gym.Env) -> None:\n super().__init__(env)\n assert isinstance(env.action_space, gym.spaces.MultiDiscrete)\n nvec = env.action_space.nvec\n assert nvec.ndim == 1\n self.bases = np.ones_like(nvec)\n for i in range(1, len(self.bases)):\n self.bases[i] = self.bases[i - 1] * nvec[-i]\n self.action_space = gym.spaces.Discrete(np.prod(nvec))\n\n def action(self, act: np.ndarray) -> np.ndarray:\n converted_act = []\n for b in np.flip(self.bases):\n converted_act.append(act // b)\n act = act % b\n return np.array(converted_act).transpose()\n\n\nclass TruncatedAsTerminated(gym.Wrapper):\n \"\"\"A wrapper that set ``terminated = terminated or truncated`` for ``step()``.\n\n It's intended to use with ``gym.wrappers.TimeLimit``.\n\n :param gym.Env env: gym environment.\n \"\"\"\n\n def __init__(self, env: gym.Env):\n super().__init__(env)\n if not version.parse(gym.__version__) >= version.parse(\"0.26.0\"):\n raise OSError(\n f\"TruncatedAsTerminated is not applicable with gym version \\\n {gym.__version__}\",\n )\n\n def step(self, act: np.ndarray) -> tuple[Any, SupportsFloat, bool, bool, dict[str, Any]]:\n observation, reward, terminated, truncated, info = super().step(act)\n terminated = terminated or truncated\n return observation, reward, terminated, truncated, info\n" }, { "path": "tianshou/env/pettingzoo_env.py", "content": "import warnings\nfrom abc import ABC\nfrom typing import Any\n\nimport pettingzoo\nfrom gymnasium import spaces\nfrom packaging import version\nfrom pettingzoo.utils.env import AECEnv\nfrom pettingzoo.utils.wrappers import BaseWrapper\n\nif version.parse(pettingzoo.__version__) < version.parse(\"1.21.0\"):\n warnings.warn(\n f\"You are using PettingZoo {pettingzoo.__version__}. \"\n f\"Future tianshou versions may not support PettingZoo<1.21.0. \"\n f\"Consider upgrading your PettingZoo version.\",\n DeprecationWarning,\n )\n\n\nclass PettingZooEnv(AECEnv, ABC):\n \"\"\"The interface for petting zoo environments which support multi-agent RL.\n\n Multi-agent environments must be wrapped as\n :class:`~tianshou.env.PettingZooEnv`. Here is the usage:\n ::\n\n env = PettingZooEnv(...)\n # obs is a dict containing obs, agent_id, and mask\n obs = env.reset()\n action = policy(obs)\n obs, rew, trunc, term, info = env.step(action)\n env.close()\n\n The available action's mask is set to True, otherwise it is set to False.\n \"\"\"\n\n def __init__(self, env: BaseWrapper):\n super().__init__()\n self.env = env\n # agent idx list\n self.agents = self.env.possible_agents\n self.agent_idx = {}\n for i, agent_id in enumerate(self.agents):\n self.agent_idx[agent_id] = i\n\n self.rewards = [0] * len(self.agents)\n\n # Get first observation space, assuming all agents have equal space\n self.observation_space: Any = self.env.observation_space(self.agents[0])\n\n # Get first action space, assuming all agents have equal space\n self.action_space: Any = self.env.action_space(self.agents[0])\n\n assert all(\n self.env.observation_space(agent) == self.observation_space for agent in self.agents\n ), (\n \"Observation spaces for all agents must be identical. Perhaps \"\n \"SuperSuit's pad_observations wrapper can help (usage: \"\n \"`supersuit.pad_observations_v0(env)`\"\n )\n\n assert all(self.env.action_space(agent) == self.action_space for agent in self.agents), (\n \"Action spaces for all agents must be identical. Perhaps \"\n \"SuperSuit's pad_action_space wrapper can help (usage: \"\n \"`supersuit.pad_action_space_v0(env)`\"\n )\n\n self.reset()\n\n def reset(self, *args: Any, **kwargs: Any) -> tuple[dict, dict]:\n self.env.reset(*args, **kwargs)\n\n observation, reward, terminated, truncated, info = self.env.last(self)\n\n if isinstance(observation, dict) and \"action_mask\" in observation:\n observation_dict = {\n \"agent_id\": self.env.agent_selection,\n \"obs\": observation[\"observation\"],\n \"mask\": [obm == 1 for obm in observation[\"action_mask\"]],\n }\n else:\n if isinstance(self.action_space, spaces.Discrete):\n observation_dict = {\n \"agent_id\": self.env.agent_selection,\n \"obs\": observation,\n \"mask\": [True] * self.env.action_space(self.env.agent_selection).n,\n }\n else:\n observation_dict = {\n \"agent_id\": self.env.agent_selection,\n \"obs\": observation,\n }\n\n return observation_dict, info\n\n def step(self, action: Any) -> tuple[dict, list[int], bool, bool, dict]:\n self.env.step(action)\n\n observation, rew, term, trunc, info = self.env.last()\n\n if isinstance(observation, dict) and \"action_mask\" in observation:\n obs = {\n \"agent_id\": self.env.agent_selection,\n \"obs\": observation[\"observation\"],\n \"mask\": [obm == 1 for obm in observation[\"action_mask\"]],\n }\n else:\n if isinstance(self.action_space, spaces.Discrete):\n obs = {\n \"agent_id\": self.env.agent_selection,\n \"obs\": observation,\n \"mask\": [True] * self.env.action_space(self.env.agent_selection).n,\n }\n else:\n obs = {\"agent_id\": self.env.agent_selection, \"obs\": observation}\n\n for agent_id, reward in self.env.rewards.items():\n self.rewards[self.agent_idx[agent_id]] = reward\n return obs, self.rewards, term, trunc, info\n\n def close(self) -> None:\n self.env.close()\n\n def seed(self, seed: Any = None) -> None:\n try:\n self.env.seed(seed)\n except (NotImplementedError, AttributeError):\n self.env.reset(seed=seed)\n\n def render(self) -> Any:\n return self.env.render()\n" }, { "path": "tianshou/env/utils.py", "content": "from typing import Any\n\nimport cloudpickle\nimport gymnasium\nimport numpy as np\n\nfrom tianshou.env.pettingzoo_env import PettingZooEnv\n\nENV_TYPE = gymnasium.Env | PettingZooEnv\n\ngym_new_venv_step_type = tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray, np.ndarray]\n\n\nclass CloudpickleWrapper:\n \"\"\"A cloudpickle wrapper used in SubprocVectorEnv.\"\"\"\n\n def __init__(self, data: Any) -> None:\n self.data = data\n\n def __getstate__(self) -> str:\n return cloudpickle.dumps(self.data)\n\n def __setstate__(self, data: str) -> None:\n self.data = cloudpickle.loads(data)\n" }, { "path": "tianshou/env/venv_wrappers.py", "content": "from typing import Any\n\nimport numpy as np\nimport torch\n\nfrom tianshou.env.utils import gym_new_venv_step_type\nfrom tianshou.env.venvs import GYM_RESERVED_KEYS, BaseVectorEnv\nfrom tianshou.utils import RunningMeanStd\n\n\nclass VectorEnvWrapper(BaseVectorEnv):\n \"\"\"Base class for vectorized environments wrapper.\"\"\"\n\n # Note: No super call because this is a wrapper with overridden __getattribute__\n # It's not a \"true\" subclass of BaseVectorEnv but it does extend its interface, so\n # it can be used as a drop-in replacement\n # noinspection PyMissingConstructor\n def __init__(self, venv: BaseVectorEnv) -> None:\n self.venv = venv\n self.is_async = venv.is_async\n\n def __len__(self) -> int:\n return len(self.venv)\n\n def __getattribute__(self, key: str) -> Any:\n if key in GYM_RESERVED_KEYS: # reserved keys in gym.Env\n return getattr(self.venv, key)\n return super().__getattribute__(key)\n\n def get_env_attr(\n self,\n key: str,\n id: int | list[int] | np.ndarray | None = None,\n ) -> list[Any]:\n return self.venv.get_env_attr(key, id)\n\n def set_env_attr(\n self,\n key: str,\n value: Any,\n id: int | list[int] | np.ndarray | None = None,\n ) -> None:\n return self.venv.set_env_attr(key, value, id)\n\n def reset(\n self,\n env_id: int | list[int] | np.ndarray | None = None,\n **kwargs: Any,\n ) -> tuple[np.ndarray, np.ndarray]:\n return self.venv.reset(env_id, **kwargs)\n\n def step(\n self,\n action: np.ndarray | torch.Tensor | None,\n id: int | list[int] | np.ndarray | None = None,\n ) -> gym_new_venv_step_type:\n return self.venv.step(action, id)\n\n def seed(self, seed: int | list[int] | None = None) -> list[list[int] | None]:\n return self.venv.seed(seed)\n\n def render(self, **kwargs: Any) -> list[Any]:\n return self.venv.render(**kwargs)\n\n def close(self) -> None:\n self.venv.close()\n\n\nclass VectorEnvNormObs(VectorEnvWrapper):\n \"\"\"An observation normalization wrapper for vectorized environments.\n\n :param update_obs_rms: whether to update obs_rms. Default to True.\n \"\"\"\n\n def __init__(self, venv: BaseVectorEnv, update_obs_rms: bool = True) -> None:\n super().__init__(venv)\n # initialize observation running mean/std\n self.update_obs_rms = update_obs_rms\n self.obs_rms = RunningMeanStd()\n\n def reset(\n self,\n env_id: int | list[int] | np.ndarray | None = None,\n **kwargs: Any,\n ) -> tuple[np.ndarray, np.ndarray]:\n obs, info = self.venv.reset(env_id, **kwargs)\n\n if isinstance(obs, tuple): # type: ignore\n raise TypeError(\n \"Tuple observation space is not supported. \",\n \"Please change it to array or dict space\",\n )\n\n if self.obs_rms and self.update_obs_rms:\n self.obs_rms.update(obs)\n obs = self._norm_obs(obs)\n return obs, info\n\n def step(\n self,\n action: np.ndarray | torch.Tensor | None,\n id: int | list[int] | np.ndarray | None = None,\n ) -> gym_new_venv_step_type:\n step_results = self.venv.step(action, id)\n if self.obs_rms and self.update_obs_rms:\n self.obs_rms.update(step_results[0])\n return (self._norm_obs(step_results[0]), *step_results[1:])\n\n def _norm_obs(self, obs: np.ndarray) -> np.ndarray:\n if self.obs_rms:\n return self.obs_rms.norm(obs) # type: ignore\n return obs\n\n def set_obs_rms(self, obs_rms: RunningMeanStd) -> None:\n \"\"\"Set with given observation running mean/std.\"\"\"\n self.obs_rms = obs_rms\n\n def get_obs_rms(self) -> RunningMeanStd:\n \"\"\"Return observation running mean/std.\"\"\"\n return self.obs_rms\n" }, { "path": "tianshou/env/venvs.py", "content": "from collections.abc import Callable, Sequence\nfrom typing import Any, Literal\n\nimport gymnasium as gym\nimport numpy as np\nimport torch\n\nfrom tianshou.env.utils import ENV_TYPE, gym_new_venv_step_type\nfrom tianshou.env.worker import (\n DummyEnvWorker,\n EnvWorker,\n RayEnvWorker,\n SubprocEnvWorker,\n)\n\nGYM_RESERVED_KEYS = [\n \"metadata\",\n \"reward_range\",\n \"spec\",\n \"action_space\",\n \"observation_space\",\n]\n\n\nclass BaseVectorEnv:\n \"\"\"Base class for vectorized environments.\n\n Usage:\n ::\n\n env_num = 8\n envs = DummyVectorEnv([lambda: gym.make(task) for _ in range(env_num)])\n assert len(envs) == env_num\n\n It accepts a list of environment generators. In other words, an environment\n generator ``efn`` of a specific task means that ``efn()`` returns the\n environment of the given task, for example, ``gym.make(task)``.\n\n All of the VectorEnv must inherit :class:`~tianshou.env.BaseVectorEnv`.\n Here are some other usages:\n ::\n\n envs.seed(2) # which is equal to the next line\n envs.seed([2, 3, 4, 5, 6, 7, 8, 9]) # set specific seed for each env\n obs = envs.reset() # reset all environments\n obs = envs.reset([0, 5, 7]) # reset 3 specific environments\n obs, rew, done, info = envs.step([1] * 8) # step synchronously\n envs.render() # render all environments\n envs.close() # close all environments\n\n .. warning::\n\n If you use your own environment, please make sure the ``seed`` method\n is set up properly, e.g.,\n ::\n\n def seed(self, seed):\n np.random.seed(seed)\n\n Otherwise, the outputs of these envs may be the same with each other.\n\n :param env_fns: a list of callable envs, ``env_fns[i]()`` generates the i-th env.\n :param worker_fn: a callable worker, ``worker_fn(env_fns[i])`` generates a\n worker which contains the i-th env.\n :param wait_num: use in asynchronous simulation if the time cost of\n ``env.step`` varies with time and synchronously waiting for all\n environments to finish a step is time-wasting. In that case, we can\n return when ``wait_num`` environments finish a step and keep on\n simulation in these environments. If ``None``, asynchronous simulation\n is disabled; else, ``1 <= wait_num <= env_num``.\n :param timeout: use in asynchronous simulation same as above, in each\n vectorized step it only deal with those environments spending time\n within ``timeout`` seconds.\n \"\"\"\n\n def __init__(\n self,\n env_fns: Sequence[Callable[[], ENV_TYPE]],\n worker_fn: Callable[[Callable[[], ENV_TYPE]], EnvWorker],\n wait_num: int | None = None,\n timeout: float | None = None,\n ) -> None:\n self._env_fns = env_fns\n # A VectorEnv contains a pool of EnvWorkers, which corresponds to\n # interact with the given envs (one worker <-> one env).\n self.workers = [worker_fn(fn) for fn in env_fns]\n self.worker_class = type(self.workers[0])\n assert issubclass(self.worker_class, EnvWorker)\n assert all(isinstance(w, self.worker_class) for w in self.workers)\n\n self.env_num = len(env_fns)\n self.wait_num = wait_num or len(env_fns)\n assert 1 <= self.wait_num <= len(env_fns), (\n f\"wait_num should be in [1, {len(env_fns)}], but got {wait_num}\"\n )\n self.timeout = timeout\n assert self.timeout is None or self.timeout > 0, (\n f\"timeout is {timeout}, it should be positive if provided!\"\n )\n self.is_async = self.wait_num != len(env_fns) or timeout is not None\n self.waiting_conn: list[EnvWorker] = []\n # environments in self.ready_id is actually ready\n # but environments in self.waiting_id are just waiting when checked,\n # and they may be ready now, but this is not known until we check it\n # in the step() function\n self.waiting_id: list[int] = []\n # all environments are ready in the beginning\n self.ready_id = list(range(self.env_num))\n self.is_closed = False\n\n def _assert_is_not_closed(self) -> None:\n assert not self.is_closed, (\n f\"Methods of {self.__class__.__name__} cannot be called after close.\"\n )\n\n def __len__(self) -> int:\n \"\"\"Return len(self), which is the number of environments.\"\"\"\n return self.env_num\n\n def __getattribute__(self, key: str) -> Any:\n \"\"\"Switch the attribute getter depending on the key.\n\n Any class who inherits ``gym.Env`` will inherit some attributes, like\n ``action_space``. However, we would like the attribute lookup to go straight\n into the worker (in fact, this vector env's action_space is always None).\n \"\"\"\n if key in GYM_RESERVED_KEYS: # reserved keys in gym.Env\n return self.get_env_attr(key)\n return super().__getattribute__(key)\n\n def get_env_attr(\n self,\n key: str,\n id: int | list[int] | np.ndarray | None = None,\n ) -> list[Any]:\n \"\"\"Get an attribute from the underlying environments.\n\n If id is an int, retrieve the attribute denoted by key from the environment\n underlying the worker at index id. The result is returned as a list with one\n element. Otherwise, retrieve the attribute for all workers at indices id and\n return a list that is ordered correspondingly to id.\n\n :param str key: The key of the desired attribute.\n :param id: Indice(s) of the desired worker(s). Default to None for all env_id.\n\n :return list: The list of environment attributes.\n \"\"\"\n self._assert_is_not_closed()\n id = self._wrap_id(id)\n if self.is_async:\n self._assert_id(id)\n\n return [self.workers[j].get_env_attr(key) for j in id]\n\n def set_env_attr(\n self,\n key: str,\n value: Any,\n id: int | list[int] | np.ndarray | None = None,\n ) -> None:\n \"\"\"Set an attribute in the underlying environments.\n\n If id is an int, set the attribute denoted by key from the environment\n underlying the worker at index id to value.\n Otherwise, set the attribute for all workers at indices id.\n\n :param str key: The key of the desired attribute.\n :param Any value: The new value of the attribute.\n :param id: Indice(s) of the desired worker(s). Default to None for all env_id.\n \"\"\"\n self._assert_is_not_closed()\n id = self._wrap_id(id)\n if self.is_async:\n self._assert_id(id)\n for j in id:\n self.workers[j].set_env_attr(key, value)\n\n def _wrap_id(\n self,\n id: int | list[int] | np.ndarray | None = None,\n ) -> list[int] | np.ndarray:\n if id is None:\n return list(range(self.env_num))\n return [id] if np.isscalar(id) else id # type: ignore\n\n def _assert_id(self, id: list[int] | np.ndarray) -> None:\n for i in id:\n assert i not in self.waiting_id, (\n f\"Cannot interact with environment {i} which is stepping now.\"\n )\n assert i in self.ready_id, f\"Can only interact with ready environments {self.ready_id}.\"\n\n # TODO: for now, has to be kept in sync with reset in EnvPoolMixin\n # In particular, can't rename env_id to env_ids\n def reset(\n self,\n env_id: int | list[int] | np.ndarray | None = None,\n **kwargs: Any,\n ) -> tuple[np.ndarray, np.ndarray]:\n \"\"\"Reset the state of some envs and return initial observations.\n\n If id is None, reset the state of all the environments and return\n initial observations, otherwise reset the specific environments with\n the given id, either an int or a list.\n \"\"\"\n self._assert_is_not_closed()\n env_id = self._wrap_id(env_id)\n if self.is_async:\n self._assert_id(env_id)\n\n # send(None) == reset() in worker\n for id in env_id:\n self.workers[id].send(None, **kwargs)\n ret_list = [self.workers[id].recv() for id in env_id]\n\n assert (\n isinstance(ret_list[0], tuple | list)\n and len(ret_list[0]) == 2\n and isinstance(ret_list[0][1], dict)\n ), \"The environment does not adhere to the Gymnasium's API.\"\n\n obs_list = [r[0] for r in ret_list]\n\n if isinstance(obs_list[0], tuple): # type: ignore\n raise TypeError(\n \"Tuple observation space is not supported. \",\n \"Please change it to array or dict space\",\n )\n try:\n obs = np.stack(obs_list)\n except ValueError: # different len(obs)\n obs = np.array(obs_list, dtype=object)\n\n infos = np.array([r[1] for r in ret_list])\n return obs, infos\n\n def step(\n self,\n action: np.ndarray | torch.Tensor | None,\n id: int | list[int] | np.ndarray | None = None,\n ) -> gym_new_venv_step_type:\n \"\"\"Run one timestep of some environments' dynamics.\n\n If id is None, run one timestep of all the environments` dynamics;\n otherwise run one timestep for some environments with given id, either\n an int or a list. When the end of episode is reached, you are\n responsible for calling reset(id) to reset this environment`s state.\n\n Accept a batch of action and return a tuple (batch_obs, batch_rew,\n batch_done, batch_info) in numpy format.\n\n :param numpy.ndarray action: a batch of action provided by the agent.\n If the venv is async, the action can be None, which will result\n in all arrays in the returned tuple being empty.\n\n :return: A tuple consisting of either:\n\n * ``obs`` a numpy.ndarray, the agent's observation of current environments\n * ``rew`` a numpy.ndarray, the amount of rewards returned after \\\n previous actions\n * ``terminated`` a numpy.ndarray, whether these episodes have been \\\n terminated\n * ``truncated`` a numpy.ndarray, whether these episodes have been truncated\n * ``info`` a numpy.ndarray, contains auxiliary diagnostic \\\n information (helpful for debugging, and sometimes learning)\n\n For the async simulation:\n\n Provide the given action to the environments. The action sequence\n should correspond to the ``id`` argument, and the ``id`` argument\n should be a subset of the ``env_id`` in the last returned ``info``\n (initially they are env_ids of all the environments). If action is\n None, fetch unfinished step() calls instead.\n \"\"\"\n self._assert_is_not_closed()\n id = self._wrap_id(id)\n if not self.is_async:\n if action is None:\n raise ValueError(\"action must be not-None for non-async\")\n assert len(action) == len(id)\n for i, j in enumerate(id):\n self.workers[j].send(action[i])\n result = []\n for j in id:\n env_return = self.workers[j].recv()\n env_return[-1][\"env_id\"] = j\n result.append(env_return)\n else:\n if action is not None:\n self._assert_id(id)\n assert len(action) == len(id)\n for act, env_id in zip(action, id, strict=True):\n self.workers[env_id].send(act)\n self.waiting_conn.append(self.workers[env_id])\n self.waiting_id.append(env_id)\n self.ready_id = [x for x in self.ready_id if x not in id]\n ready_conns: list[EnvWorker] = []\n while not ready_conns:\n ready_conns = self.worker_class.wait(self.waiting_conn, self.wait_num, self.timeout)\n result = []\n for conn in ready_conns:\n waiting_index = self.waiting_conn.index(conn)\n self.waiting_conn.pop(waiting_index)\n env_id = self.waiting_id.pop(waiting_index)\n # env_return can be (obs, reward, done, info) or\n # (obs, reward, terminated, truncated, info)\n env_return = conn.recv()\n env_return[-1][\"env_id\"] = env_id # Add `env_id` to info\n result.append(env_return)\n self.ready_id.append(env_id)\n obs_list, rew_list, term_list, trunc_list, info_list = tuple(zip(*result, strict=True))\n try:\n obs_stack = np.stack(obs_list)\n except ValueError: # different len(obs)\n obs_stack = np.array(obs_list, dtype=object)\n return (\n obs_stack,\n np.stack(rew_list),\n np.stack(term_list),\n np.stack(trunc_list),\n np.stack(info_list),\n )\n\n def seed(self, seed: int | list[int] | None = None) -> list[list[int] | None]:\n \"\"\"Set the seed for all environments.\n\n Accept ``None``, an int (which will extend ``i`` to\n ``[i, i + 1, i + 2, ...]``) or a list.\n\n :return: The list of seeds used in this env's random number generators.\n The first value in the list should be the \"main\" seed, or the value\n which a reproducer pass to \"seed\".\n \"\"\"\n self._assert_is_not_closed()\n seed_list: list[None] | list[int]\n if seed is None:\n seed_list = [seed] * self.env_num\n elif isinstance(seed, int):\n seed_list = [seed + i for i in range(self.env_num)]\n else:\n seed_list = seed\n return [w.seed(s) for w, s in zip(self.workers, seed_list, strict=True)]\n\n def render(self, **kwargs: Any) -> list[Any]:\n \"\"\"Render all of the environments.\"\"\"\n self._assert_is_not_closed()\n if self.is_async and len(self.waiting_id) > 0:\n raise RuntimeError(\n f\"Environments {self.waiting_id} are still stepping, cannot render them now.\",\n )\n return [w.render(**kwargs) for w in self.workers]\n\n def close(self) -> None:\n \"\"\"Close all of the environments.\n\n This function will be called only once (if not, it will be called during\n garbage collected). This way, ``close`` of all workers can be assured.\n \"\"\"\n self._assert_is_not_closed()\n for w in self.workers:\n w.close()\n self.is_closed = True\n\n\nclass DummyVectorEnv(BaseVectorEnv):\n \"\"\"Dummy vectorized environment wrapper, implemented in for-loop.\n\n This has the same interface as true vectorized environment, but the rollout does not happen in parallel.\n So, all workers just wait for each other and the environment is as efficient as using a single environment.\n This can be useful for testing or for demonstration purposes.\n\n A rare use-case would be using vector based interface, but parallelization is not desired\n (e.g. because of too much overhead). However, in such cases one should consider using a single environment.\n\n .. seealso::\n\n Please refer to :class:`~tianshou.env.BaseVectorEnv` for other APIs' usage.\n \"\"\"\n\n def __init__(\n self,\n env_fns: Sequence[Callable[[], ENV_TYPE]],\n wait_num: int | None = None,\n timeout: float | None = None,\n ) -> None:\n super().__init__(env_fns, DummyEnvWorker, wait_num, timeout)\n\n\nclass SubprocVectorEnv(BaseVectorEnv):\n \"\"\"Vectorized environment wrapper based on subprocess.\n\n .. seealso::\n\n Please refer to :class:`~tianshou.env.BaseVectorEnv` for other APIs' usage.\n\n Additional arguments are:\n\n :param share_memory: whether to share memory between the main process and the worker process. Allows for\n shared buffers to exchange observations\n :param context: the context to use for multiprocessing. Usually it's fine to use the default context, but\n `spawn` as well as `fork` can have non-obvious side effects, see for example\n https://github.com/google-deepmind/mujoco/issues/742, or\n https://github.com/Farama-Foundation/Gymnasium/issues/222.\n Consider using 'fork' when using macOS and additional parallelization, for example via joblib.\n Defaults to None, which will use the default system context.\n \"\"\"\n\n def __init__(\n self,\n env_fns: Sequence[Callable[[], ENV_TYPE]],\n wait_num: int | None = None,\n timeout: float | None = None,\n share_memory: bool = False,\n context: Literal[\"fork\", \"spawn\"] | None = None,\n ) -> None:\n def worker_fn(fn: Callable[[], gym.Env]) -> SubprocEnvWorker:\n return SubprocEnvWorker(fn, share_memory=share_memory, context=context)\n\n super().__init__(\n env_fns,\n worker_fn,\n wait_num,\n timeout,\n )\n\n\nclass ShmemVectorEnv(BaseVectorEnv):\n \"\"\"Optimized SubprocVectorEnv with shared buffers to exchange observations.\n\n ShmemVectorEnv has exactly the same API as SubprocVectorEnv.\n\n .. seealso::\n\n Please refer to :class:`~tianshou.env.BaseVectorEnv` for other APIs' usage.\n \"\"\"\n\n def __init__(\n self,\n env_fns: Sequence[Callable[[], ENV_TYPE]],\n wait_num: int | None = None,\n timeout: float | None = None,\n ) -> None:\n def worker_fn(fn: Callable[[], gym.Env]) -> SubprocEnvWorker:\n return SubprocEnvWorker(fn, share_memory=True)\n\n super().__init__(env_fns, worker_fn, wait_num, timeout)\n\n\nclass RayVectorEnv(BaseVectorEnv):\n \"\"\"Vectorized environment wrapper based on ray.\n\n This is a choice to run distributed environments in a cluster.\n\n .. seealso::\n\n Please refer to :class:`~tianshou.env.BaseVectorEnv` for other APIs' usage.\n \"\"\"\n\n def __init__(\n self,\n env_fns: Sequence[Callable[[], ENV_TYPE]],\n wait_num: int | None = None,\n timeout: float | None = None,\n ) -> None:\n try:\n import ray\n except ImportError as exception:\n raise ImportError(\n \"Please install ray to support RayVectorEnv: pip install ray\",\n ) from exception\n if not ray.is_initialized():\n ray.init()\n super().__init__(env_fns, lambda env_fn: RayEnvWorker(env_fn), wait_num, timeout)\n" }, { "path": "tianshou/env/worker/__init__.py", "content": "# isort:skip_file\n# NOTE: Import order is important to avoid circular import errors!\nfrom tianshou.env.worker.worker_base import EnvWorker\nfrom tianshou.env.worker.dummy import DummyEnvWorker\nfrom tianshou.env.worker.ray import RayEnvWorker\nfrom tianshou.env.worker.subproc import SubprocEnvWorker\n\n__all__ = [\n \"DummyEnvWorker\",\n \"EnvWorker\",\n \"RayEnvWorker\",\n \"SubprocEnvWorker\",\n]\n" }, { "path": "tianshou/env/worker/dummy.py", "content": "from collections.abc import Callable\nfrom typing import Any\n\nimport gymnasium as gym\nimport numpy as np\n\nfrom tianshou.env.worker import EnvWorker\n\n\nclass DummyEnvWorker(EnvWorker):\n \"\"\"Dummy worker used in sequential vector environments.\"\"\"\n\n def __init__(self, env_fn: Callable[[], gym.Env]) -> None:\n self.env = env_fn()\n super().__init__(env_fn)\n\n def get_env_attr(self, key: str) -> Any:\n return getattr(self.env.unwrapped, key)\n\n def set_env_attr(self, key: str, value: Any) -> None:\n setattr(self.env.unwrapped, key, value)\n\n def reset(self, **kwargs: Any) -> tuple[np.ndarray, dict]:\n if \"seed\" in kwargs:\n super().seed(kwargs[\"seed\"])\n return self.env.reset(**kwargs)\n\n @staticmethod\n def wait( # type: ignore\n workers: list[\"DummyEnvWorker\"],\n wait_num: int,\n timeout: float | None = None,\n ) -> list[\"DummyEnvWorker\"]:\n # Sequential EnvWorker objects are always ready\n return workers\n\n def send(self, action: np.ndarray | None, **kwargs: Any) -> None:\n if action is None:\n self.result = self.env.reset(**kwargs)\n else:\n self.result = self.env.step(action) # type: ignore\n\n def seed(self, seed: int | None = None) -> list[int] | None:\n super().seed(seed)\n try:\n return self.env.seed(seed) # type: ignore\n except (AttributeError, NotImplementedError):\n self.env.reset(seed=seed)\n return [seed] # type: ignore\n\n def render(self, **kwargs: Any) -> Any:\n return self.env.render(**kwargs)\n\n def close_env(self) -> None:\n self.env.close()\n" }, { "path": "tianshou/env/worker/ray.py", "content": "# mypy: disable-error-code=unused-ignore\nimport contextlib\nfrom collections.abc import Callable\nfrom typing import Any\n\nimport gymnasium as gym\nimport numpy as np\n\nfrom tianshou.env.utils import ENV_TYPE, gym_new_venv_step_type\nfrom tianshou.env.worker import EnvWorker\n\nwith contextlib.suppress(ImportError):\n import ray\n\n\nclass _SetAttrWrapper(gym.Wrapper):\n def set_env_attr(self, key: str, value: Any) -> None:\n setattr(self.env.unwrapped, key, value)\n\n def get_env_attr(self, key: str) -> Any:\n return getattr(self.env, key)\n\n\nclass RayEnvWorker(EnvWorker):\n \"\"\"Ray worker used in RayVectorEnv.\"\"\"\n\n def __init__(\n self,\n env_fn: Callable[[], ENV_TYPE],\n ) -> None: # TODO: is ENV_TYPE actually correct?\n self.env = ray.remote(_SetAttrWrapper).options(num_cpus=0).remote(env_fn()) # type: ignore\n super().__init__(env_fn)\n\n def get_env_attr(self, key: str) -> Any:\n return ray.get(self.env.get_env_attr.remote(key)) # type: ignore\n\n def set_env_attr(self, key: str, value: Any) -> None:\n ray.get(self.env.set_env_attr.remote(key, value)) # type: ignore\n\n def reset(self, **kwargs: Any) -> Any:\n if \"seed\" in kwargs:\n super().seed(kwargs[\"seed\"])\n return ray.get(self.env.reset.remote(**kwargs)) # type: ignore\n\n @staticmethod\n def wait( # type: ignore\n workers: list[\"RayEnvWorker\"],\n wait_num: int,\n timeout: float | None = None,\n ) -> list[\"RayEnvWorker\"]:\n results = [x.result for x in workers]\n ready_results, _ = ray.wait(results, num_returns=wait_num, timeout=timeout) # type: ignore\n return [workers[results.index(result)] for result in ready_results]\n\n def send(self, action: np.ndarray | None, **kwargs: Any) -> None:\n # self.result is actually a handle\n if action is None:\n self.result = self.env.reset.remote(**kwargs) # type: ignore\n else:\n self.result = self.env.step.remote(action) # type: ignore\n\n def recv(self) -> gym_new_venv_step_type:\n return ray.get(self.result) # type: ignore\n\n def seed(self, seed: int | None = None) -> list[int] | None:\n super().seed(seed)\n try:\n return ray.get(self.env.seed.remote(seed)) # type: ignore\n except (AttributeError, NotImplementedError):\n self.env.reset.remote(seed=seed) # type: ignore\n return None\n\n def render(self, **kwargs: Any) -> Any:\n return ray.get(self.env.render.remote(**kwargs)) # type: ignore\n\n def close_env(self) -> None:\n ray.get(self.env.close.remote()) # type: ignore\n" }, { "path": "tianshou/env/worker/subproc.py", "content": "import ctypes\nimport multiprocessing\nimport time\nfrom collections.abc import Callable\nfrom multiprocessing import connection\nfrom multiprocessing.context import BaseContext\nfrom typing import Any, Literal\n\nimport gymnasium as gym\nimport numpy as np\n\nfrom tianshou.env.utils import CloudpickleWrapper, gym_new_venv_step_type\nfrom tianshou.env.worker import EnvWorker\n\n# mypy: disable-error-code=\"unused-ignore\"\n\n\n_NP_TO_CT = {\n np.bool_: ctypes.c_bool,\n np.uint8: ctypes.c_uint8,\n np.uint16: ctypes.c_uint16,\n np.uint32: ctypes.c_uint32,\n np.uint64: ctypes.c_uint64,\n np.int8: ctypes.c_int8,\n np.int16: ctypes.c_int16,\n np.int32: ctypes.c_int32,\n np.int64: ctypes.c_int64,\n np.float32: ctypes.c_float,\n np.float64: ctypes.c_double,\n}\n\n\nclass ShArray:\n \"\"\"Wrapper of multiprocessing Array.\n\n Example usage:\n\n ::\n\n import numpy as np\n import multiprocessing as mp\n from tianshou.env.worker.subproc import ShArray\n ctx = mp.get_context('fork') # set an explicit context\n arr = ShArray(np.dtype(np.float32), (2, 3), ctx)\n arr.save(np.array([[1, 2, 3], [4, 5, 6]], dtype=np.float32))\n print(arr.get())\n\n \"\"\"\n\n def __init__(self, dtype: np.generic, shape: tuple[int], ctx: BaseContext | None) -> None:\n if ctx is None:\n ctx = multiprocessing.get_context()\n self.arr = ctx.Array(_NP_TO_CT[dtype.type], int(np.prod(shape))) # type: ignore\n self.dtype = dtype\n self.shape = shape\n\n def save(self, ndarray: np.ndarray) -> None:\n assert isinstance(ndarray, np.ndarray)\n dst = self.arr.get_obj()\n dst_np = np.frombuffer(dst, dtype=self.dtype).reshape(self.shape) # type: ignore\n np.copyto(dst_np, ndarray)\n\n def get(self) -> np.ndarray:\n obj = self.arr.get_obj()\n return np.frombuffer(obj, dtype=self.dtype).reshape(self.shape) # type: ignore\n\n\ndef _setup_buf(space: gym.Space, ctx: BaseContext) -> dict | tuple | ShArray:\n if isinstance(space, gym.spaces.Dict):\n return {k: _setup_buf(v, ctx) for k, v in space.spaces.items()}\n if isinstance(space, gym.spaces.Tuple):\n assert isinstance(space.spaces, tuple)\n return tuple([_setup_buf(t, ctx) for t in space.spaces])\n return ShArray(space.dtype, space.shape, ctx) # type: ignore\n\n\ndef _worker(\n parent: connection.Connection,\n p: connection.Connection,\n env_fn_wrapper: CloudpickleWrapper,\n obs_bufs: dict | tuple | ShArray | None = None,\n) -> None:\n def _encode_obs(\n obs: dict | tuple | np.ndarray,\n buffer: dict | tuple | ShArray,\n ) -> None:\n if isinstance(buffer, ShArray):\n # if buffer is an ShArray, obs must be array-like\n obs = np.asarray(obs, dtype=buffer.dtype)\n buffer.save(obs)\n elif isinstance(obs, tuple) and isinstance(buffer, tuple):\n for o, b in zip(obs, buffer, strict=True):\n _encode_obs(o, b)\n elif isinstance(obs, dict) and isinstance(buffer, dict):\n for k in obs:\n _encode_obs(obs[k], buffer[k])\n\n parent.close()\n env = env_fn_wrapper.data()\n try:\n while True:\n try:\n cmd, data = p.recv()\n except EOFError: # the pipe has been closed\n p.close()\n break\n if cmd == \"step\":\n env_return = env.step(data)\n if obs_bufs is not None:\n _encode_obs(env_return[0], obs_bufs)\n env_return = (None, *env_return[1:])\n p.send(env_return)\n elif cmd == \"reset\":\n obs, info = env.reset(**data)\n if obs_bufs is not None:\n _encode_obs(obs, obs_bufs)\n obs = None\n p.send((obs, info))\n elif cmd == \"close\":\n p.send(env.close())\n p.close()\n break\n elif cmd == \"render\":\n p.send(env.render(**data) if hasattr(env, \"render\") else None)\n elif cmd == \"seed\":\n if hasattr(env, \"seed\"):\n p.send(env.seed(data))\n else:\n env.action_space.seed(seed=data)\n env.reset(seed=data)\n p.send(None)\n elif cmd == \"getattr\":\n p.send(getattr(env, data) if hasattr(env, data) else None)\n elif cmd == \"setattr\":\n setattr(env.unwrapped, data[\"key\"], data[\"value\"])\n else:\n p.close()\n raise NotImplementedError\n except KeyboardInterrupt:\n p.close()\n\n\nclass SubprocEnvWorker(EnvWorker):\n \"\"\"Subprocess worker used in SubprocVectorEnv and ShmemVectorEnv.\"\"\"\n\n def __init__(\n self,\n env_fn: Callable[[], gym.Env],\n share_memory: bool = False,\n context: BaseContext | Literal[\"fork\", \"spawn\"] | None = None,\n ) -> None:\n if not isinstance(context, BaseContext):\n context = multiprocessing.get_context(context)\n self.parent_remote, self.child_remote = context.Pipe()\n self.share_memory = share_memory\n self.buffer: dict | tuple | ShArray | None = None\n assert hasattr(context, \"Process\") # for mypy\n if self.share_memory:\n dummy = env_fn()\n obs_space = dummy.observation_space\n dummy.close()\n del dummy\n self.buffer = _setup_buf(obs_space, context)\n args = (\n self.parent_remote,\n self.child_remote,\n CloudpickleWrapper(env_fn),\n self.buffer,\n )\n self.process = context.Process(target=_worker, args=args, daemon=True)\n self.process.start()\n self.child_remote.close()\n super().__init__(env_fn)\n\n def get_env_attr(self, key: str) -> Any:\n self.parent_remote.send([\"getattr\", key])\n return self.parent_remote.recv()\n\n def set_env_attr(self, key: str, value: Any) -> None:\n self.parent_remote.send([\"setattr\", {\"key\": key, \"value\": value}])\n\n def _decode_obs(self) -> dict | tuple | np.ndarray:\n def decode_obs(\n buffer: dict | tuple | ShArray | None,\n ) -> dict | tuple | np.ndarray:\n if isinstance(buffer, ShArray):\n return buffer.get()\n if isinstance(buffer, tuple):\n return tuple([decode_obs(b) for b in buffer])\n if isinstance(buffer, dict):\n return {k: decode_obs(v) for k, v in buffer.items()}\n raise NotImplementedError\n\n return decode_obs(self.buffer)\n\n @staticmethod\n def wait( # type: ignore\n workers: list[\"SubprocEnvWorker\"],\n wait_num: int,\n timeout: float | None = None,\n ) -> list[\"SubprocEnvWorker\"]:\n remain_conns = conns = [x.parent_remote for x in workers]\n ready_conns: list[connection.Connection] = []\n remain_time, t1 = timeout, time.time()\n while len(remain_conns) > 0 and len(ready_conns) < wait_num:\n if timeout:\n remain_time = timeout - (time.time() - t1)\n if remain_time <= 0:\n break\n # connection.wait hangs if the list is empty\n new_ready_conns = connection.wait(remain_conns, timeout=remain_time) # type: ignore\n ready_conns.extend(new_ready_conns) # type: ignore\n remain_conns = [conn for conn in remain_conns if conn not in ready_conns] # type: ignore\n return [workers[conns.index(con)] for con in ready_conns] # type: ignore\n\n def send(self, action: np.ndarray | None, **kwargs: Any) -> None:\n if action is None:\n if \"seed\" in kwargs:\n super().seed(kwargs[\"seed\"])\n self.parent_remote.send([\"reset\", kwargs])\n else:\n self.parent_remote.send([\"step\", action])\n\n def recv(self) -> gym_new_venv_step_type | tuple[np.ndarray, dict]:\n result = self.parent_remote.recv()\n if isinstance(result, tuple):\n if len(result) == 2:\n obs, info = result\n if self.share_memory:\n obs = self._decode_obs()\n return obs, info\n obs = result[0]\n if self.share_memory:\n obs = self._decode_obs()\n # TODO: figure out the typing issue, simplify and document this method\n return (obs, *result[1:])\n obs = result\n if self.share_memory:\n obs = self._decode_obs()\n return obs\n\n def reset(self, **kwargs: Any) -> tuple[np.ndarray, dict]:\n if \"seed\" in kwargs:\n super().seed(kwargs[\"seed\"])\n self.parent_remote.send([\"reset\", kwargs])\n\n result = self.parent_remote.recv()\n if isinstance(result, tuple):\n obs, info = result\n if self.share_memory:\n obs = self._decode_obs()\n return obs, info\n obs = result\n if self.share_memory:\n obs = self._decode_obs()\n return obs\n\n def seed(self, seed: int | None = None) -> list[int] | None:\n super().seed(seed)\n self.parent_remote.send([\"seed\", seed])\n return self.parent_remote.recv()\n\n def render(self, **kwargs: Any) -> Any:\n self.parent_remote.send([\"render\", kwargs])\n return self.parent_remote.recv()\n\n def close_env(self) -> None:\n try:\n self.parent_remote.send([\"close\", None])\n # mp may be deleted so it may raise AttributeError\n self.parent_remote.recv()\n self.process.join()\n except (BrokenPipeError, EOFError, AttributeError):\n pass\n # ensure the subproc is terminated\n self.process.terminate()\n" }, { "path": "tianshou/env/worker/worker_base.py", "content": "from abc import ABC, abstractmethod\nfrom collections.abc import Callable\nfrom typing import Any\n\nimport gymnasium as gym\nimport numpy as np\n\nfrom tianshou.env.utils import gym_new_venv_step_type\n\n\nclass EnvWorker(ABC):\n \"\"\"An abstract worker for an environment.\"\"\"\n\n def __init__(self, env_fn: Callable[[], gym.Env]) -> None:\n self._env_fn = env_fn\n self.is_closed = False\n self.result: gym_new_venv_step_type | tuple[np.ndarray, dict]\n self.action_space = self.get_env_attr(\"action_space\")\n self.is_reset = False\n\n @abstractmethod\n def get_env_attr(self, key: str) -> Any:\n pass\n\n @abstractmethod\n def set_env_attr(self, key: str, value: Any) -> None:\n pass\n\n @abstractmethod\n def send(self, action: np.ndarray | None) -> None:\n \"\"\"Send action signal to low-level worker.\n\n When action is None, it indicates sending \"reset\" signal; otherwise\n it indicates \"step\" signal. The paired return value from \"recv\"\n function is determined by such kind of different signal.\n \"\"\"\n\n def recv(self) -> gym_new_venv_step_type | tuple[np.ndarray, dict]:\n \"\"\"Receive result from low-level worker.\n\n If the last \"send\" function sends a NULL action, it only returns a\n single observation; otherwise it returns a tuple of (obs, rew, done,\n info) or (obs, rew, terminated, truncated, info), based on whether\n the environment is using the old step API or the new one.\n \"\"\"\n return self.result\n\n @abstractmethod\n def reset(self, **kwargs: Any) -> tuple[np.ndarray, dict]:\n pass\n\n def step(self, action: np.ndarray) -> gym_new_venv_step_type:\n \"\"\"Perform one timestep of the environment's dynamic.\n\n \"send\" and \"recv\" are coupled in sync simulation, so users only call\n \"step\" function. But they can be called separately in async\n simulation, i.e. someone calls \"send\" first, and calls \"recv\" later.\n \"\"\"\n self.send(action)\n return self.recv() # type: ignore\n\n @staticmethod\n def wait(\n workers: list[\"EnvWorker\"],\n wait_num: int,\n timeout: float | None = None,\n ) -> list[\"EnvWorker\"]:\n \"\"\"Given a list of workers, return those ready ones.\"\"\"\n raise NotImplementedError\n\n def seed(self, seed: int | None = None) -> list[int] | None:\n \"\"\"\n Seeds the environment's action space sampler.\n NOTE: This does *not* seed the environment itself.\n\n :param seed: the random seed\n :return: a list containing the resulting seed used\n \"\"\"\n return self.action_space.seed(seed)\n\n @abstractmethod\n def render(self, **kwargs: Any) -> Any:\n \"\"\"Render the environment.\"\"\"\n\n @abstractmethod\n def close_env(self) -> None:\n pass\n\n def close(self) -> None:\n if self.is_closed:\n return\n self.is_closed = True\n self.close_env()\n" }, { "path": "tianshou/evaluation/__init__.py", "content": "" }, { "path": "tianshou/evaluation/launcher.py", "content": "\"\"\"Provides a basic interface for launching experiments. The API is experimental and subject to change!.\"\"\"\n\nimport logging\nfrom abc import ABC, abstractmethod\nfrom collections.abc import Callable, Sequence\nfrom copy import copy\nfrom dataclasses import asdict, dataclass\nfrom enum import Enum\nfrom typing import TYPE_CHECKING, Literal\n\nfrom joblib import Parallel, delayed\n\nfrom tianshou.data import InfoStats\n\nif TYPE_CHECKING:\n from tianshou.highlevel.experiment import Experiment\n\nlog = logging.getLogger(__name__)\n\n\n@dataclass\nclass JoblibConfig:\n n_jobs: int = -1\n \"\"\"The maximum number of concurrently running jobs. If -1, all CPUs are used.\"\"\"\n backend: Literal[\"loky\", \"multiprocessing\", \"threading\"] | None = \"loky\"\n \"\"\"Allows to hard-code backend, None means it will be inferred automatically.\"\"\"\n verbose: int = 10\n \"\"\"If greater than zero, prints progress messages.\"\"\"\n\n\ndef default_experiment_execution(exp: \"Experiment\") -> InfoStats | None:\n \"\"\"The default execution simply runs the experiment and returns the trainer result.\"\"\"\n return exp.run().trainer_result\n\n\nclass ExpLauncher(ABC):\n \"\"\"Base interface for launching multiple experiments simultaneously.\"\"\"\n\n def __init__(\n self,\n experiment_runner: Callable[\n [\"Experiment\"], InfoStats | None\n ] = default_experiment_execution,\n ):\n \"\"\"\n :param experiment_runner: determines how an experiment is to be executed.\n Overriding the default can be useful, e.g., for using high-level interfaces\n to set up an experiment (or an experiment collection) and tinkering with it prior to execution.\n This need often arises when prototyping with mechanisms that are not yet supported by\n the high-level interfaces.\n Deviation from the default allows arbitrary things to happen during experiment execution,\n so use this option with caution!.\n \"\"\"\n self.experiment_runner = experiment_runner\n\n def get_name(self) -> str:\n \"\"\"Returns the name of the launcher.\"\"\"\n return self.__class__.__name__.replace(\"Launcher\", \"\").lower()\n\n @abstractmethod\n def _launch(self, experiments: Sequence[\"Experiment\"]) -> list[InfoStats | None]:\n \"\"\"Should call `self.experiment_runner` for each experiment in experiments and aggregate the results.\"\"\"\n\n def _safe_execute(self, exp: \"Experiment\") -> InfoStats | None | Literal[\"failed\"]:\n try:\n return self.experiment_runner(exp)\n except BaseException as e:\n log.error(f\"Failed to run experiment {exp}.\", exc_info=e)\n return \"failed\"\n\n @staticmethod\n def _return_from_successful_and_failed_exps(\n successful_exp_stats: list[InfoStats | None],\n failed_exps: list[\"Experiment\"],\n ) -> list[InfoStats | None]:\n if not successful_exp_stats:\n raise RuntimeError(\"All experiments failed, see error logs for more details.\")\n if failed_exps:\n log.error(\n f\"Failed to run the following \"\n f\"{len(failed_exps)}/{len(successful_exp_stats) + len(failed_exps)} experiments: {failed_exps}. \"\n f\"See the logs for more details. \"\n f\"Returning the results of {len(successful_exp_stats)} successful experiments.\",\n )\n return successful_exp_stats\n\n def launch(self, experiments: Sequence[\"Experiment\"]) -> list[InfoStats | None]:\n \"\"\"Will return the results of successfully executed experiments.\n\n If a single experiment is passed, will not use parallelism and run it in the main process.\n Failed experiments will be logged, and a RuntimeError is only raised if all experiments have failed.\n \"\"\"\n if len(experiments) == 1:\n log.info(\n \"A single experiment is being run, will not use parallelism and run it in the main process.\",\n )\n return [self.experiment_runner(experiments[0])]\n return self._launch(experiments)\n\n\nclass SequentialExpLauncher(ExpLauncher):\n \"\"\"Convenience wrapper around a simple for loop to run experiments sequentially.\"\"\"\n\n def _launch(self, experiments: Sequence[\"Experiment\"]) -> list[InfoStats | None]:\n successful_exp_stats = []\n failed_exps = []\n for exp in experiments:\n exp_stats = self._safe_execute(exp)\n if exp_stats == \"failed\":\n failed_exps.append(exp)\n else:\n successful_exp_stats.append(exp_stats)\n # noinspection PyTypeChecker\n return self._return_from_successful_and_failed_exps(successful_exp_stats, failed_exps)\n\n\nclass JoblibExpLauncher(ExpLauncher):\n def __init__(\n self,\n joblib_cfg: JoblibConfig | None = None,\n experiment_runner: Callable[\n [\"Experiment\"], InfoStats | None\n ] = default_experiment_execution,\n ) -> None:\n super().__init__(experiment_runner=experiment_runner)\n self.joblib_cfg = copy(joblib_cfg) if joblib_cfg is not None else JoblibConfig()\n # Joblib's backend is hard-coded to loky since the threading backend produces different results\n # TODO: fix this\n if self.joblib_cfg.backend != \"loky\":\n log.warning(\n f\"Ignoring the user provided joblib backend {self.joblib_cfg.backend} and using loky instead. \"\n f\"The current implementation requires loky to work and will be relaxed soon\",\n )\n self.joblib_cfg.backend = \"loky\"\n\n def _launch(self, experiments: Sequence[\"Experiment\"]) -> list[InfoStats | None]:\n results = Parallel(**asdict(self.joblib_cfg))(\n delayed(self._safe_execute)(exp) for exp in experiments\n )\n successful_exps = []\n failed_exps = []\n for exp, result in zip(experiments, results, strict=True):\n if result == \"failed\":\n failed_exps.append(exp)\n else:\n successful_exps.append(result)\n return self._return_from_successful_and_failed_exps(successful_exps, failed_exps)\n\n\nclass RegisteredExpLauncher(Enum):\n JOBLIB = \"JOBLIB\"\n SEQUENTIAL = \"SEQUENTIAL\"\n\n def create_launcher(self) -> ExpLauncher:\n match self:\n case RegisteredExpLauncher.JOBLIB:\n return JoblibExpLauncher()\n case RegisteredExpLauncher.SEQUENTIAL:\n return SequentialExpLauncher()\n case _:\n raise NotImplementedError(\n f\"Launcher {self} is not yet implemented.\",\n )\n" }, { "path": "tianshou/evaluation/rliable_evaluation.py", "content": "\"\"\"The rliable-evaluation module provides a high-level interface to evaluate the results of an experiment with multiple runs\non different seeds using the rliable library. The API is experimental and subject to change!.\n\"\"\"\n\nimport json\nimport os\nfrom collections.abc import Iterator\nfrom dataclasses import asdict, dataclass, fields\nfrom typing import Literal, cast\n\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport scipy.stats as sst\nfrom rliable import library as rly\nfrom rliable import plot_utils\nfrom sensai.util import logging\n\nfrom tianshou.utils import TensorboardLogger\nfrom tianshou.utils.logger.logger_base import DataScope\n\nlog = logging.getLogger(__name__)\n\n\n@dataclass\nclass EvaluationSequenceEntry:\n \"\"\"A single entry in an evaluation sequence, representing data collected at a fixed environment\n step.\n \"\"\"\n\n # the structure expected in benchmark.js\n env_step: int\n \"\"\"The number of environment steps at which the evaluation was performed.\"\"\"\n rew: float\n \"\"\"The mean episode return at the given env_step. Called `rew` (confusingly) to be consistent with Tianshou's\n internal naming conventions.\"\"\"\n rew_std: float\n \"\"\"The standard deviation of the episode returns at the given env_step, computed from multiple runs.\"\"\"\n iqm: float\n \"\"\"The interquartile mean (IQM) of the episode returns at the given env_step, computed from multiple runs.\"\"\"\n iqm_confidence_interval: tuple[float, float]\n \"\"\"The 95% confidence interval of the IQM of the episode returns at the given env_step.\"\"\"\n\n\n@dataclass\nclass LoggedSummaryData:\n mean: np.ndarray\n std: np.ndarray\n max: np.ndarray\n min: np.ndarray\n\n\n@dataclass\nclass LoggedCollectStats:\n env_step: np.ndarray | None = None\n n_collected_episodes: np.ndarray | None = None\n n_collected_steps: np.ndarray | None = None\n collect_time: np.ndarray | None = None\n collect_speed: np.ndarray | None = None\n returns_stat: LoggedSummaryData | None = None\n lens_stat: LoggedSummaryData | None = None\n\n @classmethod\n def from_data_dict(cls, data: dict) -> \"LoggedCollectStats\":\n \"\"\"Create a LoggedCollectStats object from a dictionary.\n\n Converts SequenceSummaryStats from dict format to dataclass format and ignores fields that are not present.\n \"\"\"\n dataclass_data = {}\n field_names = [f.name for f in fields(cls)]\n for k, v in data.items():\n if k not in field_names:\n log.info(\n f\"Key {k} in data dict is not a valid field of LoggedCollectStats, ignoring it.\",\n )\n continue\n if isinstance(v, dict):\n v = LoggedSummaryData(**v)\n dataclass_data[k] = v\n return cls(**dataclass_data)\n\n\n@dataclass\nclass MultiRunExperimentResult:\n \"\"\"The result of multiple runs of an experiment (runs usually just differing by random seeds)\n that can be used with the rliable library.\n\n Glossary:\n - R: number of runs (typically, equal to the number of different seeds)\n - E: number of environment steps at which evaluation results were computed, i.e., the evaluation points\n n_1, n_2, ..., n_E\n \"\"\"\n\n exp_dir: str\n \"\"\"The base directory where each sub-directory contains the results of one experiment run.\"\"\"\n\n exp_name: str\n \"\"\"The name of the experiment, typically the name of the algorithm or the experiment directory basename.\"\"\"\n\n test_episode_returns_RE: np.ndarray\n \"\"\"The test episode returns for each run of the experiment, where each row corresponds to one run.\"\"\"\n\n training_episode_returns_RE: np.ndarray\n \"\"\"The training episode returns for each run of the experiment, where each row corresponds to one run.\"\"\"\n\n test_env_steps_E: np.ndarray\n \"\"\"The environment steps at which the test episodes were evaluated.\"\"\"\n\n training_env_steps_E: np.ndarray\n \"\"\"The environment steps at which the training episodes were evaluated.\"\"\"\n\n @classmethod\n def load_from_disk(\n cls,\n exp_dir: str,\n exp_name: str | None = None,\n max_env_step: int | None = None,\n ) -> \"MultiRunExperimentResult\":\n \"\"\"Load the experiment result from disk.\n\n :param exp_dir: The directory from where the experiment results are restored.\n :param exp_name: The name of the experiment. If not passed, will be inferred from the experiment directory name.\n :param max_env_step: The maximum number of environment steps to consider. If None, all data is considered.\n Note: if the experiments have different numbers of steps, the minimum number is used.\n \"\"\"\n test_episode_returns_RE = []\n training_episode_returns_RE = []\n test_env_steps_E = None\n \"\"\"The number of steps of the test run,\n will try extracting it either from the loaded stats or from loaded arrays.\"\"\"\n training_env_steps_E = None\n \"\"\"The number of steps of the training run,\n will try extracting it from the loaded stats or from loaded arrays.\"\"\"\n\n if exp_name is None:\n exp_name = os.path.basename(os.path.normpath(exp_dir))\n\n from tianshou.highlevel.experiment import Experiment\n\n # TODO: test_env_steps_E should not be defined in a loop and overwritten at each iteration\n # just for retrieving them. We might need a cleaner directory structure.\n for entry in os.scandir(exp_dir):\n if entry.name.startswith(\".\") or not entry.is_dir():\n continue\n\n try:\n logger_factory = Experiment.from_directory(entry.path).logger_factory\n # only retrieve logger class to prevent creating another tfevent file\n logger_cls = logger_factory.get_logger_class()\n # Usually this means from low-level API\n except FileNotFoundError:\n log.info(\n f\"Could not find persisted experiment in {entry.path}, using default logger.\",\n )\n logger_cls = TensorboardLogger\n\n data = logger_cls.restore_logged_data(entry.path)\n if not data:\n raise ValueError(f\"Could not restore data from {entry.path}.\")\n\n if DataScope.TEST not in data or not data[DataScope.TEST]:\n continue\n restored_test_data = data[DataScope.TEST]\n restored_train_data = data[DataScope.TRAINING]\n\n assert isinstance(restored_test_data, dict)\n assert isinstance(restored_train_data, dict)\n\n for restored_data, scope in zip(\n [restored_test_data, restored_train_data],\n [DataScope.TEST, DataScope.TRAINING],\n strict=True,\n ):\n if not isinstance(restored_data, dict):\n raise RuntimeError(\n f\"Expected entry with key {scope} data to be a dictionary, \"\n f\"but got {restored_data=}.\",\n )\n test_data = LoggedCollectStats.from_data_dict(restored_test_data)\n training_data = LoggedCollectStats.from_data_dict(restored_train_data)\n\n if test_data.returns_stat is not None:\n test_episode_returns_RE.append(test_data.returns_stat.mean)\n test_env_steps_E = test_data.env_step\n\n if training_data.returns_stat is not None:\n training_episode_returns_RE.append(training_data.returns_stat.mean)\n training_env_steps_E = training_data.env_step\n\n test_data_found = True\n training_data_found = True\n if not test_episode_returns_RE or test_env_steps_E is None:\n log.warning(f\"No test experiment data found in {exp_dir}.\")\n test_data_found = False\n if not training_episode_returns_RE or training_env_steps_E is None:\n log.warning(f\"No train experiment data found in {exp_dir}.\")\n training_data_found = False\n\n if not test_data_found and not training_data_found:\n raise RuntimeError(f\"No test or train data found in {exp_dir}.\")\n\n min_training_data_len = min([len(arr) for arr in training_episode_returns_RE])\n if max_env_step is not None:\n min_training_data_len = min(min_training_data_len, max_env_step)\n min_test_data_len = min([len(arr) for arr in test_episode_returns_RE])\n if max_env_step is not None:\n min_test_data_len = min(min_test_data_len, max_env_step)\n\n assert test_env_steps_E is not None\n assert training_env_steps_E is not None\n\n test_env_steps_E = test_env_steps_E[:min_test_data_len]\n training_env_steps_E = training_env_steps_E[:min_training_data_len]\n if max_env_step:\n # find the index at which the maximum env step is reached with searchsorted\n min_test_data_len = int(np.searchsorted(test_env_steps_E, max_env_step))\n min_training_data_len = int(np.searchsorted(training_env_steps_E, max_env_step))\n test_env_steps_E = test_env_steps_E[:min_test_data_len]\n training_env_steps_E = training_env_steps_E[:min_training_data_len]\n\n test_episode_returns_RE = np.array(\n [arr[:min_test_data_len] for arr in test_episode_returns_RE]\n )\n training_episode_returns_RE = np.array(\n [arr[:min_training_data_len] for arr in training_episode_returns_RE]\n )\n\n return cls(\n test_episode_returns_RE=test_episode_returns_RE,\n test_env_steps_E=test_env_steps_E,\n exp_dir=exp_dir,\n exp_name=exp_name,\n training_episode_returns_RE=training_episode_returns_RE,\n training_env_steps_E=training_env_steps_E,\n )\n\n def _get_env_steps_and_returns(\n self,\n scope: DataScope = DataScope.TEST,\n ) -> tuple[np.ndarray, np.ndarray]:\n if scope == DataScope.TEST:\n return self.test_env_steps_E, self.test_episode_returns_RE\n elif scope == DataScope.TRAINING:\n return self.training_env_steps_E, self.training_episode_returns_RE\n else:\n raise ValueError(f\"Invalid scope {scope}, should be either 'TEST' or 'TRAINING'.\")\n\n def _get_data_in_rliable_format(\n self,\n algo_name: str | None = None,\n score_thresholds: np.ndarray | None = None,\n scope: DataScope = DataScope.TEST,\n ) -> tuple[dict, np.ndarray, np.ndarray]:\n \"\"\"Return the data in the format expected by the rliable library.\n\n :param algo_name: The name of the algorithm to be shown in the figure legend. If None, the name of the algorithm\n is set to the experiment dir.\n :param score_thresholds: The score thresholds for the performance profile. If None, the thresholds are inferred\n from the minimum and maximum test episode returns.\n\n :return: A tuple score_dict (algo_name->returns), env_steps, and score_thresholds.\n \"\"\"\n env_steps, returns = self._get_env_steps_and_returns(scope=scope)\n if score_thresholds is None:\n score_thresholds = np.linspace(\n np.min(returns),\n np.max(returns),\n 101,\n )\n\n if algo_name is None:\n algo_name = os.path.basename(self.exp_dir)\n\n score_dict = {algo_name: returns}\n\n return score_dict, env_steps, score_thresholds\n\n def _compute_iqm_scores(\n self,\n scope: DataScope = DataScope.TEST,\n ) -> tuple[dict, dict]:\n \"\"\"Compute the IQM scores and confidence intervals for the experiment in a format\n expected by the rliable library.\n\n :param scope: The scope of the evaluation, either 'TEST' or 'TRAIN'.\n\n :return: A tuple of dicts, each with a single entry,\n (self.exp_name->iqm_scores, self.exp_name->iqm_confidence_intervals), where\n confidence intervals is an array of shape (2 x E)\n where the first row contains the lower bounds while the second row contains\n the upper bound of 95% CIs.\n \"\"\"\n score_dict, _, _ = self._get_data_in_rliable_format(\n algo_name=self.exp_name,\n score_thresholds=None,\n scope=scope,\n )\n\n compute_iqm = lambda scores: sst.trim_mean(scores, proportiontocut=0.25, axis=0)\n return rly.get_interval_estimates(score_dict, compute_iqm)\n\n def eval_results(\n self,\n algo_name: str | None = None,\n score_thresholds: np.ndarray | None = None,\n save_as_json: bool = True,\n save_plots: bool = True,\n show_plots: bool = True,\n scope: DataScope = DataScope.TEST,\n ax_iqm_sample_efficiency: plt.Axes | None = None,\n ax_performance_profile: plt.Axes | None = None,\n algo2color: dict[str, str] | None = None,\n ) -> tuple[plt.Figure, plt.Axes, plt.Figure, plt.Axes]:\n \"\"\"Evaluate the results of an experiment and create a sample efficiency curve and a performance profile.\n\n :param algo_name: The name of the algorithm to be shown in the figure legend. If None, the name of the algorithm\n is set to the experiment dir.\n :param score_thresholds: The score thresholds for the performance profile. If None, they will be inferred\n from the minimum and maximum test episode returns.\n :param save_as_json: whether to save the evaluation results as a JSON file (in a format compatible by the Tianshou\n benchmarking visualization) in the experiment directory.\n :param save_plots: whether to save the plots to the experiment directory.\n :param show_plots: whether to display the plots.\n :param scope: The scope of the evaluation, either 'TEST' or 'TRAIN'.\n :param ax_iqm_sample_efficiency: The axis to plot the IQM sample efficiency curve on. If None, a new figure is created.\n :param ax_performance_profile: The axis to plot the performance profile on. If None, a new figure is created.\n :param algo2color: A dictionary mapping algorithm names to colors. Useful for plotting\n the evaluations of multiple algorithms in the same figure, e.g., by first creating an ax_iqm and ax_profile\n with one evaluation and then passing them into the other evaluation. Same as the `colors`\n kwarg in the rliable plotting utils.\n\n :return: The created figures and axes in the order: fig_iqm, ax_iqm, fig_profile, ax_profile.\n \"\"\"\n iqm_scores, iqm_confidence_intervals = self._compute_iqm_scores(scope=scope)\n # Plot IQM sample efficiency curve\n if ax_iqm_sample_efficiency is None:\n fig_iqm_sample_efficiency, ax_iqm_sample_efficiency = plt.subplots(\n ncols=1, figsize=(7, 5), constrained_layout=True\n )\n else:\n fig_iqm_sample_efficiency = ax_iqm_sample_efficiency.get_figure() # type: ignore\n score_dict, env_steps, score_thresholds = self._get_data_in_rliable_format(\n algo_name=algo_name,\n score_thresholds=score_thresholds,\n scope=scope,\n )\n plot_utils.plot_sample_efficiency_curve(\n env_steps,\n iqm_scores,\n iqm_confidence_intervals,\n algorithms=None,\n xlabel=\"env step\",\n ylabel=\"IQM episode return\",\n ax=ax_iqm_sample_efficiency,\n colors=algo2color,\n )\n if show_plots:\n plt.show(block=False)\n\n if save_plots:\n iqm_sample_efficiency_curve_path = os.path.abspath(\n os.path.join(\n self.exp_dir,\n f\"iqm_sample_efficiency_curve_{scope.value}.png\",\n ),\n )\n log.info(f\"Saving iqm sample efficiency curve to {iqm_sample_efficiency_curve_path}.\")\n fig_iqm_sample_efficiency.savefig(iqm_sample_efficiency_curve_path)\n\n final_score_dict = {algo: returns[:, [-1]] for algo, returns in score_dict.items()}\n score_distributions, score_distributions_cis = rly.create_performance_profile(\n final_score_dict,\n score_thresholds,\n )\n\n # Plot score distributions\n if ax_performance_profile is None:\n fig_performance_profile, ax_performance_profile = plt.subplots(\n ncols=1, figsize=(7, 5), constrained_layout=True\n )\n else:\n fig_performance_profile = ax_performance_profile.get_figure() # type: ignore\n plot_utils.plot_performance_profiles(\n score_distributions,\n score_thresholds,\n performance_profile_cis=score_distributions_cis,\n xlabel=r\"Episode return $(\\tau)$\",\n ax=ax_performance_profile,\n )\n\n if save_plots:\n profile_curve_path = os.path.abspath(\n os.path.join(self.exp_dir, f\"performance_profile_{scope.value}.png\"),\n )\n log.info(f\"Saving performance profile curve to {profile_curve_path}.\")\n fig_performance_profile.savefig(profile_curve_path)\n if show_plots:\n plt.show(block=False)\n\n if save_as_json:\n json_path = os.path.abspath(\n os.path.join(self.exp_dir, f\"rliable_evaluation_{scope.value.lower()}.json\"),\n )\n log.info(f\"Saving rliable evaluation results to {json_path}.\")\n eval_results_dict_sequence = [\n asdict(eval_entry) for eval_entry in self.to_evaluation_sequence(scope=scope)\n ]\n with open(json_path, \"w\", encoding=\"utf-8\") as f:\n json.dump(eval_results_dict_sequence, f, indent=4)\n\n return (\n fig_iqm_sample_efficiency,\n ax_iqm_sample_efficiency,\n fig_performance_profile,\n ax_performance_profile,\n )\n\n def to_evaluation_sequence(\n self, scope: DataScope = DataScope.TEST\n ) -> Iterator[EvaluationSequenceEntry]:\n \"\"\"Convert the experiment result to EvaluationSequence.\n\n :param scope: The scope of the evaluation, either 'TEST' or 'TRAIN'.\n\n :return: The rliable EvaluationSequence.\n \"\"\"\n env_steps_E, returns_RE = self._get_env_steps_and_returns(scope=scope)\n iqm_scores_dict, iqm_confidence_intervals_dict = self._compute_iqm_scores(scope=scope)\n iqm_scores = iqm_scores_dict[self.exp_name]\n iqm_confidence_intervals = iqm_confidence_intervals_dict[self.exp_name]\n for i, env_step in enumerate(env_steps_E):\n returns_R = returns_RE[:, i]\n returns_mean, returns_std = np.mean(returns_R), np.std(returns_R)\n yield EvaluationSequenceEntry(\n env_step=env_step,\n rew_std=returns_std,\n rew=returns_mean,\n iqm=iqm_scores[i],\n iqm_confidence_interval=tuple(iqm_confidence_intervals[:, i]),\n )\n\n\ndef load_and_eval_experiment(\n log_dir: str,\n show_plots: bool = True,\n save_plots: bool = True,\n save_as_json: bool = True,\n scope: DataScope | Literal[\"both\"] = DataScope.TEST,\n max_env_step: int | None = None,\n) -> MultiRunExperimentResult:\n \"\"\"Evaluate the experiments in the given log directory using the rliable API and return the loaded results object.\n By default, will persist the evaluation results as plots and JSON files in the experiment directory.\n\n :param log_dir: The directory containing the experiment results.\n :param show_plots: whether to display plots.\n :param save_plots: whether to save plots to the `log_dir`.\n :param save_as_json: whether to save the evaluation results as a JSON file (in a format compatible by the Tianshou\n benchmarking visualization) in the experiment directory.\n :param scope: The scope of the evaluation (training or test) or 'both'.\n :param max_env_step: The maximum number of environment steps to consider. If None, all data is considered.\n Note: if the experiments have different numbers of steps, the minimum number is used.\n \"\"\"\n rliable_result = MultiRunExperimentResult.load_from_disk(log_dir, max_env_step=max_env_step)\n scopes = [scope]\n if scope == \"both\":\n scopes = [DataScope.TEST, DataScope.TRAINING]\n for scope in scopes:\n scope = cast(DataScope, scope)\n rliable_result.eval_results(\n show_plots=show_plots,\n save_plots=save_plots,\n save_as_json=save_as_json,\n scope=scope,\n )\n return rliable_result\n" }, { "path": "tianshou/exploration/__init__.py", "content": "from tianshou.exploration.random import BaseNoise, GaussianNoise, OUNoise\n\n__all__ = [\n \"BaseNoise\",\n \"GaussianNoise\",\n \"OUNoise\",\n]\n" }, { "path": "tianshou/exploration/random.py", "content": "from abc import ABC, abstractmethod\nfrom collections.abc import Sequence\n\nimport numpy as np\n\n\nclass BaseNoise(ABC):\n \"\"\"The action noise base class.\"\"\"\n\n @abstractmethod\n def reset(self) -> None:\n \"\"\"Reset to the initial state.\"\"\"\n\n @abstractmethod\n def __call__(self, size: Sequence[int]) -> np.ndarray:\n \"\"\"Generate new noise.\"\"\"\n raise NotImplementedError\n\n\nclass GaussianNoise(BaseNoise):\n \"\"\"The vanilla Gaussian process, for exploration in DDPG by default.\"\"\"\n\n def __init__(self, mu: float = 0.0, sigma: float = 1.0) -> None:\n self._mu = mu\n assert sigma >= 0, \"Noise std should not be negative.\"\n self._sigma = sigma\n\n def __call__(self, size: Sequence[int]) -> np.ndarray:\n return np.random.normal(self._mu, self._sigma, size)\n\n def reset(self) -> None:\n pass\n\n\nclass OUNoise(BaseNoise):\n \"\"\"Class for Ornstein-Uhlenbeck process, as used for exploration in DDPG.\n\n Usage:\n ::\n\n # init\n self.noise = OUNoise()\n # generate noise\n noise = self.noise(logits.shape, eps)\n\n For required parameters, you can refer to the stackoverflow page. However,\n our experiment result shows that (similar to OpenAI SpinningUp) using\n vanilla Gaussian process has little difference from using the\n Ornstein-Uhlenbeck process.\n \"\"\"\n\n def __init__(\n self,\n mu: float = 0.0,\n sigma: float = 0.3,\n theta: float = 0.15,\n dt: float = 1e-2,\n x0: float | np.ndarray | None = None,\n ) -> None:\n super().__init__()\n self._mu = mu\n self._alpha = theta * dt\n self._beta = sigma * np.sqrt(dt)\n self._x0 = x0\n self.reset()\n\n def reset(self) -> None:\n \"\"\"Reset to the initial state.\"\"\"\n self._x = self._x0\n\n def __call__(self, size: Sequence[int], mu: float | None = None) -> np.ndarray:\n \"\"\"Generate new noise.\n\n Return an numpy array which size is equal to ``size``.\n \"\"\"\n if self._x is None or (isinstance(self._x, np.ndarray) and self._x.shape != size):\n self._x = 0.0\n if mu is None:\n mu = self._mu\n r = self._beta * np.random.normal(size=size)\n self._x = self._x + self._alpha * (mu - self._x) + r\n return self._x # type: ignore\n" }, { "path": "tianshou/highlevel/__init__.py", "content": "\n" }, { "path": "tianshou/highlevel/algorithm.py", "content": "import logging\nimport typing\nfrom abc import ABC, abstractmethod\nfrom typing import Any, Generic, TypeVar, cast\n\nimport gymnasium\nimport torch\nfrom sensai.util.string import ToStringMixin\n\nfrom tianshou.algorithm import (\n A2C,\n DDPG,\n DQN,\n IQN,\n NPG,\n PPO,\n REDQ,\n SAC,\n TD3,\n TRPO,\n Algorithm,\n DiscreteSAC,\n Reinforce,\n)\nfrom tianshou.algorithm.algorithm_base import (\n OffPolicyAlgorithm,\n OnPolicyAlgorithm,\n Policy,\n)\nfrom tianshou.algorithm.modelfree.ddpg import ContinuousDeterministicPolicy\nfrom tianshou.algorithm.modelfree.discrete_sac import DiscreteSACPolicy\nfrom tianshou.algorithm.modelfree.dqn import DiscreteQLearningPolicy\nfrom tianshou.algorithm.modelfree.iqn import IQNPolicy\nfrom tianshou.algorithm.modelfree.redq import REDQPolicy\nfrom tianshou.algorithm.modelfree.reinforce import ProbabilisticActorPolicy\nfrom tianshou.algorithm.modelfree.sac import SACPolicy\nfrom tianshou.data import ReplayBuffer, VectorReplayBuffer\nfrom tianshou.data.collector import BaseCollector\nfrom tianshou.highlevel.config import (\n OffPolicyTrainingConfig,\n OnPolicyTrainingConfig,\n TrainingConfig,\n)\nfrom tianshou.highlevel.env import Environments\nfrom tianshou.highlevel.module.actor import (\n ActorFactory,\n)\nfrom tianshou.highlevel.module.core import (\n ModuleFactory,\n TDevice,\n)\nfrom tianshou.highlevel.module.critic import CriticEnsembleFactory, CriticFactory\nfrom tianshou.highlevel.params.algorithm_params import (\n A2CParams,\n DDPGParams,\n DiscreteSACParams,\n DQNParams,\n IQNParams,\n NPGParams,\n Params,\n ParamsMixinActorAndDualCritics,\n ParamsMixinSingleModel,\n ParamTransformerData,\n PPOParams,\n REDQParams,\n ReinforceParams,\n SACParams,\n TD3Params,\n TRPOParams,\n)\nfrom tianshou.highlevel.params.algorithm_wrapper import AlgorithmWrapperFactory\nfrom tianshou.highlevel.params.collector import (\n CollectorFactory,\n CollectorFactoryDefault,\n)\nfrom tianshou.highlevel.params.optim import OptimizerFactoryFactory\nfrom tianshou.highlevel.persistence import PolicyPersistence\nfrom tianshou.highlevel.trainer import TrainerCallbacks, TrainingContext\nfrom tianshou.highlevel.world import World\nfrom tianshou.trainer import (\n OffPolicyTrainer,\n OffPolicyTrainerParams,\n OnPolicyTrainer,\n OnPolicyTrainerParams,\n Trainer,\n)\nfrom tianshou.utils.net.discrete import DiscreteActor\n\nCHECKPOINT_DICT_KEY_MODEL = \"model\"\nCHECKPOINT_DICT_KEY_OBS_RMS = \"obs_rms\"\nTParams = TypeVar(\"TParams\", bound=Params)\nTActorCriticParams = TypeVar(\n \"TActorCriticParams\",\n bound=Params | ParamsMixinSingleModel,\n)\nTActorDualCriticsParams = TypeVar(\n \"TActorDualCriticsParams\",\n bound=Params | ParamsMixinActorAndDualCritics,\n)\nTDiscreteCriticOnlyParams = TypeVar(\n \"TDiscreteCriticOnlyParams\",\n bound=Params | ParamsMixinSingleModel,\n)\nTAlgorithm = TypeVar(\"TAlgorithm\", bound=Algorithm)\nTPolicy = TypeVar(\"TPolicy\", bound=Policy)\nTTrainingConfig = TypeVar(\"TTrainingConfig\", bound=TrainingConfig)\nlog = logging.getLogger(__name__)\n\n\nclass AlgorithmFactory(ABC, ToStringMixin, Generic[TTrainingConfig]):\n \"\"\"Factory for the creation of an :class:`Algorithm` instance, its policy, trainer as well as collectors.\"\"\"\n\n def __init__(self, training_config: TTrainingConfig, optim_factory: OptimizerFactoryFactory):\n self.training_config = training_config\n self.optim_factory = optim_factory\n self.algorithm_wrapper_factory: AlgorithmWrapperFactory | None = None\n self.trainer_callbacks: TrainerCallbacks = TrainerCallbacks()\n self.collector_factory: CollectorFactory = CollectorFactoryDefault()\n\n def set_collector_factory(self, collector_factory: CollectorFactory) -> None:\n self.collector_factory = collector_factory\n\n def create_train_test_collectors(\n self,\n algorithm: Algorithm,\n envs: Environments,\n reset_collectors: bool = True,\n ) -> tuple[BaseCollector, BaseCollector]:\n \"\"\"\n Creates the collectors for training and test environments.\n\n :param algorithm: the algorithm\n :param envs: the environments wrapper\n :param reset_collectors: Whether to reset the collectors before returning them.\n Setting to True means that the envs will be reset as well.\n :return: a tuple of (training_collector, test_collector)\n \"\"\"\n buffer_size = self.training_config.buffer_size\n training_envs = envs.training_envs\n buffer: ReplayBuffer\n if len(training_envs) > 1:\n buffer = VectorReplayBuffer(\n buffer_size,\n len(training_envs),\n stack_num=self.training_config.replay_buffer_stack_num,\n save_only_last_obs=self.training_config.replay_buffer_save_only_last_obs,\n ignore_obs_next=self.training_config.replay_buffer_ignore_obs_next,\n )\n else:\n buffer = ReplayBuffer(\n buffer_size,\n stack_num=self.training_config.replay_buffer_stack_num,\n save_only_last_obs=self.training_config.replay_buffer_save_only_last_obs,\n ignore_obs_next=self.training_config.replay_buffer_ignore_obs_next,\n )\n training_collector = self.collector_factory.create_collector(\n algorithm,\n training_envs,\n buffer,\n exploration_noise=True,\n )\n test_collector = self.collector_factory.create_collector(algorithm, envs.test_envs)\n if reset_collectors:\n training_collector.reset()\n test_collector.reset()\n return training_collector, test_collector\n\n def set_policy_wrapper_factory(\n self,\n policy_wrapper_factory: AlgorithmWrapperFactory | None,\n ) -> None:\n self.algorithm_wrapper_factory = policy_wrapper_factory\n\n def set_trainer_callbacks(self, callbacks: TrainerCallbacks) -> None:\n self.trainer_callbacks = callbacks\n\n @staticmethod\n def _create_policy_from_args(\n constructor: type[TPolicy],\n params_dict: dict,\n policy_params: list[str],\n **kwargs: Any,\n ) -> TPolicy:\n params = {p: params_dict.pop(p) for p in policy_params}\n return constructor(**params, **kwargs)\n\n @abstractmethod\n def _create_algorithm(self, envs: Environments, device: TDevice) -> Algorithm:\n pass\n\n def create_algorithm(self, envs: Environments, device: TDevice) -> Algorithm:\n algorithm = self._create_algorithm(envs, device)\n if self.algorithm_wrapper_factory is not None:\n algorithm = self.algorithm_wrapper_factory.create_wrapped_algorithm(\n algorithm,\n envs,\n self.optim_factory,\n device,\n )\n return algorithm\n\n @abstractmethod\n def create_trainer(self, world: World, policy_persistence: PolicyPersistence) -> Trainer:\n pass\n\n\nclass OnPolicyAlgorithmFactory(AlgorithmFactory[OnPolicyTrainingConfig], ABC):\n def create_trainer(\n self,\n world: World,\n policy_persistence: PolicyPersistence,\n ) -> OnPolicyTrainer:\n training_config = self.training_config\n callbacks = self.trainer_callbacks\n context = TrainingContext(world.algorithm, world.envs, world.logger)\n train_fn = (\n callbacks.epoch_train_callback.get_trainer_fn(context)\n if callbacks.epoch_train_callback\n else None\n )\n test_fn = (\n callbacks.epoch_test_callback.get_trainer_fn(context)\n if callbacks.epoch_test_callback\n else None\n )\n stop_fn = (\n callbacks.epoch_stop_callback.get_trainer_fn(context)\n if callbacks.epoch_stop_callback\n else None\n )\n algorithm = cast(OnPolicyAlgorithm, world.algorithm)\n assert world.training_collector is not None\n return algorithm.create_trainer(\n OnPolicyTrainerParams(\n training_collector=world.training_collector,\n test_collector=world.test_collector,\n max_epochs=training_config.max_epochs,\n epoch_num_steps=training_config.epoch_num_steps,\n update_step_num_repetitions=training_config.update_step_num_repetitions,\n test_step_num_episodes=training_config.test_step_num_episodes,\n batch_size=training_config.batch_size,\n collection_step_num_env_steps=training_config.collection_step_num_env_steps,\n save_best_fn=policy_persistence.get_save_best_fn(world),\n save_checkpoint_fn=policy_persistence.get_save_checkpoint_fn(world),\n logger=world.logger,\n test_in_training=training_config.test_in_training,\n training_fn=train_fn,\n test_fn=test_fn,\n stop_fn=stop_fn,\n verbose=False,\n )\n )\n\n\nclass OffPolicyAlgorithmFactory(AlgorithmFactory[OffPolicyTrainingConfig], ABC):\n def create_trainer(\n self,\n world: World,\n policy_persistence: PolicyPersistence,\n ) -> OffPolicyTrainer:\n training_config = self.training_config\n callbacks = self.trainer_callbacks\n context = TrainingContext(world.algorithm, world.envs, world.logger)\n train_fn = (\n callbacks.epoch_train_callback.get_trainer_fn(context)\n if callbacks.epoch_train_callback\n else None\n )\n test_fn = (\n callbacks.epoch_test_callback.get_trainer_fn(context)\n if callbacks.epoch_test_callback\n else None\n )\n stop_fn = (\n callbacks.epoch_stop_callback.get_trainer_fn(context)\n if callbacks.epoch_stop_callback\n else None\n )\n algorithm = cast(OffPolicyAlgorithm, world.algorithm)\n assert world.training_collector is not None\n return algorithm.create_trainer(\n OffPolicyTrainerParams(\n training_collector=world.training_collector,\n test_collector=world.test_collector,\n max_epochs=training_config.max_epochs,\n epoch_num_steps=training_config.epoch_num_steps,\n collection_step_num_env_steps=training_config.collection_step_num_env_steps,\n test_step_num_episodes=training_config.test_step_num_episodes,\n batch_size=training_config.batch_size,\n save_best_fn=policy_persistence.get_save_best_fn(world),\n logger=world.logger,\n update_step_num_gradient_steps_per_sample=training_config.update_step_num_gradient_steps_per_sample,\n test_in_training=training_config.test_in_training,\n training_fn=train_fn,\n test_fn=test_fn,\n stop_fn=stop_fn,\n verbose=False,\n )\n )\n\n\nclass ReinforceAlgorithmFactory(OnPolicyAlgorithmFactory):\n def __init__(\n self,\n params: ReinforceParams,\n training_config: OnPolicyTrainingConfig,\n actor_factory: ActorFactory,\n optim_factory: OptimizerFactoryFactory,\n ):\n super().__init__(training_config, optim_factory)\n self.params = params\n self.actor_factory = actor_factory\n self.optim_factory = optim_factory\n\n def _create_algorithm(self, envs: Environments, device: TDevice) -> Reinforce:\n actor = self.actor_factory.create_module(envs, device)\n kwargs = self.params.create_kwargs(\n ParamTransformerData(\n envs=envs,\n device=device,\n optim_factory_default=self.optim_factory,\n ),\n )\n dist_fn = self.actor_factory.create_dist_fn(envs)\n assert dist_fn is not None\n policy = self._create_policy_from_args(\n ProbabilisticActorPolicy,\n kwargs,\n [\"action_scaling\", \"action_bound_method\", \"deterministic_eval\"],\n actor=actor,\n dist_fn=dist_fn,\n action_space=envs.get_action_space(),\n observation_space=envs.get_observation_space(),\n )\n return Reinforce(\n policy=policy,\n **kwargs,\n )\n\n\nclass ActorCriticOnPolicyAlgorithmFactory(\n OnPolicyAlgorithmFactory,\n Generic[TActorCriticParams, TAlgorithm],\n):\n def __init__(\n self,\n params: TActorCriticParams,\n training_config: OnPolicyTrainingConfig,\n actor_factory: ActorFactory,\n critic_factory: CriticFactory,\n optimizer_factory: OptimizerFactoryFactory,\n ):\n super().__init__(training_config, optim_factory=optimizer_factory)\n self.params = params\n self.actor_factory = actor_factory\n self.critic_factory = critic_factory\n self.optim_factory = optimizer_factory\n self.critic_use_action = False\n\n @abstractmethod\n def _get_algorithm_class(self) -> type[TAlgorithm]:\n pass\n\n @typing.no_type_check\n def _create_kwargs(self, envs: Environments, device: TDevice) -> dict[str, Any]:\n actor = self.actor_factory.create_module(envs, device)\n critic = self.critic_factory.create_module(envs, device, use_action=self.critic_use_action)\n kwargs = self.params.create_kwargs(\n ParamTransformerData(\n envs=envs,\n device=device,\n optim_factory_default=self.optim_factory,\n ),\n )\n kwargs[\"actor\"] = actor\n kwargs[\"critic\"] = critic\n kwargs[\"action_space\"] = envs.get_action_space()\n kwargs[\"observation_space\"] = envs.get_observation_space()\n kwargs[\"dist_fn\"] = self.actor_factory.create_dist_fn(envs)\n return kwargs\n\n def _create_algorithm(self, envs: Environments, device: TDevice) -> TAlgorithm:\n params = self._create_kwargs(envs, device)\n policy = self._create_policy_from_args(\n ProbabilisticActorPolicy,\n params,\n [\n \"actor\",\n \"dist_fn\",\n \"action_space\",\n \"deterministic_eval\",\n \"observation_space\",\n \"action_scaling\",\n \"action_bound_method\",\n ],\n )\n algorithm_class = self._get_algorithm_class()\n return algorithm_class(policy=policy, **params)\n\n\nclass A2CAlgorithmFactory(ActorCriticOnPolicyAlgorithmFactory[A2CParams, A2C]):\n def _get_algorithm_class(self) -> type[A2C]:\n return A2C\n\n\nclass PPOAlgorithmFactory(ActorCriticOnPolicyAlgorithmFactory[PPOParams, PPO]):\n def _get_algorithm_class(self) -> type[PPO]:\n return PPO\n\n\nclass NPGAlgorithmFactory(ActorCriticOnPolicyAlgorithmFactory[NPGParams, NPG]):\n def _get_algorithm_class(self) -> type[NPG]:\n return NPG\n\n\nclass TRPOAlgorithmFactory(ActorCriticOnPolicyAlgorithmFactory[TRPOParams, TRPO]):\n def _get_algorithm_class(self) -> type[TRPO]:\n return TRPO\n\n\nclass DiscreteCriticOnlyOffPolicyAlgorithmFactory(\n OffPolicyAlgorithmFactory,\n Generic[TDiscreteCriticOnlyParams, TAlgorithm],\n):\n def __init__(\n self,\n params: TDiscreteCriticOnlyParams,\n training_config: OffPolicyTrainingConfig,\n model_factory: ModuleFactory,\n optim_factory: OptimizerFactoryFactory,\n ):\n super().__init__(training_config, optim_factory)\n self.params = params\n self.model_factory = model_factory\n self.optim_factory = optim_factory\n\n @abstractmethod\n def _get_algorithm_class(self) -> type[TAlgorithm]:\n pass\n\n @abstractmethod\n def _create_policy(\n self,\n model: torch.nn.Module,\n params: dict,\n action_space: gymnasium.spaces.Discrete,\n observation_space: gymnasium.spaces.Space,\n ) -> Policy:\n pass\n\n @typing.no_type_check\n def _create_algorithm(self, envs: Environments, device: TDevice) -> TAlgorithm:\n model = self.model_factory.create_module(envs, device)\n params_dict = self.params.create_kwargs(\n ParamTransformerData(\n envs=envs,\n device=device,\n optim_factory_default=self.optim_factory,\n ),\n )\n envs.get_type().assert_discrete(self)\n action_space = cast(gymnasium.spaces.Discrete, envs.get_action_space())\n policy = self._create_policy(model, params_dict, action_space, envs.get_observation_space())\n algorithm_class = self._get_algorithm_class()\n return algorithm_class(\n policy=policy,\n **params_dict,\n )\n\n\nclass DQNAlgorithmFactory(DiscreteCriticOnlyOffPolicyAlgorithmFactory[DQNParams, DQN]):\n def _create_policy(\n self,\n model: torch.nn.Module,\n params: dict,\n action_space: gymnasium.spaces.Discrete,\n observation_space: gymnasium.spaces.Space,\n ) -> Policy:\n return self._create_policy_from_args(\n constructor=DiscreteQLearningPolicy,\n params_dict=params,\n policy_params=[\"eps_training\", \"eps_inference\"],\n model=model,\n action_space=action_space,\n observation_space=observation_space,\n )\n\n def _get_algorithm_class(self) -> type[DQN]:\n return DQN\n\n\nclass IQNAlgorithmFactory(DiscreteCriticOnlyOffPolicyAlgorithmFactory[IQNParams, IQN]):\n def _create_policy(\n self,\n model: torch.nn.Module,\n params: dict,\n action_space: gymnasium.spaces.Discrete,\n observation_space: gymnasium.spaces.Space,\n ) -> Policy:\n return self._create_policy_from_args(\n IQNPolicy,\n params,\n [\n \"sample_size\",\n \"online_sample_size\",\n \"target_sample_size\",\n \"eps_training\",\n \"eps_inference\",\n ],\n model=model,\n action_space=action_space,\n observation_space=observation_space,\n )\n\n def _get_algorithm_class(self) -> type[IQN]:\n return IQN\n\n\nclass DDPGAlgorithmFactory(OffPolicyAlgorithmFactory):\n def __init__(\n self,\n params: DDPGParams,\n training_config: OffPolicyTrainingConfig,\n actor_factory: ActorFactory,\n critic_factory: CriticFactory,\n optim_factory: OptimizerFactoryFactory,\n ):\n super().__init__(training_config, optim_factory)\n self.critic_factory = critic_factory\n self.actor_factory = actor_factory\n self.params = params\n self.optim_factory = optim_factory\n\n def _create_algorithm(self, envs: Environments, device: TDevice) -> Algorithm:\n actor = self.actor_factory.create_module(envs, device)\n critic = self.critic_factory.create_module(\n envs,\n device,\n True,\n )\n kwargs = self.params.create_kwargs(\n ParamTransformerData(\n envs=envs,\n device=device,\n optim_factory_default=self.optim_factory,\n ),\n )\n policy = self._create_policy_from_args(\n ContinuousDeterministicPolicy,\n kwargs,\n [\"exploration_noise\", \"action_scaling\", \"action_bound_method\"],\n actor=actor,\n action_space=envs.get_action_space(),\n observation_space=envs.get_observation_space(),\n )\n return DDPG(\n policy=policy,\n critic=critic,\n **kwargs,\n )\n\n\nclass REDQAlgorithmFactory(OffPolicyAlgorithmFactory):\n def __init__(\n self,\n params: REDQParams,\n training_config: OffPolicyTrainingConfig,\n actor_factory: ActorFactory,\n critic_ensemble_factory: CriticEnsembleFactory,\n optim_factory: OptimizerFactoryFactory,\n ):\n super().__init__(training_config, optim_factory)\n self.critic_ensemble_factory = critic_ensemble_factory\n self.actor_factory = actor_factory\n self.params = params\n self.optim_factory = optim_factory\n\n def _create_algorithm(self, envs: Environments, device: TDevice) -> Algorithm:\n envs.get_type().assert_continuous(self)\n actor = self.actor_factory.create_module(\n envs,\n device,\n )\n critic_ensemble = self.critic_ensemble_factory.create_module(\n envs,\n device,\n self.params.ensemble_size,\n True,\n )\n kwargs = self.params.create_kwargs(\n ParamTransformerData(\n envs=envs,\n device=device,\n optim_factory_default=self.optim_factory,\n ),\n )\n action_space = cast(gymnasium.spaces.Box, envs.get_action_space())\n policy = self._create_policy_from_args(\n REDQPolicy,\n kwargs,\n [\n \"exploration_noise\",\n \"deterministic_eval\",\n \"action_scaling\",\n \"action_bound_method\",\n ],\n actor=actor,\n action_space=action_space,\n observation_space=envs.get_observation_space(),\n )\n return REDQ(\n policy=policy,\n critic=critic_ensemble,\n **kwargs,\n )\n\n\nclass ActorDualCriticsOffPolicyAlgorithmFactory(\n OffPolicyAlgorithmFactory,\n Generic[TActorDualCriticsParams, TAlgorithm, TPolicy],\n):\n def __init__(\n self,\n params: TActorDualCriticsParams,\n training_config: OffPolicyTrainingConfig,\n actor_factory: ActorFactory,\n critic1_factory: CriticFactory,\n critic2_factory: CriticFactory,\n optim_factory: OptimizerFactoryFactory,\n ):\n super().__init__(training_config, optim_factory)\n self.params = params\n self.actor_factory = actor_factory\n self.critic1_factory = critic1_factory\n self.critic2_factory = critic2_factory\n self.optim_factory = optim_factory\n\n @abstractmethod\n def _get_algorithm_class(self) -> type[TAlgorithm]:\n pass\n\n def _get_discrete_last_size_use_action_shape(self) -> bool:\n return True\n\n @staticmethod\n def _get_critic_use_action(envs: Environments) -> bool:\n return envs.get_type().is_continuous()\n\n @abstractmethod\n def _create_policy(\n self, actor: torch.nn.Module | DiscreteActor, envs: Environments, params: dict\n ) -> TPolicy:\n pass\n\n @typing.no_type_check\n def _create_algorithm(self, envs: Environments, device: TDevice) -> TAlgorithm:\n actor = self.actor_factory.create_module(envs, device)\n use_action_shape = self._get_discrete_last_size_use_action_shape()\n critic_use_action = self._get_critic_use_action(envs)\n critic1 = self.critic1_factory.create_module(\n envs,\n device,\n use_action=critic_use_action,\n discrete_last_size_use_action_shape=use_action_shape,\n )\n critic2 = self.critic2_factory.create_module(\n envs,\n device,\n use_action=critic_use_action,\n discrete_last_size_use_action_shape=use_action_shape,\n )\n kwargs = self.params.create_kwargs(\n ParamTransformerData(\n envs=envs,\n device=device,\n optim_factory_default=self.optim_factory,\n ),\n )\n policy = self._create_policy(actor, envs, kwargs)\n algorithm_class = self._get_algorithm_class()\n return algorithm_class(\n policy=policy,\n critic=critic1,\n critic2=critic2,\n **kwargs,\n )\n\n\nclass SACAlgorithmFactory(ActorDualCriticsOffPolicyAlgorithmFactory[SACParams, SAC, SACPolicy]):\n def _create_policy(\n self, actor: torch.nn.Module | DiscreteActor, envs: Environments, params: dict\n ) -> SACPolicy:\n return self._create_policy_from_args(\n SACPolicy,\n params,\n [\"exploration_noise\", \"deterministic_eval\", \"action_scaling\"],\n actor=actor,\n action_space=envs.get_action_space(),\n observation_space=envs.get_observation_space(),\n )\n\n def _get_algorithm_class(self) -> type[SAC]:\n return SAC\n\n\nclass DiscreteSACAlgorithmFactory(\n ActorDualCriticsOffPolicyAlgorithmFactory[DiscreteSACParams, DiscreteSAC, DiscreteSACPolicy]\n):\n def _create_policy(\n self, actor: torch.nn.Module | DiscreteActor, envs: Environments, params: dict\n ) -> DiscreteSACPolicy:\n return self._create_policy_from_args(\n DiscreteSACPolicy,\n params,\n [\"deterministic_eval\"],\n actor=actor,\n action_space=envs.get_action_space(),\n observation_space=envs.get_observation_space(),\n )\n\n def _get_algorithm_class(self) -> type[DiscreteSAC]:\n return DiscreteSAC\n\n\nclass TD3AlgorithmFactory(\n ActorDualCriticsOffPolicyAlgorithmFactory[TD3Params, TD3, ContinuousDeterministicPolicy]\n):\n def _create_policy(\n self, actor: torch.nn.Module | DiscreteActor, envs: Environments, params: dict\n ) -> ContinuousDeterministicPolicy:\n return self._create_policy_from_args(\n ContinuousDeterministicPolicy,\n params,\n [\"exploration_noise\", \"action_scaling\", \"action_bound_method\"],\n actor=actor,\n action_space=envs.get_action_space(),\n observation_space=envs.get_observation_space(),\n )\n\n def _get_algorithm_class(self) -> type[TD3]:\n return TD3\n" }, { "path": "tianshou/highlevel/config.py", "content": "import logging\nimport multiprocessing\nfrom dataclasses import dataclass\n\nfrom sensai.util.pickle import setstate\nfrom sensai.util.string import ToStringMixin\n\nlog = logging.getLogger(__name__)\n\n\n@dataclass(kw_only=True)\nclass TrainingConfig(ToStringMixin):\n \"\"\"Training configuration.\"\"\"\n\n max_epochs: int = 100\n \"\"\"\n the (maximum) number of epochs to run training for. An **epoch** is the outermost iteration level and each\n epoch consists of a number of training steps and one test step, where each training step\n\n * [for the online case] collects environment steps/transitions (**collection step**),\n adding them to the (replay) buffer (see :attr:`collection_step_num_env_steps` and :attr:`collection_step_num_episodes`)\n * performs an **update step** via the RL algorithm being used, which can involve\n one or more actual gradient updates, depending on the algorithm\n\n and the test step collects :attr:`num_episodes_per_test` test episodes in order to evaluate\n agent performance.\n\n Training may be stopped early if the stop criterion is met (see :attr:`~tianshou.trainer.trainer.TrainerParams.stop_fn`).\n\n For online training, the number of training steps in each epoch is indirectly determined by\n :attr:`epoch_num_steps`: As many training steps will be performed as are required in\n order to reach :attr:`epoch_num_steps` total steps in the training environments.\n Specifically, if the number of transitions collected per step is `c` (see\n :attr:`collection_step_num_env_steps`) and :attr:`epoch_num_steps` is set to `s`, then the number\n of training steps per epoch is `ceil(s / c)`.\n Therefore, if `max_epochs = e`, the total number of environment steps taken during training\n can be computed as `e * ceil(s / c) * c`.\n\n For offline training, the number of training steps per epoch is equal to :attr:`epoch_num_steps`.\n \"\"\"\n\n epoch_num_steps: int = 30000\n \"\"\"\n For an online algorithm, this is the total number of environment steps to be collected per epoch, and,\n for an offline algorithm, it is the total number of training steps to take per epoch.\n See :attr:`max_epochs` for an explanation of epoch semantics.\n \"\"\"\n\n num_training_envs: int = -1\n \"\"\"the number of training environments to use. If set to -1, use number of CPUs/threads.\"\"\"\n\n num_test_envs: int = 1\n \"\"\"the number of test environments to use\"\"\"\n\n test_step_num_episodes: int = -1\n \"\"\"the total number of episodes to collect in each test step (across all test environments).\n\n -1 means this will be set to the number of test environments, i.e. each test environment\n will run exactly one episode per test step.\n \"\"\"\n\n buffer_size: int = 4096\n \"\"\"the total size of the sample/replay buffer, in which environment steps (transitions) are\n stored\"\"\"\n\n collection_step_num_env_steps: int | None = 2048\n \"\"\"\n the number of environment steps/transitions to collect in each collection step before the\n network update within each training step.\n\n This is mutually exclusive with :attr:`collection_step_num_episodes`, and one of the two must be set.\n\n Note that the exact number can be reached only if this is a multiple of the number of\n training environments being used, as each training environment will produce the same\n (non-zero) number of transitions.\n Specifically, if this is set to `n` and `m` training environments are used, then the total\n number of transitions collected per collection step is `ceil(n / m) * m =: c`.\n\n See :attr:`max_epochs` for information on the total number of environment steps being\n collected during training.\n \"\"\"\n\n collection_step_num_episodes: int | None = None\n \"\"\"\n the number of episodes to collect in each collection step before the network update within\n each training step. If this is set, the number of environment steps collected in each\n collection step is the sum of the lengths of the episodes collected.\n\n This is mutually exclusive with :attr:`collection_step_num_env_steps`, and one of the two must be set.\n \"\"\"\n\n start_timesteps: int = 0\n \"\"\"\n the number of environment steps to collect before the actual training loop begins\n \"\"\"\n\n start_timesteps_random: bool = False\n \"\"\"\n whether to use a random policy (instead of the initial or restored policy to be trained)\n when collecting the initial :attr:`start_timesteps` environment steps before training\n \"\"\"\n\n replay_buffer_ignore_obs_next: bool = False\n \"\"\"whether to ignore the `obs_next` field in the collected samples when storing them in the\n buffer and instead use the one-in-the-future of `obs` as the next observation.\n This can be useful for very large observations, like for Atari, in order to save RAM.\n\n However, setting this to True **may introduce an error** at the last steps of episodes! Should\n only be used in exceptional cases and only when you know what you are doing.\n Currently only used in Atari examples and may be removed in the future!\n \"\"\"\n\n replay_buffer_save_only_last_obs: bool = False\n \"\"\"if True, for the case where the environment outputs stacked frames (e.g. because it\n is using a `FrameStack` wrapper), save only the most recent frame so as not to duplicate\n observations in buffer memory. Specifically, if the environment outputs observations `obs` with\n shape (N, ...), only obs[-1] of shape (...) will be stored.\n Frame stacking with a fixed number of frames can then be recreated at the buffer level by setting\n :attr:`replay_buffer_stack_num`.\n\n Note: Currently only used in Atari examples and may be removed in the future!\n \"\"\"\n\n replay_buffer_stack_num: int = 1\n \"\"\"\n the number of consecutive environment observations to stack and use as the observation input\n to the agent for each time step. Setting this to a value greater than 1 can help agents learn\n temporal aspects (e.g. velocities of moving objects for which only positions are observed).\n\n Note: it is recommended to do this stacking on the environment level by using something like\n gymnasium's `FrameStack` instead. Setting this to larger than one in conjunction\n with :attr:`replay_buffer_save_only_last_obs` means that\n stacking will be recreated at the buffer level, which is more memory-efficient.\n\n Currently only used in Atari examples and may be removed in the future!\n \"\"\"\n\n def __setstate__(self, state: dict) -> None:\n setstate(\n TrainingConfig, self, state, renamed_properties={\"num_train_envs\": \"num_training_envs\"}\n )\n\n def __post_init__(self) -> None:\n if self.num_training_envs == -1:\n self.num_training_envs = multiprocessing.cpu_count()\n\n if self.test_step_num_episodes == 0 and self.num_test_envs != 0:\n log.warning(\n f\"Number of test episodes is set to 0, \"\n f\"but number of test environments is ({self.num_test_envs}). \"\n f\"This can cause unnecessary memory usage.\",\n )\n if self.test_step_num_episodes == -1:\n log.debug(\n f\"Setting test_step_num_episodes to num_test_envs ({self.num_test_envs}) since it was -1.\"\n )\n self.test_step_num_episodes = self.num_test_envs\n if (\n self.test_step_num_episodes != 0\n and self.test_step_num_episodes % self.num_test_envs != 0\n ):\n log.warning(\n f\"Number of test episodes ({self.test_step_num_episodes} \"\n f\"is not divisible by the number of test environments ({self.num_test_envs}). \"\n f\"This can cause unnecessary memory usage, it is recommended to adjust this.\",\n )\n\n assert (\n sum(\n [\n self.collection_step_num_env_steps is not None,\n self.collection_step_num_episodes is not None,\n ]\n )\n == 1\n ), (\n \"Only one of `collection_step_num_env_steps` and `collection_step_num_episodes` can be set.\",\n )\n\n\n@dataclass(kw_only=True)\nclass OnlineTrainingConfig(TrainingConfig):\n collection_step_num_env_steps: int | None = 2048\n \"\"\"\n the number of environment steps/transitions to collect in each collection step before the\n network update within each training step.\n\n This is mutually exclusive with :attr:`collection_step_num_episodes`, and one of the two must be set.\n\n Note that the exact number can be reached only if this is a multiple of the number of\n training environments being used, as each training environment will produce the same\n (non-zero) number of transitions.\n Specifically, if this is set to `n` and `m` training environments are used, then the total\n number of transitions collected per collection step is `ceil(n / m) * m =: c`.\n\n See :attr:`max_epochs` for information on the total number of environment steps being\n collected during training.\n \"\"\"\n\n collection_step_num_episodes: int | None = None\n \"\"\"\n the number of episodes to collect in each collection step before the network update within\n each training step. If this is set, the number of environment steps collected in each\n collection step is the sum of the lengths of the episodes collected.\n\n This is mutually exclusive with :attr:`collection_step_num_env_steps`, and one of the two must be set.\n \"\"\"\n\n test_in_training: bool = False\n \"\"\"\n Whether to apply a test step within a training step depending on the early stopping criterion\n (see :meth:`~tianshou.highlevel.Experiment.with_epoch_stop_callback`) being satisfied based\n on the data collected within the training step.\n Specifically, after each collect step, we check whether the early stopping criterion\n would be satisfied by data we collected (provided that at least one episode was indeed completed, such\n that we can evaluate returns, etc.). If the criterion is satisfied, we perform a full test step\n (collecting :attr:`test_step_num_episodes` episodes in order to evaluate performance), and if the early\n stopping criterion is also satisfied based on the test data, we stop training early.\n \"\"\"\n\n def __setstate__(self, state: dict) -> None:\n setstate(\n OnlineTrainingConfig,\n self,\n state,\n renamed_properties={\"test_in_train\": \"test_in_training\"},\n )\n\n\n@dataclass(kw_only=True)\nclass OnPolicyTrainingConfig(OnlineTrainingConfig):\n batch_size: int | None = 64\n \"\"\"\n Use mini-batches of this size for gradient updates (causing the gradient to be less accurate,\n a form of regularization).\n Set ``batch_size=None`` for the full buffer that was collected within the training step to be\n used for the gradient update (no mini-batching).\n \"\"\"\n\n update_step_num_repetitions: int = 1\n \"\"\"\n controls, within one update step of an on-policy algorithm, the number of times\n the full collected data is applied for gradient updates, i.e. if the parameter is\n 5, then the collected data shall be used five times to update the policy within the same\n update step.\n \"\"\"\n\n\n@dataclass(kw_only=True)\nclass OffPolicyTrainingConfig(OnlineTrainingConfig):\n batch_size: int = 64\n \"\"\"\n the the number of environment steps/transitions to sample from the buffer for a gradient update.\n \"\"\"\n\n update_step_num_gradient_steps_per_sample: float = 1.0\n \"\"\"\n the number of gradient steps to perform per sample collected (see :attr:`collection_step_num_env_steps`).\n Specifically, if this is set to `u` and the number of samples collected in the preceding\n collection step is `n`, then `round(u * n)` gradient steps will be performed.\n \"\"\"\n" }, { "path": "tianshou/highlevel/env.py", "content": "import logging\nimport platform\nfrom abc import ABC, abstractmethod\nfrom collections.abc import Callable, Sequence\nfrom enum import Enum\nfrom typing import Any, TypeAlias, cast\n\nimport gymnasium as gym\nimport gymnasium.spaces\nimport numpy as np\nfrom gymnasium import Env\nfrom sensai.util.pickle import setstate\nfrom sensai.util.string import ToStringMixin\n\nfrom tianshou.env import (\n BaseVectorEnv,\n DummyVectorEnv,\n RayVectorEnv,\n SubprocVectorEnv,\n)\nfrom tianshou.highlevel.persistence import Persistence\nfrom tianshou.utils.net.common import TActionShape\n\nTObservationShape: TypeAlias = int | Sequence[int]\n\nlog = logging.getLogger(__name__)\n\n\nclass EnvType(Enum):\n \"\"\"Enumeration of environment types.\"\"\"\n\n CONTINUOUS = \"continuous\"\n DISCRETE = \"discrete\"\n\n def is_discrete(self) -> bool:\n return self == EnvType.DISCRETE\n\n def is_continuous(self) -> bool:\n return self == EnvType.CONTINUOUS\n\n def assert_continuous(self, requiring_entity: Any) -> None:\n if not self.is_continuous():\n raise AssertionError(f\"{requiring_entity} requires continuous environments\")\n\n def assert_discrete(self, requiring_entity: Any) -> None:\n if not self.is_discrete():\n raise AssertionError(f\"{requiring_entity} requires discrete environments\")\n\n @staticmethod\n def from_env(env: Env) -> \"EnvType\":\n if isinstance(env.action_space, gymnasium.spaces.Discrete):\n return EnvType.DISCRETE\n elif isinstance(env.action_space, gymnasium.spaces.Box):\n return EnvType.CONTINUOUS\n else:\n raise Exception(f\"Unsupported environment type with action space {env.action_space}\")\n\n\nclass EnvMode(Enum):\n \"\"\"Indicates the purpose for which an environment is created.\"\"\"\n\n TRAINING = \"training\"\n TEST = \"test\"\n WATCH = \"watch\"\n\n\nclass VectorEnvType(Enum):\n DUMMY = \"dummy\"\n \"\"\"Vectorized environment without parallelization; environments are processed sequentially\"\"\"\n SUBPROC = \"subproc\"\n \"\"\"Parallelization based on `subprocess`\"\"\"\n SUBPROC_SHARED_MEM_DEFAULT_CONTEXT = \"shmem\"\n \"\"\"Parallelization based on `subprocess` with shared memory\"\"\"\n SUBPROC_SHARED_MEM_FORK_CONTEXT = \"shmem_fork\"\n \"\"\"Parallelization based on `subprocess` with shared memory and fork context (relevant for macOS, which uses `spawn`\n by default https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods)\"\"\"\n RAY = \"ray\"\n \"\"\"Parallelization based on the `ray` library\"\"\"\n SUBPROC_SHARED_MEM_AUTO = \"subproc_shared_mem_auto\"\n \"\"\"Parallelization based on `subprocess` with shared memory, using default context on windows and fork context otherwise\"\"\"\n\n def create_venv(\n self,\n factories: Sequence[Callable[[], gym.Env]],\n ) -> BaseVectorEnv:\n match self:\n case VectorEnvType.DUMMY:\n return DummyVectorEnv(factories)\n case VectorEnvType.SUBPROC:\n return SubprocVectorEnv(factories)\n case VectorEnvType.SUBPROC_SHARED_MEM_DEFAULT_CONTEXT:\n return SubprocVectorEnv(factories, share_memory=True)\n case VectorEnvType.SUBPROC_SHARED_MEM_FORK_CONTEXT:\n return SubprocVectorEnv(factories, share_memory=True, context=\"fork\")\n case VectorEnvType.SUBPROC_SHARED_MEM_AUTO:\n if platform.system().lower() == \"windows\":\n selected_venv_type = VectorEnvType.SUBPROC_SHARED_MEM_DEFAULT_CONTEXT\n else:\n selected_venv_type = VectorEnvType.SUBPROC_SHARED_MEM_FORK_CONTEXT\n return selected_venv_type.create_venv(factories)\n case VectorEnvType.RAY:\n return RayVectorEnv(factories)\n case _:\n raise NotImplementedError(self)\n\n\nclass Environments(ToStringMixin, ABC):\n \"\"\"Represents (vectorized) environments for a learning process.\"\"\"\n\n def __init__(\n self,\n env: gym.Env,\n training_envs: BaseVectorEnv,\n test_envs: BaseVectorEnv,\n watch_env: BaseVectorEnv | None = None,\n ):\n self.env = env\n self.training_envs = training_envs\n self.test_envs = test_envs\n self.watch_env = watch_env\n self.persistence: Sequence[Persistence] = []\n\n @staticmethod\n def from_factory_and_type(\n factory_fn: Callable[[EnvMode], gym.Env],\n env_type: EnvType,\n venv_type: VectorEnvType,\n num_training_envs: int,\n num_test_envs: int,\n create_watch_env: bool = False,\n ) -> \"Environments\":\n \"\"\"Creates a suitable subtype instance from a factory function that creates a single instance and the type of environment (continuous/discrete).\n\n :param factory_fn: the factory for a single environment instance\n :param env_type: the type of environments created by `factory_fn`\n :param venv_type: the vector environment type to use for parallelization\n :param num_training_envs: the number of training environments to create\n :param num_test_envs: the number of test environments to create\n :param create_watch_env: whether to create an environment for watching the agent\n :return: the instance\n \"\"\"\n training_envs = venv_type.create_venv(\n [lambda: factory_fn(EnvMode.TRAINING)] * num_training_envs,\n )\n test_envs = venv_type.create_venv(\n [lambda: factory_fn(EnvMode.TEST)] * num_test_envs,\n )\n if create_watch_env:\n watch_env = VectorEnvType.DUMMY.create_venv([lambda: factory_fn(EnvMode.WATCH)])\n else:\n watch_env = None\n env = factory_fn(EnvMode.TRAINING)\n match env_type:\n case EnvType.CONTINUOUS:\n return ContinuousEnvironments(env, training_envs, test_envs, watch_env)\n case EnvType.DISCRETE:\n return DiscreteEnvironments(env, training_envs, test_envs, watch_env)\n case _:\n raise ValueError(f\"Environment type {env_type} not handled\")\n\n def _tostring_includes(self) -> list[str]:\n return []\n\n def _tostring_additional_entries(self) -> dict[str, Any]:\n return self.info()\n\n def info(self) -> dict[str, Any]:\n return {\n \"action_shape\": self.get_action_shape(),\n \"state_shape\": self.get_observation_shape(),\n }\n\n def set_persistence(self, *p: Persistence) -> None:\n \"\"\"Associates the given persistence handlers which may persist and restore environment-specific information.\n\n :param p: persistence handlers\n \"\"\"\n self.persistence = p\n\n @abstractmethod\n def get_action_shape(self) -> TActionShape:\n pass\n\n @abstractmethod\n def get_observation_shape(self) -> TObservationShape:\n pass\n\n def get_action_space(self) -> gym.Space:\n return self.env.action_space\n\n def get_observation_space(self) -> gym.Space:\n return self.env.observation_space\n\n @abstractmethod\n def get_type(self) -> EnvType:\n pass\n\n\nclass ContinuousEnvironments(Environments):\n \"\"\"Represents (vectorized) continuous environments.\"\"\"\n\n def __init__(\n self,\n env: gym.Env,\n training_envs: BaseVectorEnv,\n test_envs: BaseVectorEnv,\n watch_env: BaseVectorEnv | None = None,\n ):\n super().__init__(env, training_envs, test_envs, watch_env)\n self.state_shape, self.action_shape, self.max_action = self._get_continuous_env_info(env)\n\n @staticmethod\n def from_factory(\n factory_fn: Callable[[EnvMode], gym.Env],\n venv_type: VectorEnvType,\n num_training_envs: int,\n num_test_envs: int,\n create_watch_env: bool = False,\n ) -> \"ContinuousEnvironments\":\n \"\"\"Creates an instance from a factory function that creates a single instance.\n\n :param factory_fn: the factory for a single environment instance\n :param venv_type: the vector environment type to use for parallelization\n :param num_training_envs: the number of training environments to create\n :param num_test_envs: the number of test environments to create\n :param create_watch_env: whether to create an environment for watching the agent\n :return: the instance\n \"\"\"\n return cast(\n ContinuousEnvironments,\n Environments.from_factory_and_type(\n factory_fn,\n EnvType.CONTINUOUS,\n venv_type,\n num_training_envs,\n num_test_envs,\n create_watch_env,\n ),\n )\n\n def info(self) -> dict[str, Any]:\n d = super().info()\n d[\"max_action\"] = self.max_action\n return d\n\n @staticmethod\n def _get_continuous_env_info(\n env: gym.Env,\n ) -> tuple[tuple[int, ...], tuple[int, ...], float]:\n if not isinstance(env.action_space, gym.spaces.Box):\n raise ValueError(\n \"Only environments with continuous action space are supported here. \"\n f\"But got env with action space: {env.action_space.__class__}.\",\n )\n state_shape = env.observation_space.shape or env.observation_space.n # type: ignore\n if not state_shape:\n raise ValueError(\"Observation space shape is not defined\")\n action_shape = env.action_space.shape\n max_action = env.action_space.high[0]\n return state_shape, action_shape, max_action\n\n def get_action_shape(self) -> TActionShape:\n return self.action_shape\n\n def get_observation_shape(self) -> TObservationShape:\n return self.state_shape\n\n def get_type(self) -> EnvType:\n return EnvType.CONTINUOUS\n\n\nclass DiscreteEnvironments(Environments):\n \"\"\"Represents (vectorized) discrete environments.\"\"\"\n\n def __init__(\n self,\n env: gym.Env,\n training_envs: BaseVectorEnv,\n test_envs: BaseVectorEnv,\n watch_env: BaseVectorEnv | None = None,\n ):\n super().__init__(env, training_envs, test_envs, watch_env)\n self.observation_shape = env.observation_space.shape or env.observation_space.n # type: ignore\n self.action_shape = env.action_space.shape or env.action_space.n # type: ignore\n\n @staticmethod\n def from_factory(\n factory_fn: Callable[[EnvMode], gym.Env],\n venv_type: VectorEnvType,\n num_training_envs: int,\n num_test_envs: int,\n create_watch_env: bool = False,\n ) -> \"DiscreteEnvironments\":\n \"\"\"Creates an instance from a factory function that creates a single instance.\n\n :param factory_fn: the factory for a single environment instance\n :param venv_type: the vector environment type to use for parallelization\n :param num_training_envs: the number of training environments to create\n :param num_test_envs: the number of test environments to create\n :param create_watch_env: whether to create an environment for watching the agent\n :return: the instance\n \"\"\"\n return cast(\n DiscreteEnvironments,\n Environments.from_factory_and_type(\n factory_fn,\n EnvType.DISCRETE,\n venv_type,\n num_training_envs,\n num_test_envs,\n create_watch_env,\n ),\n )\n\n def get_action_shape(self) -> TActionShape:\n return self.action_shape\n\n def get_observation_shape(self) -> TObservationShape:\n return self.observation_shape\n\n def get_type(self) -> EnvType:\n return EnvType.DISCRETE\n\n\nclass EnvPoolFactory:\n \"\"\"A factory for the creation of envpool-based vectorized environments, which can be used in conjunction\n with :class:`EnvFactoryRegistered`.\n \"\"\"\n\n def _transform_task(self, task: str) -> str:\n return task\n\n def _transform_kwargs(self, kwargs: dict, mode: EnvMode) -> dict:\n \"\"\"Transforms gymnasium keyword arguments to be envpool-compatible.\n\n :param kwargs: keyword arguments that would normally be passed to `gymnasium.make`.\n :param mode: the environment mode\n :return: the transformed keyword arguments\n \"\"\"\n kwargs = dict(kwargs)\n if \"render_mode\" in kwargs:\n del kwargs[\"render_mode\"]\n return kwargs\n\n def create_venv(\n self,\n task: str,\n num_envs: int,\n mode: EnvMode,\n seed: int,\n kwargs: dict,\n ) -> BaseVectorEnv:\n import envpool\n\n envpool_task = self._transform_task(task)\n envpool_kwargs = self._transform_kwargs(kwargs, mode)\n return envpool.make_gymnasium(\n envpool_task,\n num_envs=num_envs,\n seed=seed,\n **envpool_kwargs,\n )\n\n\nclass EnvFactory(ToStringMixin, ABC):\n def __init__(self, venv_type: VectorEnvType):\n \"\"\"Main interface for the creation of environments (in various forms).\n\n :param venv_type: the type of vectorized environment to use for train and test environments.\n `WATCH` environments are always created as `DUMMY` vector environments.\n \"\"\"\n self.venv_type = venv_type\n\n @staticmethod\n def _create_rng(seed: int | None) -> np.random.Generator:\n \"\"\"\n Creates a random number generator with the given seed.\n\n :param seed: the seed to use; if None, a random seed will be used\n :return: the random number generator\n \"\"\"\n return np.random.default_rng(seed=seed)\n\n @staticmethod\n def _next_seed(rng: np.random.Generator) -> int:\n \"\"\"\n Samples a random seed from the given random number generator.\n\n :param rng: the random number generator\n :return: the sampled random seed\n \"\"\"\n # int32 is needed for envpool compatibility\n return int(rng.integers(0, 2**31, dtype=np.int32))\n\n @abstractmethod\n def _create_env(self, mode: EnvMode) -> Env:\n \"\"\"Creates a single environment for the given mode.\n\n :param mode: the mode\n :return: an environment\n \"\"\"\n\n def create_env(self, mode: EnvMode, seed: int | None = None) -> Env:\n \"\"\"\n Creates a single environment for the given mode.\n\n :param mode: the mode\n :param seed: the random seed to use for the environment; if None, the seed will not be specified,\n and gymnasium will use a random seed.\n :return: the environment\n \"\"\"\n env = self._create_env(mode)\n\n # initialize the environment with the given seed (if any)\n if seed is not None:\n rng = self._create_rng(seed)\n env.np_random = rng\n # also set the seed member within the environment such that it can be retrieved\n # (gymnasium's random seed handling is, unfortunately, broken)\n if hasattr(env, \"_np_random_seed\"):\n env._np_random_seed = seed\n\n return env\n\n def create_venv(self, num_envs: int, mode: EnvMode, seed: int | None = None) -> BaseVectorEnv:\n \"\"\"Create vectorized environments.\n\n :param num_envs: the number of environments\n :param mode: the mode for which to create.\n In `WATCH` mode the resulting venv will always be of type `DUMMY` with a single env.\n\n :return: the vectorized environments\n \"\"\"\n rng = self._create_rng(seed)\n\n def create_factory_fn() -> Callable[[], Env]:\n # create a factory function that uses a sampled random seed\n return lambda random_seed=self._next_seed(rng): self.create_env(mode, seed=random_seed) # type: ignore\n\n # create the vectorized environment, seeded appropriately\n if mode == EnvMode.WATCH:\n venv = VectorEnvType.DUMMY.create_venv([create_factory_fn()])\n else:\n venv = self.venv_type.create_venv([create_factory_fn() for _ in range(num_envs)])\n\n # seed the action samplers\n venv.seed([self._next_seed(rng) for _ in range(num_envs)])\n\n return venv\n\n def create_envs(\n self,\n num_training_envs: int,\n num_test_envs: int,\n create_watch_env: bool = False,\n seed: int | None = None,\n ) -> Environments:\n \"\"\"Create environments for learning.\n\n :param num_training_envs: the number of training environments\n :param num_test_envs: the number of test environments\n :param create_watch_env: whether to create an environment for watching the agent\n :param seed: the random seed to use for environment creation\n :return: the environments\n \"\"\"\n rng = self._create_rng(seed)\n env = self.create_env(EnvMode.TRAINING)\n training_envs = self.create_venv(\n num_training_envs, EnvMode.TRAINING, seed=self._next_seed(rng)\n )\n test_envs = self.create_venv(num_test_envs, EnvMode.TEST, seed=self._next_seed(rng))\n watch_env = (\n self.create_venv(1, EnvMode.WATCH, seed=self._next_seed(rng))\n if create_watch_env\n else None\n )\n match EnvType.from_env(env):\n case EnvType.DISCRETE:\n return DiscreteEnvironments(env, training_envs, test_envs, watch_env)\n case EnvType.CONTINUOUS:\n return ContinuousEnvironments(env, training_envs, test_envs, watch_env)\n case _:\n raise ValueError\n\n\nclass EnvFactoryRegistered(EnvFactory):\n \"\"\"Factory for environments that are registered with gymnasium and thus can be created via `gymnasium.make`\n (or via `envpool.make_gymnasium`).\n \"\"\"\n\n def __init__(\n self,\n *,\n task: str,\n venv_type: VectorEnvType,\n envpool_factory: EnvPoolFactory | None = None,\n render_mode_training: str | None = None,\n render_mode_test: str | None = None,\n render_mode_watch: str = \"human\",\n **make_kwargs: Any,\n ):\n \"\"\":param task: the gymnasium task/environment identifier\n :param seed: the random seed\n :param venv_type: the type of vectorized environment to use (if `envpool_factory` is not specified)\n :param envpool_factory: the factory to use for vectorized environment creation based on envpool; envpool must be installed.\n :param render_mode_training: the render mode to use for training environments\n :param render_mode_test: the render mode to use for test environments\n :param render_mode_watch: the render mode to use for environments that are used to watch agent performance\n :param make_kwargs: additional keyword arguments to pass on to `gymnasium.make`. If envpool is used, the gymnasium parameters will be appropriately translated for use with `envpool.make_gymnasium`.\n \"\"\"\n super().__init__(venv_type)\n self.task = task\n self.envpool_factory = envpool_factory\n self.render_modes = {\n EnvMode.TRAINING: render_mode_training,\n EnvMode.TEST: render_mode_test,\n EnvMode.WATCH: render_mode_watch,\n }\n self.make_kwargs = make_kwargs\n\n def __setstate__(self, state: dict) -> None:\n if \"seed\" in state:\n if \"test_seed\" in state or \"training_seed\" in state:\n raise RuntimeError(\n f\"Cannot have both 'seed' and 'test_seed'/'training_seed' in state. \"\n f\"Something went wrong during serialization/deserialization: \"\n f\"{state=}\",\n )\n state[\"test_seed\"] = state[\"seed\"]\n state[\"training_seed\"] = state[\"seed\"]\n del state[\"seed\"]\n if \"train_seed\" in state:\n state[\"training_seed\"] = state[\"train_seed\"]\n del state[\"train_seed\"]\n setstate(EnvFactoryRegistered, self, state)\n\n def _create_kwargs(self, mode: EnvMode) -> dict:\n \"\"\"Adapts the keyword arguments for the given mode.\n\n :param mode: the mode\n :return: adapted keyword arguments\n \"\"\"\n kwargs = dict(self.make_kwargs)\n kwargs[\"render_mode\"] = self.render_modes.get(mode)\n return kwargs\n\n def _create_env(self, mode: EnvMode) -> Env:\n \"\"\"Creates a single environment for the given mode.\n\n :param mode: the mode\n :return: an environment\n \"\"\"\n kwargs = self._create_kwargs(mode)\n return gymnasium.make(self.task, **kwargs)\n\n def create_venv(self, num_envs: int, mode: EnvMode, seed: int | None = None) -> BaseVectorEnv:\n if self.envpool_factory is not None:\n rng = self._create_rng(seed)\n return self.envpool_factory.create_venv(\n self.task,\n num_envs,\n mode,\n self._next_seed(rng),\n self._create_kwargs(mode),\n )\n else:\n return super().create_venv(num_envs, mode, seed=seed)\n" }, { "path": "tianshou/highlevel/experiment.py", "content": "\"\"\"The experiment module provides high-level interfaces for setting up and running reinforcement learning experiments.\n\nThe main entry points are:\n\n* :class:`ExperimentConfig`: a dataclass for configuring the experiment. The configuration is\n different from RL specific configuration (such as policy and trainer parameters)\n and only pertains to configuration that is common to all experiments.\n* :class:`Experiment`: represents a reinforcement learning experiment.\n It is composed of configuration and factory objects, is lightweight and serializable.\n An instance of `Experiment` is usually saved as a pickle file after an experiment is executed.\n* :class:`ExperimentBuilder`: a helper class for creating experiments. It contains a lot of defaults\n and allows for easy customization of the experiment setup.\n* :class:`ExperimentCollection`: a shallow wrapper around a list of experiments providing a\n simple interface for running them with a launcher. Useful for running multiple experiments in parallel, in\n particular, for the important case of running experiments that only differ in their random seeds.\n\nVarious implementations of the `ExperimentBuilder` are provided for each of the algorithms supported by Tianshou.\n\"\"\"\n\nimport os\nimport pickle\nfrom abc import ABC, abstractmethod\nfrom collections.abc import Sequence\nfrom contextlib import suppress\nfrom copy import deepcopy\nfrom dataclasses import asdict, dataclass\nfrom pprint import pformat\nfrom typing import TYPE_CHECKING, Generic, Self\n\nimport numpy as np\nimport torch\nfrom sensai.util import logging\nfrom sensai.util.logging import datetime_tag\nfrom sensai.util.string import ToStringMixin\n\nfrom tianshou.algorithm import Algorithm\nfrom tianshou.data import BaseCollector, Collector, CollectStats, InfoStats\nfrom tianshou.env import BaseVectorEnv\nfrom tianshou.evaluation.launcher import ExpLauncher, RegisteredExpLauncher\nfrom tianshou.evaluation.rliable_evaluation import load_and_eval_experiment\nfrom tianshou.highlevel.algorithm import (\n A2CAlgorithmFactory,\n AlgorithmFactory,\n DDPGAlgorithmFactory,\n DiscreteSACAlgorithmFactory,\n DQNAlgorithmFactory,\n IQNAlgorithmFactory,\n NPGAlgorithmFactory,\n PPOAlgorithmFactory,\n REDQAlgorithmFactory,\n ReinforceAlgorithmFactory,\n SACAlgorithmFactory,\n TD3AlgorithmFactory,\n TRPOAlgorithmFactory,\n TTrainingConfig,\n)\nfrom tianshou.highlevel.config import (\n OffPolicyTrainingConfig,\n OnPolicyTrainingConfig,\n TrainingConfig,\n)\nfrom tianshou.highlevel.env import EnvFactory\nfrom tianshou.highlevel.logger import LoggerFactory, LoggerFactoryDefault, TLogger\nfrom tianshou.highlevel.module.actor import (\n ActorFactory,\n ActorFactoryDefault,\n ActorFactoryTransientStorageDecorator,\n ActorFuture,\n ActorFutureProviderProtocol,\n ContinuousActorType,\n IntermediateModuleFactoryFromActorFactory,\n)\nfrom tianshou.highlevel.module.core import (\n TDevice,\n)\nfrom tianshou.highlevel.module.critic import (\n CriticEnsembleFactory,\n CriticEnsembleFactoryDefault,\n CriticFactory,\n CriticFactoryDefault,\n CriticFactoryReuseActor,\n)\nfrom tianshou.highlevel.module.intermediate import IntermediateModuleFactory\nfrom tianshou.highlevel.module.special import ImplicitQuantileNetworkFactory\nfrom tianshou.highlevel.params.algorithm_params import (\n A2CParams,\n DDPGParams,\n DiscreteSACParams,\n DQNParams,\n IQNParams,\n NPGParams,\n PPOParams,\n REDQParams,\n ReinforceParams,\n SACParams,\n TD3Params,\n TRPOParams,\n)\nfrom tianshou.highlevel.params.algorithm_wrapper import AlgorithmWrapperFactory\nfrom tianshou.highlevel.params.collector import CollectorFactory\nfrom tianshou.highlevel.params.optim import (\n OptimizerFactoryFactory,\n OptimizerFactoryFactoryAdam,\n)\nfrom tianshou.highlevel.persistence import (\n PersistenceGroup,\n PolicyPersistence,\n)\nfrom tianshou.highlevel.trainer import (\n EpochStopCallback,\n EpochTestCallback,\n EpochTrainCallback,\n TrainerCallbacks,\n)\nfrom tianshou.highlevel.world import World\nfrom tianshou.utils import LazyLogger\nfrom tianshou.utils.net.common import ModuleType\n\nif TYPE_CHECKING:\n from tianshou.evaluation.launcher import ExpLauncher, RegisteredExpLauncher\n\nlog = logging.getLogger(__name__)\n\n\n@dataclass\nclass ExperimentConfig:\n \"\"\"Generic config for setting up the experiment, not RL or training specific.\"\"\"\n\n seed: int = 42\n \"\"\"The random seed with which to initialize random number generators.\"\"\"\n device: TDevice = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n \"\"\"The torch device to use\"\"\"\n policy_restore_directory: str | None = None\n \"\"\"Directory from which to load the policy neural network parameters (persistence directory of a previous run)\"\"\"\n train: bool = True\n \"\"\"Whether to perform training\"\"\"\n watch: bool = True\n \"\"\"Whether to watch agent performance (after training)\"\"\"\n watch_num_episodes: int = 10\n \"\"\"Number of episodes for which to watch performance (if `watch` is enabled)\"\"\"\n watch_render: float = 0.0\n \"\"\"Milliseconds between rendered frames when watching agent performance (if `watch` is enabled)\"\"\"\n persistence_base_dir: str = \"log\"\n \"\"\"Base directory in which experiment data is to be stored. Every experiment run will create a subdirectory\n in this directory based on the run's experiment name\"\"\"\n persistence_enabled: bool = True\n \"\"\"Whether persistence is enabled, allowing files to be stored\"\"\"\n log_file_enabled: bool = True\n \"\"\"Whether to write to a log file; has no effect if `persistence_enabled` is False.\n Disable this if you have externally configured log file generation.\"\"\"\n policy_persistence_mode: PolicyPersistence.Mode = PolicyPersistence.Mode.POLICY\n \"\"\"Controls the way in which the policy is persisted\"\"\"\n\n\n@dataclass\nclass ExperimentResult:\n \"\"\"Contains the results of an experiment.\"\"\"\n\n world: World\n \"\"\"The `World` contains all the essential instances of the experiment.\n Can also be created via `Experiment.create_experiment_world` for more custom setups, see docstring there.\n\n Note: it is typically not serializable, so it is not stored in the experiment pickle, and shouldn't be\n sent across processes, meaning also that `ExperimentResult` itself is typically not serializable.\n \"\"\"\n trainer_result: InfoStats | None\n \"\"\"dataclass of results as returned by the trainer (if any)\"\"\"\n\n\nclass Experiment(ToStringMixin):\n \"\"\"Represents a reinforcement learning experiment.\n\n An experiment is composed only of configuration and factory objects, which themselves\n should be designed to contain only configuration. Therefore, experiments can easily\n be stored/pickled and later restored without any problems.\n\n The main entry points are:\n\n 1. :meth:`run`: runs the experiment and returns the results\n 2. :meth:`create_experiment_world`: creates the world object for the experiment, which contains all relevant instances.\n Useful for setting up the experiment and running it in a more custom way.\n\n The methods :meth:`save` and :meth:`from_directory` can be used to store and restore experiments.\n \"\"\"\n\n LOG_FILENAME = \"log.txt\"\n EXPERIMENT_PICKLE_FILENAME = \"experiment.pkl\"\n\n def __init__(\n self,\n config: ExperimentConfig,\n env_factory: EnvFactory,\n algorithm_factory: AlgorithmFactory,\n training_config: TrainingConfig,\n name: str,\n logger_factory: LoggerFactory | None = None,\n ):\n if logger_factory is None:\n logger_factory = LoggerFactoryDefault()\n self.config = config\n self.training_config = training_config\n self.env_factory = env_factory\n self.algorithm_factory = algorithm_factory\n self.logger_factory = logger_factory\n self.name = name\n\n @classmethod\n def from_directory(cls, directory: str, restore_policy: bool = True) -> \"Experiment\":\n \"\"\"Restores an experiment from a previously stored pickle.\n\n :param directory: persistence directory of a previous run, in which a pickled experiment is found\n :param restore_policy: whether the experiment shall be configured to restore the policy that was\n persisted in the given directory\n \"\"\"\n with open(os.path.join(directory, cls.EXPERIMENT_PICKLE_FILENAME), \"rb\") as f:\n experiment: Experiment = pickle.load(f)\n if restore_policy:\n experiment.config.policy_restore_directory = directory\n return experiment\n\n @staticmethod\n def seeding_info_str_static(seed: int) -> str:\n \"\"\"Static method variant of `get_seeding_info_as_str`, which can be used without an `Experiment` instance.\"\"\"\n return f\"exp_seed={seed}\"\n\n def get_seeding_info_as_str(self) -> str:\n \"\"\"Returns information on the seeds used in the experiment as a string.\n\n This can be useful for creating unique experiment names based on seeds, e.g.\n A typical example is to do `experiment.name = f\"{experiment.name}_{experiment.get_seeding_info_as_str()}\"`.\n \"\"\"\n return self.seeding_info_str_static(self.config.seed)\n\n def _set_seed(self) -> None:\n seed = self.config.seed\n log.info(f\"Setting random seed {seed}\")\n np.random.seed(seed)\n torch.manual_seed(seed)\n\n def _build_config_dict(self) -> dict:\n return {\"experiment\": self.pprints()}\n\n def save(self, directory: str) -> None:\n path = os.path.join(directory, self.EXPERIMENT_PICKLE_FILENAME)\n log.info(\n f\"Saving serialized experiment in {path}; can be restored via Experiment.from_directory('{directory}')\",\n )\n with open(path, \"wb\") as f:\n pickle.dump(self, f)\n\n @staticmethod\n def persistence_dir_static(\n persistence_base_dir: str, experiment_name: str, seed: int | None = None\n ) -> str:\n \"\"\"Static method for constructing the persistence directory for an experiment\n from the base persistence directory and the experiment name. Useful for contexts where one\n wants access to the persistence directory without having access to the corresponding `Experiment` instance.\n\n :param persistence_base_dir: base persistence directory\n :param experiment_name: name of the experiment\n :param seed: optional seed. Experiments are saved within a subdirectory named after the seed, but\n it is often sufficient to know the base directory without the seed subdirectory in user code\n (for example, for restoring logs or performing rliable evaluations)\n \"\"\"\n result = os.path.join(persistence_base_dir, experiment_name)\n if seed is not None:\n result = os.path.join(result, Experiment.seeding_info_str_static(seed))\n return result\n\n def create_experiment_world(\n self,\n override_experiment_name: str | None = None,\n logger_run_id: str | None = None,\n raise_error_on_dirname_collision: bool = True,\n reset_collectors: bool = True,\n ) -> World:\n \"\"\"Creates the world object for the experiment.\n\n The world object contains all relevant instances for the experiment,\n such as environments, policy, collectors, etc.\n This method is the main entrypoint for users who don't want to use `run` directly. A common use case\n is that some configuration or custom logic should happen before the training loop starts, but one\n still wants to use the convenience of high-level interfaces for setting up the experiment.\n\n :param override_experiment_name: pass to override the experiment name in the resulting `World`.\n Affects the name of the persistence directory and logger configuration. If None, the experiment's\n name will be used.\n The name may contain path separators (i.e. `os.path.sep`, as used by `os.path.join`), in which case\n a nested directory structure will be created.\n :param logger_run_id: Run identifier to use for logger initialization/resumption.\n :param raise_error_on_dirname_collision: whether to raise an error on collisions when creating the\n persistence directory. Only takes effect if persistence is enabled. Set to `False` e.g., when continuing\n a previously executed experiment with the same `persistence_base_dir` and name.\n :param reset_collectors: whether to reset the collectors before training starts.\n Setting to `False` can be useful when continuing training from a previous run with restored collectors,\n or for adding custom logic before training starts.\n \"\"\"\n if override_experiment_name is not None:\n exp_name = override_experiment_name\n else:\n exp_name = self.name\n\n # initialize persistence directory\n use_persistence = self.config.persistence_enabled\n persistence_dir = os.path.join(\n self.config.persistence_base_dir, exp_name, self.get_seeding_info_as_str()\n )\n if use_persistence:\n os.makedirs(persistence_dir, exist_ok=not raise_error_on_dirname_collision)\n\n with logging.FileLoggerContext(\n os.path.join(persistence_dir, self.LOG_FILENAME),\n enabled=use_persistence and self.config.log_file_enabled,\n ):\n # log initial information\n log.info(f\"Preparing experiment world (name='{exp_name}'):\\n{self.pprints()}\")\n log.info(f\"Working directory: {os.getcwd()}\")\n\n self._set_seed()\n\n # create environments\n envs = self.env_factory.create_envs(\n self.training_config.num_training_envs,\n self.training_config.num_test_envs,\n create_watch_env=self.config.watch,\n seed=self.config.seed,\n )\n log.info(f\"Created {envs}\")\n\n # initialize persistence\n additional_persistence = PersistenceGroup(*envs.persistence, enabled=use_persistence)\n policy_persistence = PolicyPersistence(\n additional_persistence,\n enabled=use_persistence,\n mode=self.config.policy_persistence_mode,\n )\n if use_persistence:\n log.info(f\"Persistence directory: {os.path.abspath(persistence_dir)}\")\n self.save(persistence_dir)\n\n # initialize logger\n full_config = self._build_config_dict()\n full_config.update(envs.info())\n full_config[\"experiment_config\"] = asdict(self.config)\n full_config[\"training_config_config\"] = asdict(self.training_config)\n with suppress(AttributeError):\n full_config[\"policy_params\"] = asdict(self.algorithm_factory.params)\n\n logger: TLogger\n if use_persistence:\n logger = self.logger_factory.create_logger(\n log_dir=persistence_dir,\n experiment_name=exp_name,\n run_id=logger_run_id,\n config_dict=full_config,\n )\n else:\n logger = LazyLogger()\n\n # create policy and collectors\n log.info(\"Creating policy\")\n policy = self.algorithm_factory.create_algorithm(envs, self.config.device)\n\n log.info(\"Creating collectors\")\n training_collector: BaseCollector | None = None\n test_collector: BaseCollector | None = None\n if self.config.train:\n (\n training_collector,\n test_collector,\n ) = self.algorithm_factory.create_train_test_collectors(\n policy,\n envs,\n reset_collectors=reset_collectors,\n )\n\n # create context object with all relevant instances (except trainer; added later)\n world = World(\n envs=envs,\n algorithm=policy,\n training_collector=training_collector,\n test_collector=test_collector,\n logger=logger,\n persist_directory=persistence_dir,\n restore_directory=self.config.policy_restore_directory,\n )\n\n # restore policy parameters if applicable\n if self.config.policy_restore_directory:\n policy_persistence.restore(\n policy,\n world,\n self.config.device,\n )\n\n if self.config.train:\n trainer = self.algorithm_factory.create_trainer(world, policy_persistence)\n world.trainer = trainer\n\n return world\n\n def run(\n self,\n run_name: str | None = None,\n logger_run_id: str | None = None,\n raise_error_on_dirname_collision: bool = True,\n ) -> ExperimentResult:\n \"\"\"Run the experiment and return the results.\n\n :param run_name: Defines a name for this run of the experiment, which determines\n the subdirectory (within the persistence base directory) where all results will be saved.\n If None, the experiment's name will be used.\n The name may contain path separators (i.e. `os.path.sep`, as used by `os.path.join`), in which case\n a nested directory structure will be created.\n :param logger_run_id: Run identifier to use for logger initialization/resumption (applies when\n using wandb, in particular).\n :param raise_error_on_dirname_collision: set to `False` e.g., when continuing a previously executed\n experiment with the same name.\n :return:\n \"\"\"\n if run_name is None:\n run_name = self.name\n\n world = self.create_experiment_world(\n override_experiment_name=run_name,\n logger_run_id=logger_run_id,\n raise_error_on_dirname_collision=raise_error_on_dirname_collision,\n )\n\n persistence_dir = world.persist_directory\n use_persistence = self.config.persistence_enabled\n\n with logging.FileLoggerContext(\n os.path.join(persistence_dir, self.LOG_FILENAME),\n enabled=use_persistence and self.config.log_file_enabled,\n ):\n trainer_result: InfoStats | None = None\n if self.config.train:\n assert world.trainer is not None\n assert world.training_collector is not None\n assert world.test_collector is not None\n\n # prefilling buffers with either random or current agent's actions\n if self.training_config.start_timesteps > 0:\n log.info(\n f\"Collecting {self.training_config.start_timesteps} initial environment \"\n f\"steps before training (random={self.training_config.start_timesteps_random})\",\n )\n world.training_collector.collect(\n n_step=self.training_config.start_timesteps,\n random=self.training_config.start_timesteps_random,\n )\n\n log.info(\"Starting training\")\n world.trainer.run()\n if use_persistence:\n world.logger.finalize()\n log.info(f\"Training result:\\n{pformat(trainer_result)}\")\n\n # watch agent performance\n if self.config.watch:\n assert world.envs.watch_env is not None\n log.info(\"Watching agent performance\")\n self._watch_agent(\n self.config.watch_num_episodes,\n world.algorithm,\n world.envs.watch_env,\n self.config.watch_render,\n )\n\n return ExperimentResult(world=world, trainer_result=trainer_result)\n\n @staticmethod\n def _watch_agent(\n num_episodes: int,\n policy: Algorithm,\n env: BaseVectorEnv,\n render: float,\n ) -> None:\n collector = Collector[CollectStats](policy, env)\n collector.reset()\n result = collector.collect(n_episode=num_episodes, render=render)\n assert result.returns_stat is not None # for mypy\n assert result.lens_stat is not None # for mypy\n log.info(\n f\"Watched episodes: mean reward={result.returns_stat.mean}, mean episode length={result.lens_stat.mean}\",\n )\n\n\nclass ExperimentCollection:\n \"\"\"Shallow wrapper around a list of experiments providing a simple interface for running them with a launcher.\"\"\"\n\n def __init__(self, experiments: list[Experiment]):\n self.experiments = experiments\n\n def run(\n self,\n launcher: ExpLauncher | RegisteredExpLauncher | str = RegisteredExpLauncher.SEQUENTIAL,\n ) -> list[InfoStats | None]:\n if isinstance(launcher, str):\n launcher = RegisteredExpLauncher[launcher.upper()]\n if isinstance(launcher, RegisteredExpLauncher):\n launcher = launcher.create_launcher()\n log.info(\n f\"Running {len(self.experiments)} experiments using launcher {launcher.get_name()}\"\n )\n return launcher.launch(experiments=self.experiments)\n\n\nclass ExperimentBuilder(ABC, Generic[TTrainingConfig]):\n \"\"\"A helper class (following the builder pattern) for creating experiments.\n\n It contains a lot of defaults for the setup which can be adjusted using the\n various `with_` methods. For example, the default optimizer is Adam, but can be\n adjusted using :meth:`with_optim_factory`. Moreover, for simply configuring the default\n optimizer instead of using a different one, one can use :meth:`with_optim_factory_default`.\n \"\"\"\n\n def __init__(\n self,\n env_factory: EnvFactory,\n experiment_config: ExperimentConfig | None = None,\n training_config: TTrainingConfig | None = None,\n ):\n \"\"\":param env_factory: controls how environments are to be created.\n :param experiment_config: the configuration for the experiment. If None, will use the default values\n of `ExperimentConfig`.\n :param training_config: the training configuration to use. If None, use default values (not recommended).\n \"\"\"\n if experiment_config is None:\n experiment_config = ExperimentConfig()\n if training_config is None:\n training_config = self._create_training_config()\n\n self._config = experiment_config\n self._env_factory = env_factory\n self._training_config = training_config\n self._logger_factory: LoggerFactory | None = None\n self._optim_factory: OptimizerFactoryFactory | None = None\n self._algorithm_wrapper_factory: AlgorithmWrapperFactory | None = None\n self._collector_factory: CollectorFactory | None = None\n self._trainer_callbacks: TrainerCallbacks = TrainerCallbacks()\n self._name: str = self.__class__.__name__.replace(\"Builder\", \"\") + \"_\" + datetime_tag()\n\n @abstractmethod\n def _create_training_config(self) -> TTrainingConfig:\n pass\n\n def copy(self) -> Self:\n return deepcopy(self)\n\n @property\n def experiment_config(self) -> ExperimentConfig:\n return self._config\n\n @experiment_config.setter\n def experiment_config(self, experiment_config: ExperimentConfig) -> None:\n self._config = experiment_config\n\n @property\n def training_config(self) -> TrainingConfig:\n return self._training_config\n\n @training_config.setter\n def training_config(self, config: TrainingConfig) -> None:\n self._training_config = config\n\n def with_logger_factory(self, logger_factory: LoggerFactory) -> Self:\n \"\"\"Allows to customize the logger factory to use.\n\n If this method is not called, the default logger factory :class:`LoggerFactoryDefault` will be used.\n\n :param logger_factory: the factory to use\n :return: the builder\n \"\"\"\n self._logger_factory = logger_factory\n return self\n\n def with_algorithm_wrapper_factory(\n self, algorithm_wrapper_factory: AlgorithmWrapperFactory\n ) -> Self:\n \"\"\"Allows to define a wrapper around the algorithm that is created, extending the original algorithm.\n\n :param algorithm_wrapper_factory: the factory for the wrapper\n :return: the builder\n \"\"\"\n self._algorithm_wrapper_factory = algorithm_wrapper_factory\n return self\n\n def with_optim_default(self, optim_factory: OptimizerFactoryFactory) -> Self:\n \"\"\"Allows to customize the default optimizer to use.\n\n The default optimizer applies when optimizer factory factories are set to None\n in algorithm parameter objects.\n\n By default, :class:`OptimizerFactoryFactoryAdam` will be used with default parameters.\n\n :param optim_factory: the optimizer factory\n :return: the builder\n \"\"\"\n self._optim_factory = optim_factory\n return self\n\n def with_epoch_train_callback(self, callback: EpochTrainCallback) -> Self:\n \"\"\"Allows to define a callback function which is called at the beginning of every epoch during training.\n\n :param callback: the callback\n :return: the builder\n \"\"\"\n self._trainer_callbacks.epoch_train_callback = callback\n return self\n\n def with_epoch_test_callback(self, callback: EpochTestCallback) -> Self:\n \"\"\"Allows to define a callback function which is called at the beginning of testing in each epoch.\n\n :param callback: the callback\n :return: the builder\n \"\"\"\n self._trainer_callbacks.epoch_test_callback = callback\n return self\n\n def with_epoch_stop_callback(self, callback: EpochStopCallback) -> Self:\n \"\"\"Allows to define a callback that decides whether training shall stop early.\n\n The callback receives the undiscounted returns of the testing result.\n\n :param callback: the callback\n :return: the builder\n \"\"\"\n self._trainer_callbacks.epoch_stop_callback = callback\n return self\n\n def with_name(\n self,\n name: str,\n ) -> Self:\n \"\"\"Sets the name of the experiment.\n\n :param name: the name to use for this experiment, which, when the experiment is run,\n will determine the storage sub-folder by default\n :return: the builder\n \"\"\"\n self._name = name\n return self\n\n def with_collector_factory(self, collector_factory: CollectorFactory) -> Self:\n \"\"\"Allows customizing the collector factory to use.\n\n :param collector_factory: the factory to use for the creation of collectors\n :return: the builder\n \"\"\"\n self._collector_factory = collector_factory\n return self\n\n @abstractmethod\n def _create_algorithm_factory(self) -> AlgorithmFactory:\n pass\n\n def _get_optim_factory(self) -> OptimizerFactoryFactory:\n if self._optim_factory is None:\n return OptimizerFactoryFactoryAdam()\n else:\n return self._optim_factory\n\n def build(self) -> Experiment:\n \"\"\"Creates the experiment based on the options specified via this builder.\n\n :return: the experiment\n \"\"\"\n algorithm_factory = self._create_algorithm_factory()\n algorithm_factory.set_trainer_callbacks(self._trainer_callbacks)\n if self._algorithm_wrapper_factory:\n algorithm_factory.set_policy_wrapper_factory(self._algorithm_wrapper_factory)\n if self._collector_factory:\n algorithm_factory.set_collector_factory(self._collector_factory)\n experiment: Experiment = Experiment(\n config=self._config,\n env_factory=self._env_factory,\n algorithm_factory=algorithm_factory,\n training_config=self._training_config,\n name=self._name,\n logger_factory=self._logger_factory,\n )\n return experiment\n\n def build_seeded_collection(self, num_experiments: int) -> ExperimentCollection:\n \"\"\"Creates a collection of experiments with non-overlapping random seeds, starting from the configured seed.\n\n Useful for performing statistically meaningful evaluations of an algorithm's performance.\n The `rliable` recommendation is to use at least 5 experiments for computing quantities such as the\n interquantile mean and performance profiles. See the usage in example scripts\n like `examples/mujoco/mujoco_ppo_hl_multi.py`.\n\n Each experiment in the collection will have a unique name created from the original experiment name\n and the seeds used.\n \"\"\"\n seeded_experiments = []\n for i in range(num_experiments):\n builder = self.copy()\n builder.experiment_config.seed += i\n experiment = builder.build()\n seeded_experiments.append(experiment)\n return ExperimentCollection(seeded_experiments)\n\n def build_and_run(\n self,\n num_experiments: int = 1,\n launcher: ExpLauncher | RegisteredExpLauncher | str = RegisteredExpLauncher.SEQUENTIAL,\n perform_rliable_analysis: bool = True,\n ) -> list[InfoStats | None]:\n \"\"\"Build and run experiments. With multiple experiments, the seeds will be non-overlapping and the parallelism is controlled by the launcher.\n\n :param num_experiments: the number of experiments to create and run\n :param launcher: the launcher (or the corresponding enum value) to use for running the experiments\n :param perform_rliable_analysis: whether to perform rliable analysis on the results (only applicable if\n `num_experiments > 1`). This will show plots and store them in the persistence directory.\n :return: list of results, one per experiment\n \"\"\"\n collection = self.build_seeded_collection(num_experiments)\n successful_experiment_stats = collection.run(launcher)\n num_successful_experiments = len(successful_experiment_stats)\n for i, info_stats in enumerate(successful_experiment_stats, start=1):\n if info_stats is not None:\n log.info(\n f\"Training stats for successful experiment {i}/{num_successful_experiments}:\"\n )\n log.info(info_stats.pprints_asdict())\n else:\n log.info(\n f\"No training stats available for successful experiment {i}/{num_successful_experiments}.\",\n )\n if perform_rliable_analysis and num_successful_experiments > 1:\n log.info(f\"Performing rliable evaluation over {num_successful_experiments} experiments\")\n persistence_dir = Experiment.persistence_dir_static(\n self._config.persistence_base_dir, self._name\n )\n load_and_eval_experiment(persistence_dir)\n return successful_experiment_stats\n\n\nclass OnPolicyExperimentBuilder(ExperimentBuilder[OnPolicyTrainingConfig], ABC):\n def __init__(\n self,\n env_factory: EnvFactory,\n experiment_config: ExperimentConfig | None = None,\n training_config: OnPolicyTrainingConfig | None = None,\n ):\n \"\"\"\n :param env_factory: controls how environments are to be created.\n :param experiment_config: the configuration for the experiment. If None, will use the default values\n of :class:`ExperimentConfig`.\n :param training_config: the training configuration to use. If None, use default values (not recommended).\n \"\"\"\n super().__init__(env_factory, experiment_config, training_config)\n\n def _create_training_config(self) -> OnPolicyTrainingConfig:\n return OnPolicyTrainingConfig()\n\n\nclass OffPolicyExperimentBuilder(ExperimentBuilder[OffPolicyTrainingConfig], ABC):\n def __init__(\n self,\n env_factory: EnvFactory,\n experiment_config: ExperimentConfig | None = None,\n training_config: OffPolicyTrainingConfig | None = None,\n ):\n \"\"\"\n :param env_factory: controls how environments are to be created.\n :param experiment_config: the configuration for the experiment. If None, will use the default values\n of :class:`ExperimentConfig`.\n :param training_config: the training configuration to use. If None, use default values (not recommended).\n \"\"\"\n super().__init__(env_factory, experiment_config, training_config)\n\n def _create_training_config(self) -> OffPolicyTrainingConfig:\n return OffPolicyTrainingConfig()\n\n\nclass _BuilderMixinActorFactory(ActorFutureProviderProtocol):\n def __init__(self, continuous_actor_type: ContinuousActorType):\n self._continuous_actor_type = continuous_actor_type\n self._actor_future = ActorFuture()\n self._actor_factory: ActorFactory | None = None\n\n def with_actor_factory(self, actor_factory: ActorFactory) -> Self:\n \"\"\"Allows customizing the actor component via the specification of a factory.\n\n If this function is not called, a default actor factory (with default parameters) will be used.\n\n :param actor_factory: the factory to use for the creation of the actor network\n :return: the builder\n \"\"\"\n self._actor_factory = actor_factory\n return self\n\n def _with_actor_factory_default(\n self,\n hidden_sizes: Sequence[int],\n hidden_activation: ModuleType = torch.nn.ReLU,\n continuous_unbounded: bool = False,\n continuous_conditioned_sigma: bool = False,\n ) -> Self:\n \"\"\"Adds a default actor factory with the given parameters.\n\n :param hidden_sizes: the sequence of hidden dimensions to use in the network structure\n :param continuous_unbounded: whether, for continuous action spaces, to apply tanh activation on final logits\n :param continuous_conditioned_sigma: whether, for continuous action spaces, the standard deviation of continuous actions (sigma)\n shall be computed from the input; if False, sigma is an independent parameter.\n :return: the builder\n \"\"\"\n self._actor_factory = ActorFactoryDefault(\n self._continuous_actor_type,\n hidden_sizes,\n hidden_activation=hidden_activation,\n continuous_unbounded=continuous_unbounded,\n continuous_conditioned_sigma=continuous_conditioned_sigma,\n )\n return self\n\n def get_actor_future(self) -> ActorFuture:\n \"\"\":return: an object, which, in the future, will contain the actor instance that is created for the experiment.\"\"\"\n return self._actor_future\n\n def _get_actor_factory(self) -> ActorFactory:\n actor_factory: ActorFactory\n if self._actor_factory is None:\n actor_factory = ActorFactoryDefault(self._continuous_actor_type)\n else:\n actor_factory = self._actor_factory\n return ActorFactoryTransientStorageDecorator(actor_factory, self._actor_future)\n\n\nclass _BuilderMixinActorFactory_ContinuousGaussian(_BuilderMixinActorFactory):\n \"\"\"Specialization of the actor mixin where, in the continuous case, the actor component outputs Gaussian distribution parameters.\"\"\"\n\n def __init__(self) -> None:\n super().__init__(ContinuousActorType.GAUSSIAN)\n\n def with_actor_factory_default(\n self,\n hidden_sizes: Sequence[int],\n hidden_activation: ModuleType = torch.nn.ReLU,\n continuous_unbounded: bool = False,\n continuous_conditioned_sigma: bool = False,\n ) -> Self:\n \"\"\"Defines use of the default actor factory, allowing its parameters it to be customized.\n\n The default actor factory uses an MLP-style architecture.\n\n :param hidden_sizes: dimensions of hidden layers used by the network\n :param hidden_activation: the activation function to use for hidden layers\n :param continuous_unbounded: whether, for continuous action spaces, to apply tanh activation on final logits\n :param continuous_conditioned_sigma: whether, for continuous action spaces, the standard deviation of continuous actions (sigma)\n shall be computed from the input; if False, sigma is an independent parameter.\n :return: the builder\n \"\"\"\n return super()._with_actor_factory_default(\n hidden_sizes,\n hidden_activation=hidden_activation,\n continuous_unbounded=continuous_unbounded,\n continuous_conditioned_sigma=continuous_conditioned_sigma,\n )\n\n\nclass _BuilderMixinActorFactory_ContinuousDeterministic(_BuilderMixinActorFactory):\n \"\"\"Specialization of the actor mixin where, in the continuous case, the actor uses a deterministic policy.\"\"\"\n\n def __init__(self) -> None:\n super().__init__(ContinuousActorType.DETERMINISTIC)\n\n def with_actor_factory_default(\n self,\n hidden_sizes: Sequence[int],\n hidden_activation: ModuleType = torch.nn.ReLU,\n ) -> Self:\n \"\"\"Defines use of the default actor factory, allowing its parameters it to be customized.\n\n The default actor factory uses an MLP-style architecture.\n\n :param hidden_sizes: dimensions of hidden layers used by the network\n :param hidden_activation: the activation function to use for hidden layers\n :return: the builder\n \"\"\"\n return super()._with_actor_factory_default(hidden_sizes, hidden_activation)\n\n\nclass _BuilderMixinActorFactory_DiscreteOnly(_BuilderMixinActorFactory):\n \"\"\"Specialization of the actor mixin where only environments with discrete action spaces are supported.\"\"\"\n\n def __init__(self) -> None:\n super().__init__(ContinuousActorType.UNSUPPORTED)\n\n def with_actor_factory_default(\n self,\n hidden_sizes: Sequence[int],\n hidden_activation: ModuleType = torch.nn.ReLU,\n ) -> Self:\n \"\"\"Defines use of the default actor factory, allowing its parameters it to be customized.\n\n The default actor factory uses an MLP-style architecture.\n\n :param hidden_sizes: dimensions of hidden layers used by the network\n :param hidden_activation: the activation function to use for hidden layers\n :return: the builder\n \"\"\"\n return super()._with_actor_factory_default(hidden_sizes, hidden_activation)\n\n\nclass _BuilderMixinCriticsFactory:\n def __init__(self, num_critics: int, actor_future_provider: ActorFutureProviderProtocol):\n self._actor_future_provider = actor_future_provider\n self._critic_factories: list[CriticFactory | None] = [None] * num_critics\n\n def _with_critic_factory(self, idx: int, critic_factory: CriticFactory) -> Self:\n self._critic_factories[idx] = critic_factory\n return self\n\n def _with_critic_factory_default(\n self,\n idx: int,\n hidden_sizes: Sequence[int],\n hidden_activation: ModuleType = torch.nn.ReLU,\n ) -> Self:\n self._critic_factories[idx] = CriticFactoryDefault(\n hidden_sizes,\n hidden_activation=hidden_activation,\n )\n return self\n\n def _with_critic_factory_use_actor(self, idx: int) -> Self:\n self._critic_factories[idx] = CriticFactoryReuseActor(\n self._actor_future_provider.get_actor_future(),\n )\n return self\n\n def _get_critic_factory(self, idx: int) -> CriticFactory:\n factory = self._critic_factories[idx]\n if factory is None:\n return CriticFactoryDefault()\n else:\n return factory\n\n\nclass _BuilderMixinSingleCriticFactory(_BuilderMixinCriticsFactory):\n def __init__(self, actor_future_provider: ActorFutureProviderProtocol) -> None:\n super().__init__(1, actor_future_provider)\n\n def with_critic_factory(self, critic_factory: CriticFactory) -> Self:\n \"\"\"Specifies that the given factory shall be used for the critic.\n\n :param critic_factory: the critic factory\n :return: the builder\n \"\"\"\n self._with_critic_factory(0, critic_factory)\n return self\n\n def with_critic_factory_default(\n self,\n hidden_sizes: Sequence[int] = CriticFactoryDefault.DEFAULT_HIDDEN_SIZES,\n hidden_activation: ModuleType = torch.nn.ReLU,\n ) -> Self:\n \"\"\"Makes the critic use the default, MLP-style architecture with the given parameters.\n\n :param hidden_sizes: the sequence of dimensions to use in hidden layers of the network\n :param hidden_activation: the activation function to use for hidden layers\n :return: the builder\n \"\"\"\n self._with_critic_factory_default(0, hidden_sizes, hidden_activation)\n return self\n\n\nclass _BuilderMixinSingleCriticCanUseActorFactory(_BuilderMixinSingleCriticFactory):\n def __init__(self, actor_future_provider: ActorFutureProviderProtocol) -> None:\n super().__init__(actor_future_provider)\n\n def with_critic_factory_use_actor(self) -> Self:\n \"\"\"Makes the first critic reuse the actor's preprocessing network (parameter sharing).\"\"\"\n return self._with_critic_factory_use_actor(0)\n\n\nclass _BuilderMixinDualCriticFactory(_BuilderMixinCriticsFactory):\n def __init__(self, actor_future_provider: ActorFutureProviderProtocol) -> None:\n super().__init__(2, actor_future_provider)\n\n def with_common_critic_factory(self, critic_factory: CriticFactory) -> Self:\n \"\"\"Specifies that the given factory shall be used for both critics.\n\n :param critic_factory: the critic factory\n :return: the builder\n \"\"\"\n for i in range(len(self._critic_factories)):\n self._with_critic_factory(i, critic_factory)\n return self\n\n def with_common_critic_factory_default(\n self,\n hidden_sizes: Sequence[int] = CriticFactoryDefault.DEFAULT_HIDDEN_SIZES,\n hidden_activation: ModuleType = torch.nn.ReLU,\n ) -> Self:\n \"\"\"Makes both critics use the default, MLP-style architecture with the given parameters.\n\n :param hidden_sizes: the sequence of dimensions to use in hidden layers of the network\n :param hidden_activation: the activation function to use for hidden layers\n :return: the builder\n \"\"\"\n for i in range(len(self._critic_factories)):\n self._with_critic_factory_default(i, hidden_sizes, hidden_activation)\n return self\n\n def with_common_critic_factory_use_actor(self) -> Self:\n \"\"\"Makes both critics reuse the actor's preprocessing network (parameter sharing).\"\"\"\n for i in range(len(self._critic_factories)):\n self._with_critic_factory_use_actor(i)\n return self\n\n def with_critic1_factory(self, critic_factory: CriticFactory) -> Self:\n \"\"\"Specifies that the given factory shall be used for the first critic.\n\n :param critic_factory: the critic factory\n :return: the builder\n \"\"\"\n self._with_critic_factory(0, critic_factory)\n return self\n\n def with_critic1_factory_default(\n self,\n hidden_sizes: Sequence[int] = CriticFactoryDefault.DEFAULT_HIDDEN_SIZES,\n hidden_activation: ModuleType = torch.nn.ReLU,\n ) -> Self:\n \"\"\"Makes the first critic use the default, MLP-style architecture with the given parameters.\n\n :param hidden_sizes: the sequence of dimensions to use in hidden layers of the network\n :param hidden_activation: the activation function to use for hidden layers\n :return: the builder\n \"\"\"\n self._with_critic_factory_default(0, hidden_sizes, hidden_activation)\n return self\n\n def with_critic1_factory_use_actor(self) -> Self:\n \"\"\"Makes the first critic reuse the actor's preprocessing network (parameter sharing).\"\"\"\n return self._with_critic_factory_use_actor(0)\n\n def with_critic2_factory(self, critic_factory: CriticFactory) -> Self:\n \"\"\"Specifies that the given factory shall be used for the second critic.\n\n :param critic_factory: the critic factory\n :return: the builder\n \"\"\"\n self._with_critic_factory(1, critic_factory)\n return self\n\n def with_critic2_factory_default(\n self,\n hidden_sizes: Sequence[int] = CriticFactoryDefault.DEFAULT_HIDDEN_SIZES,\n hidden_activation: ModuleType = torch.nn.ReLU,\n ) -> Self:\n \"\"\"Makes the second critic use the default, MLP-style architecture with the given parameters.\n\n :param hidden_sizes: the sequence of dimensions to use in hidden layers of the network\n :param hidden_activation: the activation function to use for hidden layers\n :return: the builder\n \"\"\"\n self._with_critic_factory_default(1, hidden_sizes, hidden_activation)\n return self\n\n def with_critic2_factory_use_actor(self) -> Self:\n \"\"\"Makes the second critic reuse the actor's preprocessing network (parameter sharing).\"\"\"\n return self._with_critic_factory_use_actor(1)\n\n\nclass _BuilderMixinCriticEnsembleFactory:\n def __init__(self) -> None:\n self.critic_ensemble_factory: CriticEnsembleFactory | None = None\n\n def with_critic_ensemble_factory(self, factory: CriticEnsembleFactory) -> Self:\n \"\"\"Specifies that the given factory shall be used for the critic ensemble.\n\n If unspecified, the default factory (:class:`CriticEnsembleFactoryDefault`) is used.\n\n :param factory: the critic ensemble factory\n :return: the builder\n \"\"\"\n self.critic_ensemble_factory = factory\n return self\n\n def with_critic_ensemble_factory_default(\n self,\n hidden_sizes: Sequence[int] = CriticFactoryDefault.DEFAULT_HIDDEN_SIZES,\n ) -> Self:\n \"\"\"Allows to customize the parameters of the default critic ensemble factory.\n\n :param hidden_sizes: the sequence of sizes of hidden layers in the network architecture\n :return: the builder\n \"\"\"\n self.critic_ensemble_factory = CriticEnsembleFactoryDefault(hidden_sizes)\n return self\n\n def _get_critic_ensemble_factory(self) -> CriticEnsembleFactory:\n if self.critic_ensemble_factory is None:\n return CriticEnsembleFactoryDefault()\n else:\n return self.critic_ensemble_factory\n\n\nclass ReinforceExperimentBuilder(\n OnPolicyExperimentBuilder,\n _BuilderMixinActorFactory_ContinuousGaussian,\n):\n def __init__(\n self,\n env_factory: EnvFactory,\n experiment_config: ExperimentConfig | None = None,\n training_config: OnPolicyTrainingConfig | None = None,\n ):\n super().__init__(env_factory, experiment_config, training_config)\n _BuilderMixinActorFactory_ContinuousGaussian.__init__(self)\n self._params: ReinforceParams = ReinforceParams()\n self._env_config = None\n\n def with_reinforce_params(self, params: ReinforceParams) -> Self:\n self._params = params\n return self\n\n def _create_algorithm_factory(self) -> AlgorithmFactory:\n return ReinforceAlgorithmFactory(\n self._params,\n self._training_config,\n self._get_actor_factory(),\n self._get_optim_factory(),\n )\n\n\nclass A2CExperimentBuilder(\n OnPolicyExperimentBuilder,\n _BuilderMixinActorFactory_ContinuousGaussian,\n _BuilderMixinSingleCriticCanUseActorFactory,\n):\n def __init__(\n self,\n env_factory: EnvFactory,\n experiment_config: ExperimentConfig | None = None,\n training_config: OnPolicyTrainingConfig | None = None,\n ):\n super().__init__(env_factory, experiment_config, training_config)\n _BuilderMixinActorFactory_ContinuousGaussian.__init__(self)\n _BuilderMixinSingleCriticCanUseActorFactory.__init__(self, self)\n self._params: A2CParams = A2CParams()\n self._env_config = None\n\n def with_a2c_params(self, params: A2CParams) -> Self:\n self._params = params\n return self\n\n def _create_algorithm_factory(self) -> AlgorithmFactory:\n return A2CAlgorithmFactory(\n self._params,\n self._training_config,\n self._get_actor_factory(),\n self._get_critic_factory(0),\n self._get_optim_factory(),\n )\n\n\nclass PPOExperimentBuilder(\n OnPolicyExperimentBuilder,\n _BuilderMixinActorFactory_ContinuousGaussian,\n _BuilderMixinSingleCriticCanUseActorFactory,\n):\n def __init__(\n self,\n env_factory: EnvFactory,\n experiment_config: ExperimentConfig | None = None,\n training_config: OnPolicyTrainingConfig | None = None,\n ):\n super().__init__(env_factory, experiment_config, training_config)\n _BuilderMixinActorFactory_ContinuousGaussian.__init__(self)\n _BuilderMixinSingleCriticCanUseActorFactory.__init__(self, self)\n self._params: PPOParams = PPOParams()\n\n def with_ppo_params(self, params: PPOParams) -> Self:\n self._params = params\n return self\n\n def _create_algorithm_factory(self) -> AlgorithmFactory:\n return PPOAlgorithmFactory(\n self._params,\n self._training_config,\n self._get_actor_factory(),\n self._get_critic_factory(0),\n self._get_optim_factory(),\n )\n\n\nclass NPGExperimentBuilder(\n OnPolicyExperimentBuilder,\n _BuilderMixinActorFactory_ContinuousGaussian,\n _BuilderMixinSingleCriticCanUseActorFactory,\n):\n def __init__(\n self,\n env_factory: EnvFactory,\n experiment_config: ExperimentConfig | None = None,\n training_config: OnPolicyTrainingConfig | None = None,\n ):\n super().__init__(env_factory, experiment_config, training_config)\n _BuilderMixinActorFactory_ContinuousGaussian.__init__(self)\n _BuilderMixinSingleCriticCanUseActorFactory.__init__(self, self)\n self._params: NPGParams = NPGParams()\n\n def with_npg_params(self, params: NPGParams) -> Self:\n self._params = params\n return self\n\n def _create_algorithm_factory(self) -> AlgorithmFactory:\n return NPGAlgorithmFactory(\n self._params,\n self._training_config,\n self._get_actor_factory(),\n self._get_critic_factory(0),\n self._get_optim_factory(),\n )\n\n\nclass TRPOExperimentBuilder(\n OnPolicyExperimentBuilder,\n _BuilderMixinActorFactory_ContinuousGaussian,\n _BuilderMixinSingleCriticCanUseActorFactory,\n):\n def __init__(\n self,\n env_factory: EnvFactory,\n experiment_config: ExperimentConfig | None = None,\n training_config: OnPolicyTrainingConfig | None = None,\n ):\n super().__init__(env_factory, experiment_config, training_config)\n _BuilderMixinActorFactory_ContinuousGaussian.__init__(self)\n _BuilderMixinSingleCriticCanUseActorFactory.__init__(self, self)\n self._params: TRPOParams = TRPOParams()\n\n def with_trpo_params(self, params: TRPOParams) -> Self:\n self._params = params\n return self\n\n def _create_algorithm_factory(self) -> AlgorithmFactory:\n return TRPOAlgorithmFactory(\n self._params,\n self._training_config,\n self._get_actor_factory(),\n self._get_critic_factory(0),\n self._get_optim_factory(),\n )\n\n\nclass DQNExperimentBuilder(\n OffPolicyExperimentBuilder,\n):\n def __init__(\n self,\n env_factory: EnvFactory,\n experiment_config: ExperimentConfig | None = None,\n training_config: OffPolicyTrainingConfig | None = None,\n ):\n super().__init__(env_factory, experiment_config, training_config)\n self._params: DQNParams = DQNParams()\n self._model_factory: IntermediateModuleFactory = IntermediateModuleFactoryFromActorFactory(\n ActorFactoryDefault(ContinuousActorType.UNSUPPORTED, discrete_softmax=False),\n )\n\n def with_dqn_params(self, params: DQNParams) -> Self:\n self._params = params\n return self\n\n def with_model_factory(self, module_factory: IntermediateModuleFactory) -> Self:\n \"\"\":param module_factory: factory for a module which maps environment observations to a vector of Q-values (one for each action)\n :return: the builder\n \"\"\"\n self._model_factory = module_factory\n return self\n\n def with_model_factory_default(\n self,\n hidden_sizes: Sequence[int],\n hidden_activation: ModuleType = torch.nn.ReLU,\n ) -> Self:\n \"\"\"Allows to configure the default factory for the model of the Q function, which maps environment observations to a vector of\n Q-values (one for each action). The default model is a multi-layer perceptron.\n\n :param hidden_sizes: the sequence of dimensions used for hidden layers\n :param hidden_activation: the activation function to use for hidden layers (not used for the output layer)\n :return: the builder\n \"\"\"\n self._model_factory = IntermediateModuleFactoryFromActorFactory(\n ActorFactoryDefault(\n ContinuousActorType.UNSUPPORTED,\n hidden_sizes=hidden_sizes,\n hidden_activation=hidden_activation,\n discrete_softmax=False,\n ),\n )\n return self\n\n def _create_algorithm_factory(self) -> AlgorithmFactory:\n return DQNAlgorithmFactory(\n self._params,\n self._training_config,\n self._model_factory,\n self._get_optim_factory(),\n )\n\n\nclass IQNExperimentBuilder(OffPolicyExperimentBuilder):\n def __init__(\n self,\n env_factory: EnvFactory,\n experiment_config: ExperimentConfig | None = None,\n training_config: OffPolicyTrainingConfig | None = None,\n ):\n super().__init__(env_factory, experiment_config, training_config)\n self._params: IQNParams = IQNParams()\n self._preprocess_network_factory: IntermediateModuleFactory = (\n IntermediateModuleFactoryFromActorFactory(\n ActorFactoryDefault(ContinuousActorType.UNSUPPORTED, discrete_softmax=False),\n )\n )\n\n def with_iqn_params(self, params: IQNParams) -> Self:\n self._params = params\n return self\n\n def with_preprocess_network_factory(self, module_factory: IntermediateModuleFactory) -> Self:\n self._preprocess_network_factory = module_factory\n return self\n\n def _create_algorithm_factory(self) -> AlgorithmFactory:\n model_factory = ImplicitQuantileNetworkFactory(\n self._preprocess_network_factory,\n hidden_sizes=self._params.hidden_sizes,\n num_cosines=self._params.num_cosines,\n )\n return IQNAlgorithmFactory(\n self._params,\n self._training_config,\n model_factory,\n self._get_optim_factory(),\n )\n\n\nclass DDPGExperimentBuilder(\n OffPolicyExperimentBuilder,\n _BuilderMixinActorFactory_ContinuousDeterministic,\n _BuilderMixinSingleCriticCanUseActorFactory,\n):\n def __init__(\n self,\n env_factory: EnvFactory,\n experiment_config: ExperimentConfig | None = None,\n training_config: OffPolicyTrainingConfig | None = None,\n ):\n super().__init__(env_factory, experiment_config, training_config)\n _BuilderMixinActorFactory_ContinuousDeterministic.__init__(self)\n _BuilderMixinSingleCriticCanUseActorFactory.__init__(self, self)\n self._params: DDPGParams = DDPGParams()\n\n def with_ddpg_params(self, params: DDPGParams) -> Self:\n self._params = params\n return self\n\n def _create_algorithm_factory(self) -> AlgorithmFactory:\n return DDPGAlgorithmFactory(\n self._params,\n self._training_config,\n self._get_actor_factory(),\n self._get_critic_factory(0),\n self._get_optim_factory(),\n )\n\n\nclass REDQExperimentBuilder(\n OffPolicyExperimentBuilder,\n _BuilderMixinActorFactory_ContinuousGaussian,\n _BuilderMixinCriticEnsembleFactory,\n):\n def __init__(\n self,\n env_factory: EnvFactory,\n experiment_config: ExperimentConfig | None = None,\n training_config: OffPolicyTrainingConfig | None = None,\n ):\n super().__init__(env_factory, experiment_config, training_config)\n _BuilderMixinActorFactory_ContinuousGaussian.__init__(self)\n _BuilderMixinCriticEnsembleFactory.__init__(self)\n self._params: REDQParams = REDQParams()\n\n def with_redq_params(self, params: REDQParams) -> Self:\n self._params = params\n return self\n\n def _create_algorithm_factory(self) -> AlgorithmFactory:\n return REDQAlgorithmFactory(\n self._params,\n self._training_config,\n self._get_actor_factory(),\n self._get_critic_ensemble_factory(),\n self._get_optim_factory(),\n )\n\n\nclass SACExperimentBuilder(\n OffPolicyExperimentBuilder,\n _BuilderMixinActorFactory_ContinuousGaussian,\n _BuilderMixinDualCriticFactory,\n):\n def __init__(\n self,\n env_factory: EnvFactory,\n experiment_config: ExperimentConfig | None = None,\n training_config: OffPolicyTrainingConfig | None = None,\n ):\n super().__init__(env_factory, experiment_config, training_config)\n _BuilderMixinActorFactory_ContinuousGaussian.__init__(self)\n _BuilderMixinDualCriticFactory.__init__(self, self)\n self._params: SACParams = SACParams()\n\n def with_sac_params(self, params: SACParams) -> Self:\n self._params = params\n return self\n\n def _create_algorithm_factory(self) -> AlgorithmFactory:\n return SACAlgorithmFactory(\n self._params,\n self._training_config,\n self._get_actor_factory(),\n self._get_critic_factory(0),\n self._get_critic_factory(1),\n self._get_optim_factory(),\n )\n\n\nclass DiscreteSACExperimentBuilder(\n OffPolicyExperimentBuilder,\n _BuilderMixinActorFactory_DiscreteOnly,\n _BuilderMixinDualCriticFactory,\n):\n def __init__(\n self,\n env_factory: EnvFactory,\n experiment_config: ExperimentConfig | None = None,\n training_config: OffPolicyTrainingConfig | None = None,\n ):\n super().__init__(env_factory, experiment_config, training_config)\n _BuilderMixinActorFactory_DiscreteOnly.__init__(self)\n _BuilderMixinDualCriticFactory.__init__(self, self)\n self._params: DiscreteSACParams = DiscreteSACParams()\n\n def with_sac_params(self, params: DiscreteSACParams) -> Self:\n self._params = params\n return self\n\n def _create_algorithm_factory(self) -> AlgorithmFactory:\n return DiscreteSACAlgorithmFactory(\n self._params,\n self._training_config,\n self._get_actor_factory(),\n self._get_critic_factory(0),\n self._get_critic_factory(1),\n self._get_optim_factory(),\n )\n\n\nclass TD3ExperimentBuilder(\n OffPolicyExperimentBuilder,\n _BuilderMixinActorFactory_ContinuousDeterministic,\n _BuilderMixinDualCriticFactory,\n):\n def __init__(\n self,\n env_factory: EnvFactory,\n experiment_config: ExperimentConfig | None = None,\n training_config: OffPolicyTrainingConfig | None = None,\n ):\n super().__init__(env_factory, experiment_config, training_config)\n _BuilderMixinActorFactory_ContinuousDeterministic.__init__(self)\n _BuilderMixinDualCriticFactory.__init__(self, self)\n self._params: TD3Params = TD3Params()\n\n def with_td3_params(self, params: TD3Params) -> Self:\n self._params = params\n return self\n\n def _create_algorithm_factory(self) -> AlgorithmFactory:\n return TD3AlgorithmFactory(\n self._params,\n self._training_config,\n self._get_actor_factory(),\n self._get_critic_factory(0),\n self._get_critic_factory(1),\n self._get_optim_factory(),\n )\n" }, { "path": "tianshou/highlevel/logger.py", "content": "import os\nfrom abc import ABC, abstractmethod\nfrom typing import Literal, TypeAlias\n\nfrom sensai.util.string import ToStringMixin\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom tianshou.utils import BaseLogger, TensorboardLogger, WandbLogger\n\nTLogger: TypeAlias = BaseLogger\n\n\nclass LoggerFactory(ToStringMixin, ABC):\n @abstractmethod\n def create_logger(\n self,\n log_dir: str,\n experiment_name: str,\n run_id: str | None,\n config_dict: dict | None = None,\n ) -> TLogger:\n \"\"\"Creates the logger.\n\n :param log_dir: path to the directory in which log data is to be stored\n :param experiment_name: the name of the job, which may contain `os.path.delimiter`\n :param run_id: a unique name, which, depending on the logging framework, may be used to identify the logger\n :param config_dict: a dictionary with data that is to be logged\n :return: the logger\n \"\"\"\n\n @abstractmethod\n def get_logger_class(self) -> type[TLogger]:\n \"\"\"Returns the class of the logger that is to be created.\"\"\"\n\n\nclass LoggerFactoryDefault(LoggerFactory):\n \"\"\"\n :param save_interval: the interval size (in env steps) after which the checkpoint and end\n of epoch related logs will be saved.\n \"\"\"\n\n def __init__(\n self,\n logger_type: Literal[\"tensorboard\", \"wandb\", \"pandas\"] = \"tensorboard\",\n wand_entity: str | None = None,\n wandb_project: str | None = None,\n group: str | None = None,\n job_type: str | None = None,\n save_interval: int | None = None,\n ):\n if logger_type == \"wandb\" and wandb_project is None:\n raise ValueError(\"Must provide 'wandb_project'\")\n self.logger_type = logger_type\n self.wandb_entity = wand_entity\n self.wandb_project = wandb_project\n self.group = group\n self.job_type = job_type\n self.save_interval = save_interval\n\n def create_logger(\n self,\n log_dir: str,\n experiment_name: str,\n run_id: str | None,\n config_dict: dict | None = None,\n ) -> TLogger:\n match self.logger_type:\n case \"wandb\":\n logger = WandbLogger(\n save_interval=self.save_interval,\n name=experiment_name.replace(os.path.sep, \"__\"),\n run_id=run_id,\n config=config_dict,\n entity=self.wandb_entity,\n project=self.wandb_project,\n group=self.group,\n job_type=self.job_type,\n log_dir=log_dir,\n )\n writer = self._create_writer(log_dir) # writer has to be created after wandb.init!\n logger.load(writer)\n return logger\n case \"tensorboard\":\n writer = self._create_writer(log_dir)\n return TensorboardLogger(writer, save_interval=self.save_interval)\n case _:\n raise ValueError(f\"Unknown logger type '{self.logger_type}'\")\n\n def _create_writer(self, log_dir: str) -> SummaryWriter:\n \"\"\"Creates a tensorboard writer and adds a text artifact.\"\"\"\n writer = SummaryWriter(log_dir)\n writer.add_text(\n \"args\",\n str(\n dict(\n log_dir=log_dir,\n logger_type=self.logger_type,\n wandb_project=self.wandb_project,\n ),\n ),\n )\n return writer\n\n def get_logger_class(self) -> type[TLogger]:\n match self.logger_type:\n case \"wandb\":\n return WandbLogger\n case \"tensorboard\":\n return TensorboardLogger\n case _:\n raise ValueError(f\"Unknown logger type '{self.logger_type}'\")\n" }, { "path": "tianshou/highlevel/module/__init__.py", "content": "" }, { "path": "tianshou/highlevel/module/actor.py", "content": "from abc import ABC, abstractmethod\nfrom collections.abc import Sequence\nfrom dataclasses import dataclass\nfrom enum import Enum\nfrom typing import Protocol\n\nimport torch\nfrom sensai.util.string import ToStringMixin\nfrom torch import nn\n\nfrom tianshou.algorithm.modelfree.reinforce import TDistFnDiscrOrCont\nfrom tianshou.highlevel.env import Environments, EnvType\nfrom tianshou.highlevel.module.core import (\n ModuleFactory,\n TDevice,\n init_linear_orthogonal,\n)\nfrom tianshou.highlevel.module.intermediate import (\n IntermediateModule,\n IntermediateModuleFactory,\n)\nfrom tianshou.highlevel.params.dist_fn import (\n DistributionFunctionFactoryCategorical,\n DistributionFunctionFactoryIndependentGaussians,\n)\nfrom tianshou.utils.net import continuous, discrete\nfrom tianshou.utils.net.common import (\n Actor,\n ModuleType,\n ModuleWithVectorOutput,\n Net,\n)\n\n\nclass ContinuousActorType(Enum):\n GAUSSIAN = \"gaussian\"\n DETERMINISTIC = \"deterministic\"\n UNSUPPORTED = \"unsupported\"\n\n\n@dataclass\nclass ActorFuture:\n \"\"\"Container, which, in the future, will hold an actor instance.\"\"\"\n\n actor: Actor | nn.Module | None = None\n\n\nclass ActorFutureProviderProtocol(Protocol):\n def get_actor_future(self) -> ActorFuture:\n pass\n\n\nclass ActorFactory(ModuleFactory, ToStringMixin, ABC):\n @abstractmethod\n def create_module(self, envs: Environments, device: TDevice) -> Actor | nn.Module:\n pass\n\n @abstractmethod\n def create_dist_fn(self, envs: Environments) -> TDistFnDiscrOrCont | None:\n \"\"\"\n :param envs: the environments\n :return: the distribution function, which converts the actor's output into a distribution, or None\n if the actor does not output distribution parameters\n \"\"\"\n\n @staticmethod\n def _init_linear(actor: torch.nn.Module) -> None:\n \"\"\"Initializes linear layers of an actor module using default mechanisms.\n\n :param actor: the actor module.\n \"\"\"\n init_linear_orthogonal(actor)\n if hasattr(actor, \"mu\"):\n # For continuous action spaces with Gaussian policies\n # do last policy layer scaling, this will make initial actions have (close to)\n # 0 mean and std, and will help boost performances,\n # see https://arxiv.org/abs/2006.05990, Fig.24 for details\n for m in actor.mu.modules():\n if isinstance(m, torch.nn.Linear):\n m.weight.data.copy_(0.01 * m.weight.data)\n\n\nclass ActorFactoryDefault(ActorFactory):\n \"\"\"An actor factory which, depending on the type of environment, creates a suitable MLP-based policy.\"\"\"\n\n DEFAULT_HIDDEN_SIZES = (64, 64)\n\n def __init__(\n self,\n continuous_actor_type: ContinuousActorType,\n hidden_sizes: Sequence[int] = DEFAULT_HIDDEN_SIZES,\n hidden_activation: ModuleType = nn.ReLU,\n continuous_unbounded: bool = False,\n continuous_conditioned_sigma: bool = False,\n discrete_softmax: bool = True,\n ):\n self.continuous_actor_type = continuous_actor_type\n self.continuous_unbounded = continuous_unbounded\n self.continuous_conditioned_sigma = continuous_conditioned_sigma\n self.hidden_sizes = hidden_sizes\n self.hidden_activation = hidden_activation\n self.discrete_softmax = discrete_softmax\n\n def _create_factory(self, envs: Environments) -> ActorFactory:\n env_type = envs.get_type()\n factory: (\n ActorFactoryContinuousDeterministicNet\n | ActorFactoryContinuousGaussianNet\n | ActorFactoryDiscreteNet\n )\n if env_type == EnvType.CONTINUOUS:\n match self.continuous_actor_type:\n case ContinuousActorType.GAUSSIAN:\n factory = ActorFactoryContinuousGaussianNet(\n self.hidden_sizes,\n activation=self.hidden_activation,\n unbounded=self.continuous_unbounded,\n conditioned_sigma=self.continuous_conditioned_sigma,\n )\n case ContinuousActorType.DETERMINISTIC:\n factory = ActorFactoryContinuousDeterministicNet(\n self.hidden_sizes,\n activation=self.hidden_activation,\n )\n case ContinuousActorType.UNSUPPORTED:\n raise ValueError(\"Continuous action spaces are not supported by the algorithm\")\n case _:\n raise ValueError(self.continuous_actor_type)\n elif env_type == EnvType.DISCRETE:\n factory = ActorFactoryDiscreteNet(\n self.hidden_sizes,\n activation=self.hidden_activation,\n softmax_output=self.discrete_softmax,\n )\n else:\n raise ValueError(f\"{env_type} not supported\")\n return factory\n\n def create_module(self, envs: Environments, device: TDevice) -> Actor | nn.Module:\n factory = self._create_factory(envs)\n return factory.create_module(envs, device)\n\n def create_dist_fn(self, envs: Environments) -> TDistFnDiscrOrCont | None:\n factory = self._create_factory(envs)\n return factory.create_dist_fn(envs)\n\n\nclass ActorFactoryContinuous(ActorFactory, ABC):\n \"\"\"Serves as a type bound for actor factories that are suitable for continuous action spaces.\"\"\"\n\n\nclass ActorFactoryContinuousDeterministicNet(ActorFactoryContinuous):\n def __init__(self, hidden_sizes: Sequence[int], activation: ModuleType = nn.ReLU):\n self.hidden_sizes = hidden_sizes\n self.activation = activation\n\n def create_module(self, envs: Environments, device: TDevice) -> Actor:\n net_a = Net(\n state_shape=envs.get_observation_shape(),\n hidden_sizes=self.hidden_sizes,\n activation=self.activation,\n )\n return continuous.ContinuousActorDeterministic(\n preprocess_net=net_a,\n action_shape=envs.get_action_shape(),\n hidden_sizes=(),\n ).to(device)\n\n def create_dist_fn(self, envs: Environments) -> TDistFnDiscrOrCont | None:\n return None\n\n\nclass ActorFactoryContinuousGaussianNet(ActorFactoryContinuous):\n def __init__(\n self,\n hidden_sizes: Sequence[int],\n unbounded: bool = True,\n conditioned_sigma: bool = False,\n activation: ModuleType = nn.ReLU,\n ):\n \"\"\"For actors with Gaussian policies.\n\n :param hidden_sizes: the sequence of hidden dimensions to use in the network structure\n :param unbounded: whether to apply tanh activation on final logits\n :param conditioned_sigma: if True, the standard deviation of continuous actions (sigma) is computed from the\n input; if False, sigma is an independent parameter\n \"\"\"\n self.hidden_sizes = hidden_sizes\n self.unbounded = unbounded\n self.conditioned_sigma = conditioned_sigma\n self.activation = activation\n\n def create_module(self, envs: Environments, device: TDevice) -> Actor:\n net_a = Net(\n state_shape=envs.get_observation_shape(),\n hidden_sizes=self.hidden_sizes,\n activation=self.activation,\n )\n actor = continuous.ContinuousActorProbabilistic(\n preprocess_net=net_a,\n action_shape=envs.get_action_shape(),\n unbounded=self.unbounded,\n conditioned_sigma=self.conditioned_sigma,\n ).to(device)\n\n # init params\n if not self.conditioned_sigma:\n torch.nn.init.constant_(actor.sigma_param, -0.5)\n self._init_linear(actor)\n\n return actor\n\n def create_dist_fn(self, envs: Environments) -> TDistFnDiscrOrCont | None:\n return DistributionFunctionFactoryIndependentGaussians().create_dist_fn(envs)\n\n\nclass ActorFactoryDiscreteNet(ActorFactory):\n def __init__(\n self,\n hidden_sizes: Sequence[int],\n softmax_output: bool = True,\n activation: ModuleType = nn.ReLU,\n ):\n self.hidden_sizes = hidden_sizes\n self.softmax_output = softmax_output\n self.activation = activation\n\n def create_module(self, envs: Environments, device: TDevice) -> Actor:\n net_a = Net(\n state_shape=envs.get_observation_shape(),\n hidden_sizes=self.hidden_sizes,\n activation=self.activation,\n )\n return discrete.DiscreteActor(\n preprocess_net=net_a,\n action_shape=envs.get_action_shape(),\n hidden_sizes=(),\n softmax_output=self.softmax_output,\n ).to(device)\n\n def create_dist_fn(self, envs: Environments) -> TDistFnDiscrOrCont | None:\n return DistributionFunctionFactoryCategorical(\n is_probs_input=self.softmax_output,\n ).create_dist_fn(envs)\n\n\nclass ActorFactoryTransientStorageDecorator(ActorFactory):\n \"\"\"Wraps an actor factory, storing the most recently created actor instance such that it can be retrieved.\"\"\"\n\n def __init__(self, actor_factory: ActorFactory, actor_future: ActorFuture):\n self.actor_factory = actor_factory\n self._actor_future = actor_future\n\n def __getstate__(self) -> dict:\n d = dict(self.__dict__)\n del d[\"_actor_future\"]\n return d\n\n def __setstate__(self, state: dict) -> None:\n self.__dict__ = state\n self._actor_future = ActorFuture()\n\n def _tostring_excludes(self) -> list[str]:\n return [*super()._tostring_excludes(), \"_actor_future\"]\n\n def create_module(self, envs: Environments, device: TDevice) -> Actor | nn.Module:\n module = self.actor_factory.create_module(envs, device)\n self._actor_future.actor = module\n return module\n\n def create_dist_fn(self, envs: Environments) -> TDistFnDiscrOrCont | None:\n return self.actor_factory.create_dist_fn(envs)\n\n\nclass IntermediateModuleFactoryFromActorFactory(IntermediateModuleFactory):\n def __init__(self, actor_factory: ActorFactory):\n self.actor_factory = actor_factory\n\n def create_intermediate_module(self, envs: Environments, device: TDevice) -> IntermediateModule:\n actor = self.actor_factory.create_module(envs, device)\n assert isinstance(actor, ModuleWithVectorOutput), (\n \"Actor factory must produce an actor with known vector output dimension\"\n )\n return IntermediateModule(actor, actor.get_output_dim())\n" }, { "path": "tianshou/highlevel/module/core.py", "content": "from abc import ABC, abstractmethod\nfrom typing import TypeAlias\n\nimport numpy as np\nimport torch\n\nfrom tianshou.highlevel.env import Environments\n\nTDevice: TypeAlias = str | torch.device\n\n\ndef init_linear_orthogonal(module: torch.nn.Module) -> None:\n \"\"\"Applies orthogonal initialization to linear layers of the given module and sets bias weights to 0.\n\n :param module: the module whose submodules are to be processed\n \"\"\"\n for m in module.modules():\n if isinstance(m, torch.nn.Linear):\n torch.nn.init.orthogonal_(m.weight, gain=np.sqrt(2))\n torch.nn.init.zeros_(m.bias)\n\n\nclass ModuleFactory(ABC):\n \"\"\"Represents a factory for the creation of a torch module given an environment and target device.\"\"\"\n\n @abstractmethod\n def create_module(self, envs: Environments, device: TDevice) -> torch.nn.Module:\n pass\n" }, { "path": "tianshou/highlevel/module/critic.py", "content": "from abc import ABC, abstractmethod\nfrom collections.abc import Sequence\n\nimport numpy as np\nfrom sensai.util.string import ToStringMixin\nfrom torch import nn\n\nfrom tianshou.highlevel.env import Environments, EnvType\nfrom tianshou.highlevel.module.actor import ActorFuture\nfrom tianshou.highlevel.module.core import TDevice, init_linear_orthogonal\nfrom tianshou.utils.net import continuous\nfrom tianshou.utils.net.common import Actor, EnsembleLinear, ModuleType, Net\nfrom tianshou.utils.net.continuous import ContinuousCritic\nfrom tianshou.utils.net.discrete import DiscreteCritic\n\n\nclass CriticFactory(ToStringMixin, ABC):\n \"\"\"Represents a factory for the generation of a critic module.\"\"\"\n\n @abstractmethod\n def create_module(\n self,\n envs: Environments,\n device: TDevice,\n use_action: bool,\n discrete_last_size_use_action_shape: bool = False,\n ) -> nn.Module:\n \"\"\"Creates the critic module.\n\n :param envs: the environments\n :param device: the torch device\n :param use_action: whether to expect the action as an additional input (in addition to the observations)\n :param discrete_last_size_use_action_shape: whether, for the discrete case, the output dimension shall use the action shape\n :return: the module\n \"\"\"\n\n\nclass CriticFactoryDefault(CriticFactory):\n \"\"\"A critic factory which, depending on the type of environment, creates a suitable MLP-based critic.\"\"\"\n\n DEFAULT_HIDDEN_SIZES = (64, 64)\n\n def __init__(\n self,\n hidden_sizes: Sequence[int] = DEFAULT_HIDDEN_SIZES,\n hidden_activation: ModuleType = nn.ReLU,\n ):\n self.hidden_sizes = hidden_sizes\n self.hidden_activation = hidden_activation\n\n def create_module(\n self,\n envs: Environments,\n device: TDevice,\n use_action: bool,\n discrete_last_size_use_action_shape: bool = False,\n ) -> nn.Module:\n factory: CriticFactory\n env_type = envs.get_type()\n match env_type:\n case EnvType.CONTINUOUS:\n factory = CriticFactoryContinuousNet(\n self.hidden_sizes,\n activation=self.hidden_activation,\n )\n case EnvType.DISCRETE:\n factory = CriticFactoryDiscreteNet(\n self.hidden_sizes,\n activation=self.hidden_activation,\n )\n case _:\n raise ValueError(f\"{env_type} not supported\")\n return factory.create_module(\n envs,\n device,\n use_action,\n discrete_last_size_use_action_shape=discrete_last_size_use_action_shape,\n )\n\n\nclass CriticFactoryContinuousNet(CriticFactory):\n def __init__(self, hidden_sizes: Sequence[int], activation: ModuleType = nn.ReLU):\n self.hidden_sizes = hidden_sizes\n self.activation = activation\n\n def create_module(\n self,\n envs: Environments,\n device: TDevice,\n use_action: bool,\n discrete_last_size_use_action_shape: bool = False,\n ) -> nn.Module:\n action_shape = envs.get_action_shape() if use_action else 0\n net_c = Net(\n state_shape=envs.get_observation_shape(),\n action_shape=action_shape,\n hidden_sizes=self.hidden_sizes,\n concat=use_action,\n activation=self.activation,\n )\n critic = continuous.ContinuousCritic(preprocess_net=net_c).to(device)\n init_linear_orthogonal(critic)\n return critic\n\n\nclass CriticFactoryDiscreteNet(CriticFactory):\n def __init__(self, hidden_sizes: Sequence[int], activation: ModuleType = nn.ReLU):\n self.hidden_sizes = hidden_sizes\n self.activation = activation\n\n def create_module(\n self,\n envs: Environments,\n device: TDevice,\n use_action: bool,\n discrete_last_size_use_action_shape: bool = False,\n ) -> nn.Module:\n action_shape = envs.get_action_shape() if use_action else 0\n net_c = Net(\n state_shape=envs.get_observation_shape(),\n action_shape=action_shape,\n hidden_sizes=self.hidden_sizes,\n concat=use_action,\n activation=self.activation,\n )\n last_size = (\n int(np.prod(envs.get_action_shape())) if discrete_last_size_use_action_shape else 1\n )\n critic = DiscreteCritic(preprocess_net=net_c, last_size=last_size).to(device)\n init_linear_orthogonal(critic)\n return critic\n\n\nclass CriticFactoryReuseActor(CriticFactory):\n \"\"\"A critic factory which reuses the actor's preprocessing component.\n\n This class is for internal use in experiment builders only.\n\n Reuse of the actor network is supported through the concept of an actor future (:class:`ActorFuture`).\n When the user declares that he wants to reuse the actor for the critic, we use this factory to support this,\n but the actor does not exist yet. So the factory instead receives the future, which will eventually be filled\n when the actor factory is called. When the creation method of this factory is eventually called, it can use the\n then-filled actor to create the critic.\n \"\"\"\n\n def __init__(self, actor_future: ActorFuture):\n \"\"\":param actor_future: the object, which will hold the actor instance later when the critic is to be created\"\"\"\n self.actor_future = actor_future\n\n def _tostring_excludes(self) -> list[str]:\n return [\"actor_future\"]\n\n def create_module(\n self,\n envs: Environments,\n device: TDevice,\n use_action: bool,\n discrete_last_size_use_action_shape: bool = False,\n ) -> nn.Module:\n actor = self.actor_future.actor\n if not isinstance(actor, Actor):\n raise ValueError(\n f\"Option critic_use_action can only be used if actor is of type {Actor.__class__.__name__}\",\n )\n if envs.get_type().is_discrete():\n # TODO get rid of this prod pattern here and elsewhere\n last_size = (\n int(np.prod(envs.get_action_shape())) if discrete_last_size_use_action_shape else 1\n )\n return DiscreteCritic(\n preprocess_net=actor.get_preprocess_net(),\n last_size=last_size,\n ).to(device)\n elif envs.get_type().is_continuous():\n return ContinuousCritic(\n preprocess_net=actor.get_preprocess_net(),\n apply_preprocess_net_to_obs_only=True,\n ).to(device)\n else:\n raise ValueError\n\n\nclass CriticEnsembleFactory:\n @abstractmethod\n def create_module(\n self,\n envs: Environments,\n device: TDevice,\n ensemble_size: int,\n use_action: bool,\n ) -> nn.Module:\n pass\n\n\nclass CriticEnsembleFactoryDefault(CriticEnsembleFactory):\n \"\"\"A critic ensemble factory which, depending on the type of environment, creates a suitable MLP-based critic.\"\"\"\n\n DEFAULT_HIDDEN_SIZES = (64, 64)\n\n def __init__(self, hidden_sizes: Sequence[int] = DEFAULT_HIDDEN_SIZES):\n self.hidden_sizes = hidden_sizes\n\n def create_module(\n self,\n envs: Environments,\n device: TDevice,\n ensemble_size: int,\n use_action: bool,\n ) -> nn.Module:\n env_type = envs.get_type()\n factory: CriticEnsembleFactory\n match env_type:\n case EnvType.CONTINUOUS:\n factory = CriticEnsembleFactoryContinuousNet(self.hidden_sizes)\n case EnvType.DISCRETE:\n raise NotImplementedError(\"No default is implemented for the discrete case\")\n case _:\n raise ValueError(f\"{env_type} not supported\")\n return factory.create_module(\n envs,\n device,\n ensemble_size,\n use_action,\n )\n\n\nclass CriticEnsembleFactoryContinuousNet(CriticEnsembleFactory):\n def __init__(self, hidden_sizes: Sequence[int]):\n self.hidden_sizes = hidden_sizes\n\n def create_module(\n self,\n envs: Environments,\n device: TDevice,\n ensemble_size: int,\n use_action: bool,\n ) -> nn.Module:\n def linear_layer(x: int, y: int) -> EnsembleLinear:\n return EnsembleLinear(ensemble_size, x, y)\n\n action_shape = envs.get_action_shape() if use_action else 0\n net_c = Net(\n state_shape=envs.get_observation_shape(),\n action_shape=action_shape,\n hidden_sizes=self.hidden_sizes,\n concat=use_action,\n activation=nn.Tanh,\n linear_layer=linear_layer,\n )\n critic = continuous.ContinuousCritic(\n preprocess_net=net_c,\n linear_layer=linear_layer,\n flatten_input=False,\n ).to(device)\n init_linear_orthogonal(critic)\n return critic\n" }, { "path": "tianshou/highlevel/module/intermediate.py", "content": "from abc import ABC, abstractmethod\nfrom dataclasses import dataclass\n\nimport torch\nfrom sensai.util.string import ToStringMixin\n\nfrom tianshou.highlevel.env import Environments\nfrom tianshou.highlevel.module.core import ModuleFactory, TDevice\nfrom tianshou.utils.net.common import ModuleWithVectorOutput\n\n\n@dataclass\nclass IntermediateModule:\n \"\"\"Container for a module which computes an intermediate representation (with a known dimension).\"\"\"\n\n module: torch.nn.Module\n output_dim: int\n\n def get_module_with_vector_output(self) -> ModuleWithVectorOutput:\n if isinstance(self.module, ModuleWithVectorOutput):\n return self.module\n else:\n return ModuleWithVectorOutput.from_module(self.module, self.output_dim)\n\n\nclass IntermediateModuleFactory(ToStringMixin, ModuleFactory, ABC):\n \"\"\"Factory for the generation of a module which computes an intermediate representation.\"\"\"\n\n @abstractmethod\n def create_intermediate_module(self, envs: Environments, device: TDevice) -> IntermediateModule:\n pass\n\n def create_module(self, envs: Environments, device: TDevice) -> torch.nn.Module:\n return self.create_intermediate_module(envs, device).module\n" }, { "path": "tianshou/highlevel/module/special.py", "content": "from collections.abc import Sequence\n\nfrom sensai.util.string import ToStringMixin\n\nfrom tianshou.highlevel.env import Environments\nfrom tianshou.highlevel.module.core import ModuleFactory, TDevice\nfrom tianshou.highlevel.module.intermediate import IntermediateModuleFactory\nfrom tianshou.utils.net.discrete import ImplicitQuantileNetwork\n\n\nclass ImplicitQuantileNetworkFactory(ModuleFactory, ToStringMixin):\n def __init__(\n self,\n preprocess_net_factory: IntermediateModuleFactory,\n hidden_sizes: Sequence[int] = (),\n num_cosines: int = 64,\n ):\n self.preprocess_net_factory = preprocess_net_factory\n self.hidden_sizes = hidden_sizes\n self.num_cosines = num_cosines\n\n def create_module(self, envs: Environments, device: TDevice) -> ImplicitQuantileNetwork:\n preprocess_net = self.preprocess_net_factory.create_intermediate_module(envs, device)\n return ImplicitQuantileNetwork(\n preprocess_net=preprocess_net.get_module_with_vector_output(),\n action_shape=envs.get_action_shape(),\n hidden_sizes=self.hidden_sizes,\n num_cosines=self.num_cosines,\n ).to(device)\n" }, { "path": "tianshou/highlevel/params/__init__.py", "content": "" }, { "path": "tianshou/highlevel/params/algorithm_params.py", "content": "from abc import ABC, abstractmethod\nfrom collections.abc import Callable, Sequence\nfrom dataclasses import asdict, dataclass\nfrom typing import Any, Literal, Protocol\n\nfrom sensai.util.string import ToStringMixin\n\nfrom tianshou.exploration import BaseNoise\nfrom tianshou.highlevel.env import Environments\nfrom tianshou.highlevel.module.core import TDevice\nfrom tianshou.highlevel.params.alpha import AutoAlphaFactory\nfrom tianshou.highlevel.params.env_param import EnvValueFactory, FloatEnvValueFactory\nfrom tianshou.highlevel.params.lr_scheduler import LRSchedulerFactoryFactory\nfrom tianshou.highlevel.params.noise import NoiseFactory\nfrom tianshou.highlevel.params.optim import OptimizerFactoryFactory\n\n\n@dataclass(kw_only=True)\nclass ParamTransformerData:\n \"\"\"Holds data that can be used by `ParamTransformer` instances to perform their transformation.\n\n The representation contains the superset of all data items that are required by different types of agent factories.\n An agent factory is expected to set only the attributes that are relevant to its parameters.\n \"\"\"\n\n envs: Environments\n device: TDevice\n optim_factory_default: OptimizerFactoryFactory\n\n\nclass ParamTransformer(ABC):\n \"\"\"Base class for parameter transformations from high to low-level API.\n\n Transforms one or more parameters from the representation used by the high-level API\n to the representation required by the (low-level) policy implementation.\n It operates directly on a dictionary of keyword arguments, which is initially\n generated from the parameter dataclass (subclass of `Params`).\n \"\"\"\n\n @abstractmethod\n def transform(self, params: dict[str, Any], data: ParamTransformerData) -> None:\n pass\n\n @staticmethod\n def get(\n d: dict[str, Any],\n key: str,\n drop: bool = False,\n default_factory: Callable[[], Any] | None = None,\n ) -> Any:\n try:\n value = d[key]\n except KeyError as e:\n raise Exception(f\"Key not found: '{key}'; available keys: {list(d.keys())}\") from e\n if value is None and default_factory is not None:\n value = default_factory()\n if drop:\n del d[key]\n return value\n\n\nclass ParamTransformerDrop(ParamTransformer):\n def __init__(self, *keys: str):\n self.keys = keys\n\n def transform(self, kwargs: dict[str, Any], data: ParamTransformerData) -> None:\n for k in self.keys:\n del kwargs[k]\n\n\nclass ParamTransformerRename(ParamTransformer):\n def __init__(self, renamed_params: dict[str, str]):\n self.renamed_params = renamed_params\n\n def transform(self, kwargs: dict[str, Any], data: ParamTransformerData) -> None:\n for old_name, new_name in self.renamed_params.items():\n v = kwargs[old_name]\n del kwargs[old_name]\n kwargs[new_name] = v\n\n\nclass ParamTransformerChangeValue(ParamTransformer):\n def __init__(self, key: str):\n self.key = key\n\n def transform(self, params: dict[str, Any], data: ParamTransformerData) -> None:\n params[self.key] = self.change_value(params[self.key], data)\n\n @abstractmethod\n def change_value(self, value: Any, data: ParamTransformerData) -> Any:\n pass\n\n\nclass ParamTransformerOptimFactory(ParamTransformer):\n \"\"\"Transformer for learning rate scheduler params.\n\n Transforms a key containing a learning rate scheduler factory (removed) into a key containing\n a learning rate scheduler (added) for the data member `optim`.\n \"\"\"\n\n def __init__(\n self,\n key_optim_factory_factory: str,\n key_lr: str,\n key_lr_scheduler_factory_factory: str,\n key_optim_output: str,\n ):\n self.key_optim_factory_factory = key_optim_factory_factory\n self.key_lr = key_lr\n self.key_scheduler_factory = key_lr_scheduler_factory_factory\n self.key_optim_output = key_optim_output\n\n def transform(self, params: dict[str, Any], data: ParamTransformerData) -> None:\n optim_factory_factory: OptimizerFactoryFactory = self.get(\n params,\n self.key_optim_factory_factory,\n drop=True,\n default_factory=lambda: data.optim_factory_default,\n )\n lr_scheduler_factory_factory: LRSchedulerFactoryFactory | None = self.get(\n params, self.key_scheduler_factory, drop=True\n )\n lr: float = self.get(params, self.key_lr, drop=True)\n optim_factory = optim_factory_factory.create_optimizer_factory(lr)\n if lr_scheduler_factory_factory is not None:\n optim_factory.with_lr_scheduler_factory(\n lr_scheduler_factory_factory.create_lr_scheduler_factory()\n )\n params[self.key_optim_output] = optim_factory\n\n\nclass ParamTransformerAutoAlpha(ParamTransformer):\n def __init__(self, key: str):\n self.key = key\n\n def transform(self, kwargs: dict[str, Any], data: ParamTransformerData) -> None:\n alpha = self.get(kwargs, self.key)\n if isinstance(alpha, AutoAlphaFactory):\n kwargs[self.key] = alpha.create_auto_alpha(data.envs, data.device)\n\n\nclass ParamTransformerNoiseFactory(ParamTransformerChangeValue):\n def change_value(self, value: Any, data: ParamTransformerData) -> Any:\n if isinstance(value, NoiseFactory):\n value = value.create_noise(data.envs)\n return value\n\n\nclass ParamTransformerFloatEnvParamFactory(ParamTransformerChangeValue):\n def change_value(self, value: Any, data: ParamTransformerData) -> Any:\n if isinstance(value, EnvValueFactory):\n value = value.create_value(data.envs)\n return value\n\n\nclass ParamTransformerActionScaling(ParamTransformerChangeValue):\n def change_value(self, value: Any, data: ParamTransformerData) -> Any:\n if value == \"default\":\n return data.envs.get_type().is_continuous()\n else:\n return value\n\n\nclass GetParamTransformersProtocol(Protocol):\n def _get_param_transformers(self) -> list[ParamTransformer]:\n pass\n\n\n@dataclass(kw_only=True)\nclass Params(GetParamTransformersProtocol, ToStringMixin):\n def create_kwargs(self, data: ParamTransformerData) -> dict[str, Any]:\n params = asdict(self)\n for transformer in self._get_param_transformers():\n transformer.transform(params, data)\n return params\n\n def _get_param_transformers(self) -> list[ParamTransformer]:\n return []\n\n\n@dataclass(kw_only=True)\nclass ParamsMixinSingleModel(GetParamTransformersProtocol):\n optim: OptimizerFactoryFactory | None = None\n \"\"\"the factory for the creation of the model's optimizer; if None, use default\"\"\"\n lr: float = 1e-3\n \"\"\"the learning rate to use in the gradient-based optimizer\"\"\"\n lr_scheduler: LRSchedulerFactoryFactory | None = None\n \"\"\"factory for the creation of a learning rate scheduler\"\"\"\n\n def _get_param_transformers(self) -> list[ParamTransformer]:\n return [\n ParamTransformerOptimFactory(\"optim\", \"lr\", \"lr_scheduler\", \"optim\"),\n ]\n\n\n@dataclass(kw_only=True)\nclass ParamsMixinActorAndCritic(GetParamTransformersProtocol):\n actor_optim: OptimizerFactoryFactory | None = None\n \"\"\"the factory for the creation of the actor's optimizer; if None, use default\"\"\"\n critic_optim: OptimizerFactoryFactory | None = None\n \"\"\"the factory for the creation of the critic's optimizer; if None, use default\"\"\"\n actor_lr: float = 1e-3\n \"\"\"the learning rate to use for the actor network\"\"\"\n critic_lr: float = 1e-3\n \"\"\"the learning rate to use for the critic network\"\"\"\n actor_lr_scheduler: LRSchedulerFactoryFactory | None = None\n \"\"\"factory for the creation of a learning rate scheduler to use for the actor network (if any)\"\"\"\n critic_lr_scheduler: LRSchedulerFactoryFactory | None = None\n \"\"\"factory for the creation of a learning rate scheduler to use for the critic network (if any)\"\"\"\n\n def _get_param_transformers(self) -> list[ParamTransformer]:\n return [\n ParamTransformerOptimFactory(\n \"actor_optim\", \"actor_lr\", \"actor_lr_scheduler\", \"policy_optim\"\n ),\n ParamTransformerOptimFactory(\n \"critic_optim\", \"critic_lr\", \"critic_lr_scheduler\", \"critic_optim\"\n ),\n ]\n\n\n@dataclass(kw_only=True)\nclass ParamsMixinActionScaling(GetParamTransformersProtocol):\n action_scaling: bool | Literal[\"default\"] = \"default\"\n \"\"\"\n flag indicating whether, for continuous action spaces, actions\n should be scaled from the standard neural network output range [-1, 1] to the\n environment's action space range [action_space.low, action_space.high].\n This applies to continuous action spaces only (gym.spaces.Box) and has no effect\n for discrete spaces.\n When enabled, policy outputs are expected to be in the normalized range [-1, 1]\n (after bounding), and are then linearly transformed to the actual required range.\n This improves neural network training stability, allows the same algorithm to work\n across environments with different action ranges, and standardizes exploration\n strategies.\n Should be disabled if the actor model already produces outputs in the correct range.\n \"\"\"\n\n def _get_param_transformers(self) -> list[ParamTransformer]:\n return [ParamTransformerActionScaling(\"action_scaling\")]\n\n\n@dataclass(kw_only=True)\nclass ParamsMixinActionScalingAndBounding(ParamsMixinActionScaling):\n action_bound_method: Literal[\"clip\", \"tanh\"] | None = \"clip\"\n \"\"\"\n the method used for bounding actions in continuous action spaces\n to the range [-1, 1] before scaling them to the environment's action space (provided\n that `action_scaling` is enabled).\n This applies to continuous action spaces only (`gym.spaces.Box`) and should be set to None\n for discrete spaces.\n When set to \"clip\", actions exceeding the [-1, 1] range are simply clipped to this\n range. When set to \"tanh\", a hyperbolic tangent function is applied, which smoothly\n constrains outputs to [-1, 1] while preserving gradients.\n The choice of bounding method affects both training dynamics and exploration behavior.\n Clipping provides hard boundaries but may create plateau regions in the gradient\n landscape, while tanh provides smoother transitions but can compress sensitivity\n near the boundaries.\n Should be set to None if the actor model inherently produces bounded outputs.\n Typically used together with `action_scaling=True`.\n \"\"\"\n\n\n@dataclass(kw_only=True)\nclass ParamsMixinExplorationNoise(GetParamTransformersProtocol):\n exploration_noise: BaseNoise | Literal[\"default\"] | NoiseFactory | None = None\n \"\"\"\n If not None, add noise to actions for exploration.\n This is useful when solving \"hard exploration\" problems.\n It can either be a distribution, a factory for the creation of a distribution or \"default\".\n When set to \"default\", use Gaussian noise with standard deviation 0.1.\n \"\"\"\n\n def _get_param_transformers(self) -> list[ParamTransformer]:\n return [ParamTransformerNoiseFactory(\"exploration_noise\")]\n\n\n@dataclass(kw_only=True)\nclass ParamsMixinNStepReturnHorizon:\n n_step_return_horizon: int = 1\n \"\"\"\n the number of future steps (> 0) to consider when computing temporal difference (TD) targets.\n Controls the balance between TD learning and Monte Carlo methods:\n Higher values reduce bias (by relying less on potentially inaccurate value estimates)\n but increase variance (by incorporating more environmental stochasticity and reducing\n the averaging effect).\n A value of 1 corresponds to standard TD learning with immediate bootstrapping, while very\n large values approach Monte Carlo-like estimation that uses complete episode returns.\n \"\"\"\n\n\n@dataclass(kw_only=True)\nclass ParamsMixinGamma:\n gamma: float = 0.99\n \"\"\"\n the discount factor in [0, 1] for future rewards.\n This determines how much future rewards are valued compared to immediate ones.\n Lower values (closer to 0) make the agent focus on immediate rewards, creating \"myopic\"\n behavior. Higher values (closer to 1) make the agent value long-term rewards more,\n potentially improving performance in tasks where delayed rewards are important but\n increasing training variance by incorporating more environmental stochasticity.\n Typically set between 0.9 and 0.99 for most reinforcement learning tasks\n \"\"\"\n\n\n@dataclass(kw_only=True)\nclass ParamsMixinTau:\n tau: float = 0.005\n \"\"\"\n the soft update coefficient for target networks, controlling the rate at which\n target networks track the learned networks.\n When the parameters of the target network are updated with the current (source) network's\n parameters, a weighted average is used: target = tau * source + (1 - tau) * target.\n Smaller values (closer to 0) create more stable but slower learning as target networks\n change more gradually. Higher values (closer to 1) allow faster learning but may reduce\n stability.\n Typically set to a small value (0.001 to 0.01) for most reinforcement learning tasks.\n \"\"\"\n\n\n@dataclass(kw_only=True)\nclass ParamsMixinDeterministicEval:\n deterministic_eval: bool = False\n \"\"\"\n flag indicating whether the policy should use deterministic\n actions (using the mode of the action distribution) instead of stochastic ones\n (using random sampling) during evaluation.\n When enabled, the policy will always select the most probable action according to\n the learned distribution during evaluation phases, while still using stochastic\n sampling during training. This creates a clear distinction between exploration\n (training) and exploitation (evaluation) behaviors.\n Deterministic actions are generally preferred for final deployment and reproducible\n evaluation as they provide consistent behavior, reduce variance in performance\n metrics, and are more interpretable for human observers.\n Note that this parameter only affects behavior when the policy is not within a\n training step. When collecting rollouts for training, actions remain stochastic\n regardless of this setting to maintain proper exploration behaviour.\n \"\"\"\n\n\nclass OnPolicyAlgorithmParams(\n Params,\n ParamsMixinGamma,\n ParamsMixinActionScalingAndBounding,\n ParamsMixinSingleModel,\n ParamsMixinDeterministicEval,\n):\n def _get_param_transformers(self) -> list[ParamTransformer]:\n transformers = super()._get_param_transformers()\n transformers.extend(ParamsMixinActionScalingAndBounding._get_param_transformers(self))\n transformers.extend(ParamsMixinSingleModel._get_param_transformers(self))\n return transformers\n\n\n@dataclass(kw_only=True)\nclass ReinforceParams(OnPolicyAlgorithmParams):\n return_standardization: bool = False\n \"\"\"\n whether to standardize episode returns by subtracting the running mean and\n dividing by the running standard deviation.\n Note that this is known to be detrimental to performance in many cases!\n \"\"\"\n\n\n@dataclass(kw_only=True)\nclass ParamsMixinGeneralAdvantageEstimation(GetParamTransformersProtocol):\n gae_lambda: float = 0.95\n \"\"\"\n the lambda parameter in [0, 1] for generalized advantage estimation (GAE).\n Controls the bias-variance tradeoff in advantage estimates, acting as a\n weighting factor for combining different n-step advantage estimators. Higher values\n (closer to 1) reduce bias but increase variance by giving more weight to longer\n trajectories, while lower values (closer to 0) reduce variance but increase bias\n by relying more on the immediate TD error and value function estimates. At λ=0,\n GAE becomes equivalent to the one-step TD error (high bias, low variance); at λ=1,\n it becomes equivalent to Monte Carlo advantage estimation (low bias, high variance).\n Intermediate values create a weighted average of n-step returns, with exponentially\n decaying weights for longer-horizon returns. Typically set between 0.9 and 0.99 for\n most policy gradient methods.\n \"\"\"\n max_batchsize: int = 256\n \"\"\"the maximum number of samples to process at once when computing\n generalized advantage estimation (GAE) and value function predictions.\n Controls memory usage by breaking large batches into smaller chunks processed sequentially.\n Higher values may increase speed but require more GPU/CPU memory; lower values\n reduce memory requirements but may increase computation time. Should be adjusted\n based on available hardware resources and total batch size of your training data.\"\"\"\n\n def _get_param_transformers(self) -> list[ParamTransformer]:\n return []\n\n\n@dataclass(kw_only=True)\nclass ActorCriticOnPolicyParams(OnPolicyAlgorithmParams):\n return_scaling: bool = False\n \"\"\"\n flag indicating whether to enable scaling of estimated returns by\n dividing them by their running standard deviation without centering the mean.\n This reduces the magnitude variation of advantages across different episodes while\n preserving their signs and relative ordering.\n The use of running statistics (rather than batch-specific scaling) means that early\n training experiences may be scaled differently than later ones as the statistics evolve.\n When enabled, this improves training stability in environments with highly variable\n reward scales and makes the algorithm less sensitive to learning rate settings.\n However, it may reduce the algorithm's ability to distinguish between episodes with\n different absolute return magnitudes.\n Best used in environments where the relative ordering of actions is more important\n than the absolute scale of returns.\n \"\"\"\n\n\n@dataclass(kw_only=True)\nclass A2CParams(ActorCriticOnPolicyParams, ParamsMixinGeneralAdvantageEstimation):\n vf_coef: float = 0.5\n \"\"\"\n coefficient that weights the value loss relative to the actor loss in the overall\n loss function.\n Higher values prioritize accurate value function estimation over policy improvement.\n Controls the trade-off between policy optimization and value function fitting.\n Typically set between 0.5 and 1.0 for most actor-critic implementations.\n \"\"\"\n ent_coef: float = 0.01\n \"\"\"\n coefficient that weights the entropy bonus relative to the actor loss.\n Controls the exploration-exploitation trade-off by encouraging policy entropy.\n Higher values promote more exploration by encouraging a more uniform action distribution.\n Lower values focus more on exploitation of the current policy's knowledge.\n Typically set between 0.01 and 0.05 for most actor-critic implementations.\n \"\"\"\n max_grad_norm: float | None = None\n \"\"\"\n the maximum L2 norm threshold for gradient clipping.\n When not None, gradients will be rescaled using to ensure their L2 norm does not\n exceed this value. This prevents exploding gradients and stabilizes training by\n limiting the magnitude of parameter updates.\n Set to None to disable gradient clipping.\n \"\"\"\n\n def _get_param_transformers(self) -> list[ParamTransformer]:\n transformers = super()._get_param_transformers()\n transformers.extend(ParamsMixinGeneralAdvantageEstimation._get_param_transformers(self))\n return transformers\n\n\n@dataclass(kw_only=True)\nclass PPOParams(A2CParams):\n eps_clip: float = 0.2\n \"\"\"\n determines the range of allowed change in the policy during a policy update:\n The ratio of action probabilities indicated by the new and old policy is\n constrained to stay in the interval [1 - eps_clip, 1 + eps_clip].\n Small values thus force the new policy to stay close to the old policy.\n Typical values range between 0.1 and 0.3, the value of 0.2 is recommended\n in the original PPO paper.\n The optimal value depends on the environment; more stochastic environments may\n need larger values.\n \"\"\"\n dual_clip: float | None = None\n \"\"\"\n a clipping parameter (denoted as c in the literature) that prevents\n excessive pessimism in policy updates for negative-advantage actions.\n Excessive pessimism occurs when the policy update too strongly reduces the probability\n of selecting actions that led to negative advantages, potentially eliminating useful\n actions based on limited negative experiences.\n When enabled (c > 1), the objective for negative advantages becomes:\n max(min(r(θ)A, clip(r(θ), 1-ε, 1+ε)A), c*A), where min(r(θ)A, clip(r(θ), 1-ε, 1+ε)A)\n is the original single-clipping objective determined by `eps_clip`.\n This creates a floor on negative policy gradients, maintaining some probability\n of exploring actions despite initial negative outcomes.\n Larger values (e.g., 2.0 to 5.0) maintain more exploration, while values closer\n to 1.0 provide less protection against pessimistic updates.\n Set to None to disable dual clipping.\n \"\"\"\n value_clip: bool = False\n \"\"\"\n flag indicating whether to enable clipping for value function updates.\n When enabled, restricts how much the value function estimate can change from its\n previous prediction, using the same clipping range as the policy updates (eps_clip).\n This stabilizes training by preventing large fluctuations in value estimates,\n particularly useful in environments with high reward variance.\n The clipped value loss uses a pessimistic approach, taking the maximum of the\n original and clipped value errors:\n max((returns - value)², (returns - v_clipped)²)\n Setting to True often improves training stability but may slow convergence.\n Implementation follows the approach mentioned in arXiv:1811.02553v3 Sec. 4.1.\n \"\"\"\n advantage_normalization: bool = True\n \"\"\"whether to apply per mini-batch advantage normalization.\"\"\"\n recompute_advantage: bool = False\n \"\"\"\n whether to recompute advantage every update repeat as described in\n https://arxiv.org/pdf/2006.05990.pdf, Sec. 3.5.\n The original PPO implementation splits the data in each policy iteration\n step into individual transitions and then randomly assigns them to minibatches.\n This makes it impossible to compute advantages as the temporal structure is broken.\n Therefore, the advantages are computed once at the beginning of each policy iteration step and\n then used in minibatch policy and value function optimization.\n This results in higher diversity of data in each minibatch at the cost of\n using slightly stale advantage estimations.\n Enabling this option will, as a remedy to this problem, recompute the advantages at the beginning\n of each pass over the data instead of just once per iteration.\n \"\"\"\n\n\n@dataclass(kw_only=True)\nclass NPGParams(ActorCriticOnPolicyParams, ParamsMixinGeneralAdvantageEstimation):\n optim_critic_iters: int = 5\n \"\"\"\n the number of optimization steps performed on the critic network for each policy (actor) update.\n Controls the learning rate balance between critic and actor.\n Higher values prioritize critic accuracy by training the value function more\n extensively before each policy update, which can improve stability but slow down\n training. Lower values maintain a more even learning pace between policy and value\n function but may lead to less reliable advantage estimates.\n Typically set between 1 and 10, depending on the complexity of the value function.\n \"\"\"\n trust_region_size: float = 0.5\n \"\"\"\n the parameter delta - a scalar multiplier for policy updates in the natural gradient direction.\n The mathematical meaning is the trust region size, which is the maximum KL divergence\n allowed between the old and new policy distributions.\n Controls how far the policy parameters move in the calculated direction\n during each update. Higher values allow for faster learning but may cause instability\n or policy deterioration; lower values provide more stable but slower learning. Unlike\n regular policy gradients, natural gradients already account for the local geometry of\n the parameter space, making this step size more robust to different parameterizations.\n Typically set between 0.1 and 1.0 for most reinforcement learning tasks.\n \"\"\"\n advantage_normalization: bool = True\n \"\"\"whether to do per mini-batch advantage normalization.\"\"\"\n\n def _get_param_transformers(self) -> list[ParamTransformer]:\n transformers = super()._get_param_transformers()\n transformers.extend(ParamsMixinGeneralAdvantageEstimation._get_param_transformers(self))\n return transformers\n\n\n@dataclass(kw_only=True)\nclass TRPOParams(NPGParams):\n max_kl: float = 0.01\n \"\"\"\n maximum KL divergence, used to constrain each actor network update.\n \"\"\"\n backtrack_coeff: float = 0.8\n \"\"\"\n coefficient with which to reduce the step size when constraints are not met.\n \"\"\"\n max_backtracks: int = 10\n \"\"\"maximum number of times to backtrack in line search when the constraints are not met.\"\"\"\n\n\n@dataclass(kw_only=True)\nclass ParamsMixinActorAndDualCritics(GetParamTransformersProtocol):\n actor_optim: OptimizerFactoryFactory | None = None\n \"\"\"the factory for the creation of the actor's optimizer; if None, use default\"\"\"\n critic1_optim: OptimizerFactoryFactory | None = None\n \"\"\"the factory for the creation of the first critic's optimizer; if None, use default\"\"\"\n critic2_optim: OptimizerFactoryFactory | None = None\n \"\"\"the factory for the creation of the second critic's optimizer; if None, use default\"\"\"\n actor_lr: float = 1e-3\n \"\"\"the learning rate to use for the actor network\"\"\"\n critic1_lr: float = 1e-3\n \"\"\"the learning rate to use for the first critic network\"\"\"\n critic2_lr: float = 1e-3\n \"\"\"the learning rate to use for the second critic network\"\"\"\n actor_lr_scheduler: LRSchedulerFactoryFactory | None = None\n \"\"\"factory for the creation of a learning rate scheduler to use for the actor network (if any)\"\"\"\n critic1_lr_scheduler: LRSchedulerFactoryFactory | None = None\n \"\"\"factory for the creation of a learning rate scheduler to use for the first critic network (if any)\"\"\"\n critic2_lr_scheduler: LRSchedulerFactoryFactory | None = None\n \"\"\"factory for the creation of a learning rate scheduler to use for the second critic network (if any)\"\"\"\n\n def _get_param_transformers(self) -> list[ParamTransformer]:\n return [\n ParamTransformerOptimFactory(\n \"actor_optim\", \"actor_lr\", \"actor_lr_scheduler\", \"policy_optim\"\n ),\n ParamTransformerOptimFactory(\n \"critic1_optim\", \"critic1_lr\", \"critic1_lr_scheduler\", \"critic_optim\"\n ),\n ParamTransformerOptimFactory(\n \"critic2_optim\", \"critic2_lr\", \"critic2_lr_scheduler\", \"critic2_optim\"\n ),\n ]\n\n\n@dataclass(kw_only=True)\nclass ParamsMixinAlpha(GetParamTransformersProtocol):\n alpha: float | AutoAlphaFactory = 0.2\n \"\"\"\n the entropy regularization coefficient, which balances exploration and exploitation.\n This coefficient controls how much the agent values randomness in its policy versus\n pursuing higher rewards.\n Higher values (e.g., 0.5-1.0) strongly encourage exploration by rewarding the agent\n for maintaining diverse action choices, even if this means selecting some lower-value actions.\n Lower values (e.g., 0.01-0.1) prioritize exploitation, allowing the policy to become\n more focused on the highest-value actions.\n A value of 0 would completely remove entropy regularization, potentially leading to\n premature convergence to suboptimal deterministic policies.\n Can be provided as a fixed float (0.2 is a reasonable default) or via a factory\n to support automatic tuning during training.\n \"\"\"\n\n def _get_param_transformers(self) -> list[ParamTransformer]:\n return [ParamTransformerAutoAlpha(\"alpha\")]\n\n\n@dataclass(kw_only=True)\nclass _SACParams(\n Params,\n ParamsMixinGamma,\n ParamsMixinActorAndDualCritics,\n ParamsMixinNStepReturnHorizon,\n ParamsMixinTau,\n ParamsMixinDeterministicEval,\n ParamsMixinAlpha,\n):\n def _get_param_transformers(self) -> list[ParamTransformer]:\n transformers = super()._get_param_transformers()\n transformers.extend(ParamsMixinActorAndDualCritics._get_param_transformers(self))\n transformers.extend(ParamsMixinAlpha._get_param_transformers(self))\n return transformers\n\n\n@dataclass(kw_only=True)\nclass SACParams(_SACParams, ParamsMixinExplorationNoise, ParamsMixinActionScaling):\n def _get_param_transformers(self) -> list[ParamTransformer]:\n transformers = super()._get_param_transformers()\n transformers.extend(ParamsMixinExplorationNoise._get_param_transformers(self))\n transformers.extend(ParamsMixinActionScalingAndBounding._get_param_transformers(self))\n return transformers\n\n\n@dataclass(kw_only=True)\nclass DiscreteSACParams(_SACParams):\n pass\n\n\n@dataclass(kw_only=True)\nclass QLearningOffPolicyParams(\n Params, ParamsMixinGamma, ParamsMixinSingleModel, ParamsMixinNStepReturnHorizon\n):\n target_update_freq: int = 0\n \"\"\"\n the number of training iterations between each complete update of the target network.\n Controls how frequently the target Q-network parameters are updated with the current\n Q-network values.\n A value of 0 disables the target network entirely, using only a single network for both\n action selection and bootstrap targets.\n Higher values provide more stable learning targets but slow down the propagation of new\n value estimates. Lower positive values allow faster learning but may lead to instability\n due to rapidly changing targets.\n Typically set between 100-10000 for DQN variants, with exact values depending on\n environment complexity.\n \"\"\"\n eps_training: float = 0.0\n \"\"\"\n the epsilon value for epsilon-greedy exploration during training.\n When collecting data for training, this is the probability of choosing a random action\n instead of the action chosen by the policy.\n A value of 0.0 means no exploration (fully greedy) and a value of 1.0 means full\n exploration (fully random).\n \"\"\"\n eps_inference: float = 0.0\n \"\"\"\n the epsilon value for epsilon-greedy exploration during inference,\n i.e. non-training cases (such as evaluation during test steps).\n The epsilon value is the probability of choosing a random action instead of the action\n chosen by the policy.\n A value of 0.0 means no exploration (fully greedy) and a value of 1.0 means full\n exploration (fully random).\n \"\"\"\n\n def _get_param_transformers(self) -> list[ParamTransformer]:\n transformers = super()._get_param_transformers()\n transformers.extend(ParamsMixinSingleModel._get_param_transformers(self))\n return transformers\n\n\n@dataclass(kw_only=True)\nclass DQNParams(QLearningOffPolicyParams):\n is_double: bool = True\n \"\"\"\n flag indicating whether to use the Double DQN algorithm for target value computation.\n If True, the algorithm uses the online network to select actions and the target network to\n evaluate their Q-values. This approach helps reduce the overestimation bias in Q-learning\n by decoupling action selection from action evaluation.\n If False, the algorithm follows the vanilla DQN method that directly takes the maximum Q-value\n from the target network.\n Note: Double Q-learning will only be effective when a target network is used (target_update_freq > 0).\n \"\"\"\n huber_loss_delta: float | None = None\n \"\"\"\n controls whether to use the Huber loss instead of the MSE loss for the TD error and the threshold for\n the Huber loss.\n If None, the MSE loss is used.\n If not None, uses the Huber loss as described in the Nature DQN paper (nature14236) with the given delta,\n which limits the influence of outliers.\n Unlike the MSE loss where the gradients grow linearly with the error magnitude, the Huber\n loss causes the gradients to plateau at a constant value for large errors, providing more stable training.\n NOTE: The magnitude of delta should depend on the scale of the returns obtained in the environment.\n \"\"\"\n\n def _get_param_transformers(self) -> list[ParamTransformer]:\n return super()._get_param_transformers()\n\n\n@dataclass(kw_only=True)\nclass IQNParams(QLearningOffPolicyParams):\n sample_size: int = 32\n \"\"\"the number of samples for policy evaluation\"\"\"\n online_sample_size: int = 8\n \"\"\"the number of samples for online model in training\"\"\"\n target_sample_size: int = 8\n \"\"\"the number of samples for target model in training.\"\"\"\n num_quantiles: int = 200\n \"\"\"the number of quantile midpoints in the inverse cumulative distribution function of the value\"\"\"\n hidden_sizes: Sequence[int] = ()\n \"\"\"hidden dimensions to use in the IQN network\"\"\"\n num_cosines: int = 64\n \"\"\"number of cosines to use in the IQN network\"\"\"\n\n def _get_param_transformers(self) -> list[ParamTransformer]:\n transformers = super()._get_param_transformers()\n transformers.append(ParamTransformerDrop(\"hidden_sizes\", \"num_cosines\"))\n return transformers\n\n\n@dataclass(kw_only=True)\nclass DDPGParams(\n Params,\n ParamsMixinGamma,\n ParamsMixinActorAndCritic,\n ParamsMixinExplorationNoise,\n ParamsMixinActionScalingAndBounding,\n ParamsMixinNStepReturnHorizon,\n ParamsMixinTau,\n):\n def _get_param_transformers(self) -> list[ParamTransformer]:\n transformers = super()._get_param_transformers()\n transformers.extend(ParamsMixinActorAndCritic._get_param_transformers(self))\n transformers.extend(ParamsMixinExplorationNoise._get_param_transformers(self))\n transformers.extend(ParamsMixinActionScalingAndBounding._get_param_transformers(self))\n return transformers\n\n\n@dataclass(kw_only=True)\nclass REDQParams(DDPGParams, ParamsMixinDeterministicEval, ParamsMixinAlpha):\n ensemble_size: int = 10\n \"\"\"\n the total number of critic networks in the ensemble.\n This parameter implements the randomized ensemble approach described in REDQ.\n The algorithm maintains `ensemble_size` different critic networks that all share the same architecture.\n During target value computation, a random subset of these networks (determined by `subset_size`) is used.\n Larger values increase the diversity of the ensemble but require more memory and computation.\n The original paper recommends a value of 10 for most tasks, balancing performance and computational efficiency.\n \"\"\"\n subset_size: int = 2\n \"\"\"\n the number of critic networks randomly selected from the ensemble for computing target Q-values.\n During each update, the algorithm samples `subset_size` networks from the ensemble of\n `ensemble_size` networks without replacement.\n The target Q-value is then calculated as either the minimum or mean (based on target_mode)\n of the predictions from this subset.\n Smaller values increase randomization and sample efficiency but may introduce more variance.\n Larger values provide more stable estimates but reduce the benefits of randomization.\n The REDQ paper recommends a value of 2 for optimal sample efficiency.\n Must satisfy 0 < subset_size <= ensemble_size.\n \"\"\"\n actor_delay: int = 20\n \"\"\"\n the number of critic updates performed before each actor update.\n The actor network is only updated once for every actor_delay critic updates, implementing\n a delayed policy update strategy similar to TD3.\n Larger values stabilize training by allowing critics to become more accurate before policy updates.\n Smaller values allow the policy to adapt more quickly but may lead to less stable learning.\n The REDQ paper recommends a value of 20 for most tasks.\n \"\"\"\n target_mode: Literal[\"mean\", \"min\"] = \"min\"\n \"\"\"\n the method used to aggregate Q-values from the subset of critic networks.\n Can be either \"min\" or \"mean\".\n If \"min\", uses the minimum Q-value across the selected subset of critics for each state-action pair.\n If \"mean\", uses the average Q-value across the selected subset of critics.\n Using \"min\" helps prevent overestimation bias but may lead to more conservative value estimates.\n Using \"mean\" provides more optimistic value estimates but may suffer from overestimation bias.\n Default is \"min\" following the conservative value estimation approach common in recent Q-learning\n algorithms.\n \"\"\"\n\n def _get_param_transformers(self) -> list[ParamTransformer]:\n transformers = super()._get_param_transformers()\n transformers.extend(ParamsMixinAlpha._get_param_transformers(self))\n return transformers\n\n\n@dataclass(kw_only=True)\nclass TD3Params(\n Params,\n ParamsMixinGamma,\n ParamsMixinActorAndDualCritics,\n ParamsMixinExplorationNoise,\n ParamsMixinActionScalingAndBounding,\n ParamsMixinNStepReturnHorizon,\n ParamsMixinTau,\n):\n policy_noise: float | FloatEnvValueFactory = 0.2\n \"\"\"\n scaling factor for the Gaussian noise added to target policy actions.\n This parameter implements target policy smoothing, a regularization technique described in the TD3 paper.\n The noise is sampled from a normal distribution and multiplied by this value before being added to actions.\n Higher values increase exploration in the target policy, helping to address function approximation error.\n The added noise is optionally clipped to a range determined by the noise_clip parameter.\n Typically set between 0.1 and 0.5 relative to the action scale of the environment.\n \"\"\"\n noise_clip: float | FloatEnvValueFactory = 0.5\n \"\"\"\n defines the maximum absolute value of the noise added to target policy actions, i.e. noise values\n are clipped to the range [-noise_clip, noise_clip] (after generating and scaling the noise\n via `policy_noise`).\n This parameter implements bounded target policy smoothing as described in the TD3 paper.\n It prevents extreme noise values from causing unrealistic target values during training.\n Setting it 0.0 (or a negative value) disables clipping entirely.\n It is typically set to about twice the `policy_noise` value (e.g. 0.5 when `policy_noise` is 0.2).\n \"\"\"\n update_actor_freq: int = 2\n \"\"\"\n the frequency of actor network updates relative to critic network updates\n (the actor network is only updated once for every `update_actor_freq` critic updates).\n This implements the \"delayed\" policy updates from the TD3 algorithm, where the actor is\n updated less frequently than the critics.\n Higher values (e.g., 2-5) help stabilize training by allowing the critic to become more\n accurate before updating the policy.\n The default value of 2 follows the original TD3 paper's recommendation of updating the\n policy at half the rate of the Q-functions.\n \"\"\"\n\n def _get_param_transformers(self) -> list[ParamTransformer]:\n transformers = super()._get_param_transformers()\n transformers.extend(ParamsMixinActorAndDualCritics._get_param_transformers(self))\n transformers.extend(ParamsMixinExplorationNoise._get_param_transformers(self))\n transformers.extend(ParamsMixinActionScalingAndBounding._get_param_transformers(self))\n transformers.append(ParamTransformerFloatEnvParamFactory(\"policy_noise\"))\n transformers.append(ParamTransformerFloatEnvParamFactory(\"noise_clip\"))\n return transformers\n" }, { "path": "tianshou/highlevel/params/algorithm_wrapper.py", "content": "from abc import ABC, abstractmethod\nfrom collections.abc import Sequence\nfrom typing import Generic, TypeVar\n\nfrom sensai.util.string import ToStringMixin\n\nfrom tianshou.algorithm import Algorithm, ICMOffPolicyWrapper\nfrom tianshou.algorithm.algorithm_base import OffPolicyAlgorithm, OnPolicyAlgorithm\nfrom tianshou.algorithm.modelbased.icm import ICMOnPolicyWrapper\nfrom tianshou.highlevel.env import Environments\nfrom tianshou.highlevel.module.core import TDevice\nfrom tianshou.highlevel.module.intermediate import IntermediateModuleFactory\nfrom tianshou.highlevel.params.optim import OptimizerFactoryFactory\nfrom tianshou.utils.net.discrete import IntrinsicCuriosityModule\n\nTAlgorithmOut = TypeVar(\"TAlgorithmOut\", bound=Algorithm)\n\n\nclass AlgorithmWrapperFactory(Generic[TAlgorithmOut], ToStringMixin, ABC):\n @abstractmethod\n def create_wrapped_algorithm(\n self,\n policy: Algorithm,\n envs: Environments,\n optim_factory: OptimizerFactoryFactory,\n device: TDevice,\n ) -> TAlgorithmOut:\n pass\n\n\nclass AlgorithmWrapperFactoryIntrinsicCuriosity(\n AlgorithmWrapperFactory[ICMOffPolicyWrapper | ICMOnPolicyWrapper],\n):\n def __init__(\n self,\n *,\n feature_net_factory: IntermediateModuleFactory,\n hidden_sizes: Sequence[int],\n lr: float,\n lr_scale: float,\n reward_scale: float,\n forward_loss_weight: float,\n optim: OptimizerFactoryFactory | None = None,\n ):\n self.feature_net_factory = feature_net_factory\n self.hidden_sizes = hidden_sizes\n self.lr = lr\n self.lr_scale = lr_scale\n self.reward_scale = reward_scale\n self.forward_loss_weight = forward_loss_weight\n self.optim_factory = optim\n\n def create_wrapped_algorithm(\n self,\n algorithm: Algorithm,\n envs: Environments,\n optim_factory_default: OptimizerFactoryFactory,\n device: TDevice,\n ) -> ICMOffPolicyWrapper | ICMOnPolicyWrapper:\n feature_net = self.feature_net_factory.create_intermediate_module(envs, device)\n action_dim = envs.get_action_shape()\n if not isinstance(action_dim, int):\n raise ValueError(f\"Environment action shape must be an integer, got {action_dim}\")\n feature_dim = feature_net.output_dim\n icm_net = IntrinsicCuriosityModule(\n feature_net=feature_net.module,\n feature_dim=feature_dim,\n action_dim=action_dim,\n hidden_sizes=self.hidden_sizes,\n )\n optim_factory = self.optim_factory or optim_factory_default\n icm_optim = optim_factory.create_optimizer_factory(lr=self.lr)\n if isinstance(algorithm, OffPolicyAlgorithm):\n return ICMOffPolicyWrapper(\n wrapped_algorithm=algorithm,\n model=icm_net,\n optim=icm_optim,\n lr_scale=self.lr_scale,\n reward_scale=self.reward_scale,\n forward_loss_weight=self.forward_loss_weight,\n ).to(device)\n elif isinstance(algorithm, OnPolicyAlgorithm):\n return ICMOnPolicyWrapper(\n wrapped_algorithm=algorithm,\n model=icm_net,\n optim=icm_optim,\n lr_scale=self.lr_scale,\n reward_scale=self.reward_scale,\n forward_loss_weight=self.forward_loss_weight,\n ).to(device)\n else:\n raise ValueError(f\"{algorithm} is not supported by ICM\")\n" }, { "path": "tianshou/highlevel/params/alpha.py", "content": "from abc import ABC, abstractmethod\n\nimport numpy as np\nfrom sensai.util.string import ToStringMixin\n\nfrom tianshou.algorithm.modelfree.sac import Alpha, AutoAlpha\nfrom tianshou.highlevel.env import Environments\nfrom tianshou.highlevel.module.core import TDevice\nfrom tianshou.highlevel.params.optim import OptimizerFactoryFactory\n\n\nclass AutoAlphaFactory(ToStringMixin, ABC):\n @abstractmethod\n def create_auto_alpha(\n self,\n envs: Environments,\n device: TDevice,\n ) -> Alpha:\n pass\n\n\nclass AutoAlphaFactoryDefault(AutoAlphaFactory):\n def __init__(\n self,\n lr: float = 3e-4,\n target_entropy_coefficient: float = -1.0,\n log_alpha: float = 0.0,\n optim: OptimizerFactoryFactory | None = None,\n ) -> None:\n \"\"\"\n :param lr: the learning rate for the optimizer of the alpha parameter\n :param target_entropy_coefficient: the coefficient with which to multiply the target entropy;\n The base value being scaled is `dim(A)` for continuous action spaces and `log(|A|)` for discrete action spaces,\n i.e. with the default coefficient -1, we obtain `-dim(A)` and `-log(dim(A))` for continuous and discrete action\n spaces respectively, which gives a reasonable trade-off between exploration and exploitation.\n For decidedly stochastic exploration, you can use a positive value closer to 1 (e.g. 0.98);\n 1.0 would give full entropy exploration.\n :param log_alpha: the (initial) value of the log of the entropy regularization coefficient alpha.\n :param optim: the optimizer factory to use; if None, use default\n \"\"\"\n self.lr = lr\n self.target_entropy_coefficient = target_entropy_coefficient\n self.log_alpha = log_alpha\n self.optimizer_factory_factory = optim or OptimizerFactoryFactory.default()\n\n def create_auto_alpha(\n self,\n envs: Environments,\n device: TDevice,\n ) -> AutoAlpha:\n action_dim = np.prod(envs.get_action_shape())\n if envs.get_type().is_continuous():\n target_entropy = self.target_entropy_coefficient * float(action_dim)\n else:\n target_entropy = self.target_entropy_coefficient * np.log(action_dim)\n optim_factory = self.optimizer_factory_factory.create_optimizer_factory(lr=self.lr)\n return AutoAlpha(target_entropy, self.log_alpha, optim_factory)\n" }, { "path": "tianshou/highlevel/params/collector.py", "content": "from abc import ABC, abstractmethod\n\nfrom tianshou.algorithm import Algorithm\nfrom tianshou.data import BaseCollector, Collector, ReplayBuffer\nfrom tianshou.env import BaseVectorEnv\n\n\nclass CollectorFactory(ABC):\n @abstractmethod\n def create_collector(\n self,\n algorithm: Algorithm,\n vector_env: BaseVectorEnv,\n buffer: ReplayBuffer | None = None,\n exploration_noise: bool = False,\n ) -> BaseCollector:\n \"\"\"\n Creates a collector for the given algorithm and vectorized environment.\n\n :param algorithm: the algorithm\n :param vector_env: the vectorized environment\n :param buffer: the replay buffer to be used by the collector;\n if None, a new buffer will be created with default parameters\n :param exploration_noise: whether action shall be modified using the policy's exploration noise\n :return: the collector\n \"\"\"\n\n\nclass CollectorFactoryDefault(CollectorFactory):\n def create_collector(\n self,\n algorithm: Algorithm,\n vector_env: BaseVectorEnv,\n buffer: ReplayBuffer | None = None,\n exploration_noise: bool = False,\n ) -> BaseCollector:\n return Collector(\n algorithm.policy, vector_env, buffer=buffer, exploration_noise=exploration_noise\n )\n" }, { "path": "tianshou/highlevel/params/dist_fn.py", "content": "from abc import ABC, abstractmethod\nfrom collections.abc import Callable\nfrom typing import Any\n\nimport torch\nfrom sensai.util.string import ToStringMixin\n\nfrom tianshou.algorithm.modelfree.reinforce import TDistFnDiscrete, TDistFnDiscrOrCont\nfrom tianshou.highlevel.env import Environments\n\n\nclass DistributionFunctionFactory(ToStringMixin, ABC):\n # True return type defined in subclasses\n @abstractmethod\n def create_dist_fn(\n self,\n envs: Environments,\n ) -> Callable[[Any], torch.distributions.Distribution]:\n pass\n\n\nclass DistributionFunctionFactoryCategorical(DistributionFunctionFactory):\n def __init__(self, is_probs_input: bool = True):\n \"\"\"\n :param is_probs_input: If True, the distribution function shall create a categorical distribution from a\n tensor containing probabilities; otherwise the tensor is assumed to contain logits.\n \"\"\"\n self.is_probs_input = is_probs_input\n\n def create_dist_fn(self, envs: Environments) -> TDistFnDiscrete:\n envs.get_type().assert_discrete(self)\n if self.is_probs_input:\n return self._dist_fn_probs\n else:\n return self._dist_fn\n\n # NOTE: Do not move/rename because a reference to the function can appear in persisted policies\n @staticmethod\n def _dist_fn(logits: torch.Tensor) -> torch.distributions.Categorical:\n return torch.distributions.Categorical(logits=logits)\n\n # NOTE: Do not move/rename because a reference to the function can appear in persisted policies\n @staticmethod\n def _dist_fn_probs(probs: torch.Tensor) -> torch.distributions.Categorical:\n return torch.distributions.Categorical(probs=probs)\n\n\nclass DistributionFunctionFactoryIndependentGaussians(DistributionFunctionFactory):\n def create_dist_fn(self, envs: Environments) -> TDistFnDiscrOrCont:\n envs.get_type().assert_continuous(self)\n return self._dist_fn\n\n # NOTE: Do not move/rename because a reference to the function can appear in persisted policies\n @staticmethod\n def _dist_fn(\n loc_scale: tuple[torch.Tensor, torch.Tensor],\n ) -> torch.distributions.Distribution:\n loc, scale = loc_scale\n return torch.distributions.Independent(torch.distributions.Normal(loc, scale), 1)\n" }, { "path": "tianshou/highlevel/params/env_param.py", "content": "\"\"\"Factories for the generation of environment-dependent parameters.\"\"\"\n\nfrom abc import ABC, abstractmethod\nfrom typing import Generic, TypeVar\n\nfrom sensai.util.string import ToStringMixin\n\nfrom tianshou.highlevel.env import ContinuousEnvironments, Environments\n\nTValue = TypeVar(\"TValue\")\nTEnvs = TypeVar(\"TEnvs\", bound=Environments)\n\n\nclass EnvValueFactory(Generic[TValue, TEnvs], ToStringMixin, ABC):\n @abstractmethod\n def create_value(self, envs: TEnvs) -> TValue:\n pass\n\n\nclass FloatEnvValueFactory(EnvValueFactory[float, TEnvs], Generic[TEnvs], ABC):\n \"\"\"Serves as a type bound for float value factories.\"\"\"\n\n\nclass FloatEnvValueFactoryMaxActionScaled(FloatEnvValueFactory[ContinuousEnvironments]):\n def __init__(self, value: float):\n \"\"\":param value: value with which to scale the max action value\"\"\"\n self.value = value\n\n def create_value(self, envs: ContinuousEnvironments) -> float:\n envs.get_type().assert_continuous(self)\n return envs.max_action * self.value\n\n\nclass MaxActionScaled(FloatEnvValueFactoryMaxActionScaled):\n pass\n" }, { "path": "tianshou/highlevel/params/lr_scheduler.py", "content": "from abc import ABC, abstractmethod\n\nfrom sensai.util.string import ToStringMixin\n\nfrom tianshou.algorithm.optim import LRSchedulerFactory, LRSchedulerFactoryLinear\nfrom tianshou.highlevel.config import TrainingConfig\n\n\nclass LRSchedulerFactoryFactory(ToStringMixin, ABC):\n \"\"\"Factory for the creation of a learning rate scheduler factory.\"\"\"\n\n @abstractmethod\n def create_lr_scheduler_factory(self) -> LRSchedulerFactory:\n pass\n\n\nclass LRSchedulerFactoryFactoryLinear(LRSchedulerFactoryFactory):\n def __init__(self, training_config: TrainingConfig):\n self.training_config = training_config\n\n def create_lr_scheduler_factory(self) -> LRSchedulerFactory:\n if (\n self.training_config.epoch_num_steps is None\n or self.training_config.collection_step_num_env_steps is None\n ):\n raise ValueError(\n f\"{self.__class__.__name__} requires epoch_num_steps and collection_step_num_env_steps to be set \"\n f\"in order for the scheduling to be well-defined.\"\n )\n return LRSchedulerFactoryLinear(\n max_epochs=self.training_config.max_epochs,\n epoch_num_steps=self.training_config.epoch_num_steps,\n collection_step_num_env_steps=self.training_config.collection_step_num_env_steps,\n )\n" }, { "path": "tianshou/highlevel/params/noise.py", "content": "from abc import ABC, abstractmethod\n\nfrom sensai.util.string import ToStringMixin\n\nfrom tianshou.exploration import BaseNoise, GaussianNoise\nfrom tianshou.highlevel.env import ContinuousEnvironments, Environments\n\n\nclass NoiseFactory(ToStringMixin, ABC):\n @abstractmethod\n def create_noise(self, envs: Environments) -> BaseNoise:\n pass\n\n\nclass NoiseFactoryMaxActionScaledGaussian(NoiseFactory):\n def __init__(self, std_fraction: float):\n \"\"\"Factory for Gaussian noise where the standard deviation is a fraction of the maximum action value.\n\n This factory can only be applied to continuous action spaces.\n\n :param std_fraction: fraction (between 0 and 1) of the maximum action value that\n shall be used as the standard deviation\n \"\"\"\n self.std_fraction = std_fraction\n\n def create_noise(self, envs: Environments) -> GaussianNoise:\n envs.get_type().assert_continuous(self)\n envs: ContinuousEnvironments\n return GaussianNoise(sigma=envs.max_action * self.std_fraction)\n\n\nclass MaxActionScaledGaussian(NoiseFactoryMaxActionScaledGaussian):\n pass\n" }, { "path": "tianshou/highlevel/params/optim.py", "content": "from abc import ABC, abstractmethod\nfrom collections.abc import Iterable\nfrom typing import Any, Protocol, TypeAlias\n\nimport torch\nfrom sensai.util.string import ToStringMixin\n\nfrom tianshou.algorithm.optim import (\n AdamOptimizerFactory,\n OptimizerFactory,\n RMSpropOptimizerFactory,\n TorchOptimizerFactory,\n)\n\nTParams: TypeAlias = Iterable[torch.Tensor] | Iterable[dict[str, Any]]\n\n\nclass OptimizerWithLearningRateProtocol(Protocol):\n def __call__(self, parameters: Any, lr: float, **kwargs: Any) -> torch.optim.Optimizer:\n pass\n\n\nclass OptimizerFactoryFactory(ABC, ToStringMixin):\n @staticmethod\n def default() -> \"OptimizerFactoryFactory\":\n return OptimizerFactoryFactoryAdam()\n\n @abstractmethod\n def create_optimizer_factory(self, lr: float) -> OptimizerFactory:\n pass\n\n\nclass OptimizerFactoryFactoryTorch(OptimizerFactoryFactory):\n def __init__(self, optim_class: OptimizerWithLearningRateProtocol, **kwargs: Any):\n \"\"\"Factory for torch optimizers.\n\n :param optim_class: the optimizer class (e.g. subclass of `torch.optim.Optimizer`),\n which will be passed the module parameters, the learning rate as `lr` and the\n kwargs provided.\n :param kwargs: keyword arguments to provide at optimizer construction\n \"\"\"\n self.optim_class = optim_class\n self.kwargs = kwargs\n\n def create_optimizer_factory(self, lr: float) -> OptimizerFactory:\n return TorchOptimizerFactory(optim_class=self.optim_class, lr=lr)\n\n\nclass OptimizerFactoryFactoryAdam(OptimizerFactoryFactory):\n def __init__(\n self,\n betas: tuple[float, float] = (0.9, 0.999),\n eps: float = 1e-08,\n weight_decay: float = 0,\n ):\n self.weight_decay = weight_decay\n self.eps = eps\n self.betas = betas\n\n def create_optimizer_factory(self, lr: float) -> AdamOptimizerFactory:\n return AdamOptimizerFactory(\n lr=lr,\n betas=self.betas,\n eps=self.eps,\n weight_decay=self.weight_decay,\n )\n\n\nclass OptimizerFactoryFactoryRMSprop(OptimizerFactoryFactory):\n def __init__(\n self,\n alpha: float = 0.99,\n eps: float = 1e-08,\n weight_decay: float = 0,\n momentum: float = 0,\n centered: bool = False,\n ):\n self.alpha = alpha\n self.momentum = momentum\n self.centered = centered\n self.weight_decay = weight_decay\n self.eps = eps\n\n def create_optimizer_factory(self, lr: float) -> RMSpropOptimizerFactory:\n return RMSpropOptimizerFactory(\n lr=lr,\n alpha=self.alpha,\n eps=self.eps,\n weight_decay=self.weight_decay,\n momentum=self.momentum,\n centered=self.centered,\n )\n" }, { "path": "tianshou/highlevel/persistence.py", "content": "import logging\nfrom abc import ABC, abstractmethod\nfrom collections.abc import Callable\nfrom enum import Enum\nfrom pathlib import Path\nfrom typing import TYPE_CHECKING\n\nimport torch\n\nfrom tianshou.highlevel.world import World\n\nif TYPE_CHECKING:\n from tianshou.highlevel.module.core import TDevice\n\nlog = logging.getLogger(__name__)\n\n\nclass PersistEvent(Enum):\n \"\"\"Enumeration of persistence events that Persistence objects can react to.\"\"\"\n\n PERSIST_POLICY = \"persist_policy\"\n \"\"\"Policy neural network is persisted (new best found)\"\"\"\n\n\nclass RestoreEvent(Enum):\n \"\"\"Enumeration of restoration events that Persistence objects can react to.\"\"\"\n\n RESTORE_POLICY = \"restore_policy\"\n \"\"\"Policy neural network parameters are restored\"\"\"\n\n\nclass Persistence(ABC):\n @abstractmethod\n def persist(self, event: PersistEvent, world: World) -> None:\n pass\n\n @abstractmethod\n def restore(self, event: RestoreEvent, world: World) -> None:\n pass\n\n\nclass PersistenceGroup(Persistence):\n \"\"\"Groups persistence handler such that they can be applied collectively.\"\"\"\n\n def __init__(self, *p: Persistence, enabled: bool = True):\n self.items = p\n self.enabled = enabled\n\n def persist(self, event: PersistEvent, world: World) -> None:\n if not self.enabled:\n return\n for item in self.items:\n item.persist(event, world)\n\n def restore(self, event: RestoreEvent, world: World) -> None:\n for item in self.items:\n item.restore(event, world)\n\n\nclass PolicyPersistence:\n class Mode(Enum):\n \"\"\"Mode of persistence.\"\"\"\n\n POLICY_STATE_DICT = \"policy_state_dict\"\n \"\"\"Persist only the policy's state dictionary. Note that for a policy to be restored from\n such a dictionary, it is necessary to first create a structurally equivalent object which can\n accept the respective state.\"\"\"\n POLICY = \"policy\"\n \"\"\"Persist the entire policy. This is larger but has the advantage of the policy being loadable\n without requiring an environment to be instantiated.\n It has the potential disadvantage that upon breaking code changes in the policy implementation\n (e.g. renamed/moved class), it will no longer be loadable.\n Note that a precondition is that the policy be picklable in its entirety.\n \"\"\"\n\n def get_filename(self) -> str:\n return self.value + \".pt\"\n\n def __init__(\n self,\n additional_persistence: Persistence | None = None,\n enabled: bool = True,\n mode: Mode = Mode.POLICY,\n ):\n \"\"\"Handles persistence of the policy.\n\n :param additional_persistence: a persistence instance which is to be invoked whenever\n this object is used to persist/restore data\n :param enabled: whether persistence is enabled (restoration is always enabled)\n :param mode: the persistence mode\n \"\"\"\n self.additional_persistence = additional_persistence\n self.enabled = enabled\n self.mode = mode\n\n def persist(self, policy: torch.nn.Module, world: World) -> None:\n if not self.enabled:\n return\n path = world.persist_path(self.mode.get_filename())\n match self.mode:\n case self.Mode.POLICY_STATE_DICT:\n log.info(f\"Saving policy state dictionary in {path}\")\n torch.save(policy.state_dict(), path)\n case self.Mode.POLICY:\n log.info(f\"Saving policy object in {path}\")\n torch.save(policy, path)\n case _:\n raise NotImplementedError\n if self.additional_persistence is not None:\n self.additional_persistence.persist(PersistEvent.PERSIST_POLICY, world)\n\n def restore(self, policy: torch.nn.Module, world: World, device: \"TDevice\") -> None:\n path = world.restore_path(self.mode.get_filename())\n log.info(f\"Restoring policy from {path}\")\n match self.mode:\n case self.Mode.POLICY_STATE_DICT:\n state_dict = torch.load(path, map_location=device)\n case self.Mode.POLICY:\n loaded_policy: torch.nn.Module = torch.load(path, map_location=device)\n state_dict = loaded_policy.state_dict()\n case _:\n raise NotImplementedError\n policy.load_state_dict(state_dict)\n if self.additional_persistence is not None:\n self.additional_persistence.restore(RestoreEvent.RESTORE_POLICY, world)\n\n def get_save_best_fn(self, world: World) -> Callable[[torch.nn.Module], None]:\n def save_best_fn(pol: torch.nn.Module) -> None:\n self.persist(pol, world)\n\n return save_best_fn\n\n def get_save_checkpoint_fn(self, world: World) -> Callable[[int, int, int], str] | None:\n if not self.enabled:\n return None\n\n def save_checkpoint_fn(epoch: int, env_step: int, gradient_step: int) -> str:\n path = Path(self.mode.get_filename())\n path_with_epoch = path.with_stem(f\"{path.stem}_epoch_{epoch}\")\n path = world.persist_path(path_with_epoch.name)\n match self.mode:\n case self.Mode.POLICY_STATE_DICT:\n log.info(f\"Saving policy state dictionary in {path}\")\n torch.save(world.algorithm.state_dict(), path)\n case self.Mode.POLICY:\n log.info(f\"Saving policy object in {path}\")\n torch.save(world.algorithm, path)\n case _:\n raise NotImplementedError\n if self.additional_persistence is not None:\n self.additional_persistence.persist(PersistEvent.PERSIST_POLICY, world)\n return path\n\n return save_checkpoint_fn\n" }, { "path": "tianshou/highlevel/trainer.py", "content": "import logging\nfrom abc import ABC, abstractmethod\nfrom collections.abc import Callable\nfrom dataclasses import dataclass\nfrom typing import TypeVar, cast\n\nfrom sensai.util.string import ToStringMixin\n\nfrom tianshou.algorithm import DQN, Algorithm\nfrom tianshou.algorithm.modelfree.dqn import DiscreteQLearningPolicy\nfrom tianshou.highlevel.env import Environments\nfrom tianshou.highlevel.logger import TLogger\n\nTAlgorithm = TypeVar(\"TAlgorithm\", bound=Algorithm)\nlog = logging.getLogger(__name__)\n\n\nclass TrainingContext:\n def __init__(self, algorithm: TAlgorithm, envs: Environments, logger: TLogger):\n self.algorithm = algorithm\n self.envs = envs\n self.logger = logger\n\n\nclass EpochTrainCallback(ToStringMixin, ABC):\n \"\"\"Callback which is called at the beginning of each epoch, i.e. prior to the data collection phase\n of each epoch.\n \"\"\"\n\n @abstractmethod\n def callback(self, epoch: int, env_step: int, context: TrainingContext) -> None:\n pass\n\n def get_trainer_fn(self, context: TrainingContext) -> Callable[[int, int], None]:\n def fn(epoch: int, env_step: int) -> None:\n return self.callback(epoch, env_step, context)\n\n return fn\n\n\nclass EpochTestCallback(ToStringMixin, ABC):\n \"\"\"Callback which is called at the beginning of the test phase of each epoch.\"\"\"\n\n @abstractmethod\n def callback(self, epoch: int, env_step: int | None, context: TrainingContext) -> None:\n pass\n\n def get_trainer_fn(self, context: TrainingContext) -> Callable[[int, int | None], None]:\n def fn(epoch: int, env_step: int | None) -> None:\n return self.callback(epoch, env_step, context)\n\n return fn\n\n\nclass EpochStopCallback(ToStringMixin, ABC):\n \"\"\"Callback which is called after the test phase of each epoch in order to determine\n whether training should stop early.\n \"\"\"\n\n @abstractmethod\n def should_stop(self, mean_rewards: float, context: TrainingContext) -> bool:\n \"\"\"Determines whether training should stop.\n\n :param mean_rewards: the average undiscounted returns of the testing result\n :param context: the training context\n :return: True if the goal has been reached and training should stop, False otherwise\n \"\"\"\n\n def get_trainer_fn(self, context: TrainingContext) -> Callable[[float], bool]:\n def fn(mean_rewards: float) -> bool:\n return self.should_stop(mean_rewards, context)\n\n return fn\n\n\n@dataclass\nclass TrainerCallbacks:\n \"\"\"Container for callbacks used during training.\"\"\"\n\n epoch_train_callback: EpochTrainCallback | None = None\n epoch_test_callback: EpochTestCallback | None = None\n epoch_stop_callback: EpochStopCallback | None = None\n\n\nclass EpochTrainCallbackDQNSetEps(EpochTrainCallback):\n \"\"\"Sets the epsilon value for DQN-based policies at the beginning of the training\n stage in each epoch.\n \"\"\"\n\n def __init__(self, eps: float):\n self.eps = eps\n\n def callback(self, epoch: int, env_step: int, context: TrainingContext) -> None:\n algorithm = cast(DQN, context.algorithm)\n policy: DiscreteQLearningPolicy = algorithm.policy\n policy.set_eps_training(self.eps)\n\n\nclass EpochTrainCallbackDQNEpsLinearDecay(EpochTrainCallback):\n \"\"\"Sets the epsilon value for DQN-based policies at the beginning of the training\n stage in each epoch, using a linear decay in the first `decay_steps` steps.\n \"\"\"\n\n def __init__(self, eps_train: float, eps_train_final: float, decay_steps: int = 1000000):\n self.eps_train = eps_train\n self.eps_train_final = eps_train_final\n self.decay_steps = decay_steps\n\n def callback(self, epoch: int, env_step: int, context: TrainingContext) -> None:\n algorithm = cast(DQN, context.algorithm)\n policy: DiscreteQLearningPolicy = algorithm.policy\n logger = context.logger\n if env_step <= self.decay_steps:\n eps = self.eps_train - env_step / self.decay_steps * (\n self.eps_train - self.eps_train_final\n )\n else:\n eps = self.eps_train_final\n policy.set_eps_training(eps)\n logger.write(\"train/env_step\", env_step, {\"train/eps\": eps})\n\n\nclass EpochTestCallbackDQNSetEps(EpochTestCallback):\n \"\"\"Sets the epsilon value for DQN-based policies at the beginning of the test\n stage in each epoch.\n \"\"\"\n\n def __init__(self, eps: float):\n self.eps = eps\n\n def callback(self, epoch: int, env_step: int | None, context: TrainingContext) -> None:\n algorithm = cast(DQN, context.algorithm)\n policy: DiscreteQLearningPolicy = algorithm.policy\n policy.set_eps_inference(self.eps)\n\n\nclass EpochStopCallbackRewardThreshold(EpochStopCallback):\n \"\"\"Stops training once the mean rewards exceed the given reward threshold or the threshold that\n is specified in the gymnasium environment (i.e. `env.spec.reward_threshold`).\n \"\"\"\n\n def __init__(self, threshold: float | None = None):\n \"\"\"\n :param threshold: the reward threshold beyond which to stop training.\n If it is None, will use threshold specified by the environment, i.e. `env.spec.reward_threshold`.\n \"\"\"\n self.threshold = threshold\n\n def should_stop(self, mean_rewards: float, context: TrainingContext) -> bool:\n threshold = self.threshold\n if threshold is None:\n threshold = context.envs.env.spec.reward_threshold # type: ignore\n assert threshold is not None\n is_reached = mean_rewards >= threshold\n if is_reached:\n log.info(f\"Reward threshold ({threshold}) exceeded\")\n return is_reached\n" }, { "path": "tianshou/highlevel/world.py", "content": "import os\nfrom dataclasses import dataclass\nfrom typing import TYPE_CHECKING, Optional\n\nif TYPE_CHECKING:\n from tianshou.algorithm import Algorithm\n from tianshou.data import BaseCollector\n from tianshou.highlevel.env import Environments\n from tianshou.highlevel.logger import TLogger\n from tianshou.trainer import Trainer\n\n\n@dataclass(kw_only=True)\nclass World:\n \"\"\"Container for instances and configuration items that are relevant to an experiment.\"\"\"\n\n envs: \"Environments\"\n algorithm: \"Algorithm\"\n training_collector: Optional[\"BaseCollector\"] = None\n test_collector: Optional[\"BaseCollector\"] = None\n logger: \"TLogger\"\n persist_directory: str\n restore_directory: str | None\n trainer: Optional[\"Trainer\"] = None\n\n def persist_path(self, filename: str) -> str:\n return os.path.abspath(os.path.join(self.persist_directory, filename))\n\n def restore_path(self, filename: str) -> str:\n if self.restore_directory is None:\n raise ValueError(\n \"Path cannot be formed because no directory for restoration was provided\",\n )\n return os.path.join(self.restore_directory, filename)\n" }, { "path": "tianshou/py.typed", "content": "" }, { "path": "tianshou/trainer.py", "content": "\"\"\"\nThis module contains Tianshou's trainer classes, which orchestrate the training and call upon an RL algorithm's\nspecific network updating logic to perform the actual gradient updates.\n\nTraining is structured as follows (hierarchical glossary):\n\n- **epoch**: the outermost iteration level of the training loop. Each epoch consists of a number of training steps\n and one test step (see :attr:`TrainerParams.max_epoch` for a detailed explanation).\n\n - **training step**: a training step performs the steps necessary in order to apply a single update of the neural\n network components as defined by the underlying RL algorithm (:class:`Algorithm`). This involves the following sub-steps:\n\n - for online learning algorithms:\n\n - **collection step**: collecting environment steps/transitions to be used for training.\n\n - (Potentially) a test step (see below) if the early stopping criterion is satisfied based on\n the data collected (see :attr:`OnlineTrainerParams.test_in_train`).\n\n - **update step**: applying the actual gradient updates using the RL algorithm.\n The update is based on either:\n\n - data from only the preceding collection step (on-policy learning),\n - data from the collection step and previously collected data (off-policy learning), or\n - data from the user-provided replay buffer (offline learning).\n\n For offline learning algorithms, a training step is thus equivalent to an update step.\n\n - **test step**: collects test episodes from dedicated test environments which are used to evaluate the performance\n of the policy. Optionally, the performance result can be used to determine whether training shall stop early\n (see :attr:`TrainerParams.stop_fn`).\n\"\"\"\n\nimport logging\nimport time\nfrom abc import ABC, abstractmethod\nfrom collections import defaultdict\nfrom collections.abc import Callable\nfrom dataclasses import asdict, dataclass\nfrom functools import partial\nfrom typing import Generic, TypeVar\n\nimport numpy as np\nimport torch\nimport tqdm\nfrom sensai.util.helper import count_none\nfrom sensai.util.pickle import setstate\nfrom sensai.util.string import ToStringMixin\n\nfrom tianshou.algorithm.algorithm_base import (\n Algorithm,\n OfflineAlgorithm,\n OffPolicyAlgorithm,\n OnPolicyAlgorithm,\n TrainingStats,\n)\nfrom tianshou.data import (\n AsyncCollector,\n CollectStats,\n EpochStats,\n InfoStats,\n ReplayBuffer,\n SequenceSummaryStats,\n TimingStats,\n)\nfrom tianshou.data.buffer.buffer_base import MalformedBufferError\nfrom tianshou.data.collector import BaseCollector, CollectStatsBase\nfrom tianshou.utils import (\n BaseLogger,\n LazyLogger,\n MovAvg,\n)\nfrom tianshou.utils.determinism import TraceLogger, torch_param_hash\nfrom tianshou.utils.logging import set_numerical_fields_to_precision\nfrom tianshou.utils.torch_utils import policy_within_training_step\n\nlog = logging.getLogger(__name__)\n\n\n@dataclass(kw_only=True)\nclass TrainerParams(ToStringMixin):\n max_epochs: int = 100\n \"\"\"\n the (maximum) number of epochs to run training for. An **epoch** is the outermost iteration level and each\n epoch consists of a number of training steps and one test step, where each training step\n\n * [for the online case] collects environment steps/transitions (**collection step**),\n adding them to the (replay) buffer (see :attr:`collection_step_num_env_steps` and :attr:`collection_step_num_episodes`)\n * performs an **update step** via the RL algorithm being used, which can involve\n one or more actual gradient updates, depending on the algorithm\n\n and the test step collects :attr:`num_episodes_per_test` test episodes in order to evaluate\n agent performance.\n\n Training may be stopped early if the stop criterion is met (see :attr:`stop_fn`).\n\n For online training, the number of training steps in each epoch is indirectly determined by\n :attr:`epoch_num_steps`: As many training steps will be performed as are required in\n order to reach :attr:`epoch_num_steps` total steps in the training environments.\n Specifically, if the number of transitions collected per step is `c` (see\n :attr:`collection_step_num_env_steps`) and :attr:`epoch_num_steps` is set to `s`, then the number\n of training steps per epoch is `ceil(s / c)`.\n Therefore, if `max_epochs = e`, the total number of environment steps taken during training\n can be computed as `e * ceil(s / c) * c`.\n\n For offline training, the number of training steps per epoch is equal to :attr:`epoch_num_steps`.\n \"\"\"\n\n epoch_num_steps: int = 30000\n \"\"\"\n For an online algorithm, this is the total number of environment steps to be collected per epoch, and,\n for an offline algorithm, it is the total number of training steps to take per epoch.\n See :attr:`max_epochs` for an explanation of epoch semantics.\n \"\"\"\n\n test_collector: BaseCollector | None = None\n \"\"\"\n the collector to use for test episode collection (test steps); if None, perform no test steps.\n \"\"\"\n\n test_step_num_episodes: int = 1\n \"\"\"the number of episodes to collect in each test step.\n \"\"\"\n\n training_fn: Callable[[int, int], None] | None = None\n \"\"\"\n a callback function which is called at the beginning of each training step.\n It can be used to perform custom additional operations, with the\n signature ``f(num_epoch: int, step_idx: int) -> None``.\n \"\"\"\n\n test_fn: Callable[[int, int | None], None] | None = None\n \"\"\"\n a callback function to be called at the beginning of each test step.\n It can be used to perform custom additional operations, with the\n signature ``f(num_epoch: int, step_idx: int) -> None``.\n \"\"\"\n\n stop_fn: Callable[[float], bool] | None = None\n \"\"\"\n a callback function with signature ``f(score: float) -> bool``, which\n is used to decide whether training shall be stopped early based on the score\n achieved in a test step.\n The score it receives is computed by the :attr:`compute_score_fn` callback\n (which defaults to the mean reward if the function is not provided).\n\n Requires test steps to be activated and thus :attr:`test_collector` to be set.\n\n Note: The function is also used when :attr:`test_in_train` is activated (see docstring).\n \"\"\"\n\n compute_score_fn: Callable[[CollectStats], float] | None = None\n \"\"\"\n the callback function to use in order to compute the test batch performance score, which is used to\n determine what the best model is (score is maximized); if None, use the mean reward.\n \"\"\"\n\n save_best_fn: Callable[[\"Algorithm\"], None] | None = None\n \"\"\"\n the callback function to call in order to save the best model whenever a new best score (see :attr:`compute_score_fn`)\n is achieved in a test step. It should have the signature ``f(algorithm: Algorithm) -> None``.\n \"\"\"\n\n save_checkpoint_fn: Callable[[int, int, int], str] | None = None\n \"\"\"\n the callback function with which to save checkpoint data after each training step,\n which can save whatever data is desired to a file and returns the path of the file.\n Signature: ``f(epoch: int, env_step: int, gradient_step: int) -> str``.\n \"\"\"\n\n resume_from_log: bool = False\n \"\"\"\n whether to load env_step/gradient_step and other metadata from the existing log,\n which is given in :attr:`logger`.\n \"\"\"\n\n multi_agent_return_reduction: Callable[[np.ndarray], np.ndarray] | None = None\n \"\"\"\n a function with signature\n ``f(returns: np.ndarray with shape (num_episode, agent_num)) -> np.ndarray with shape (num_episode,)``,\n which is used in multi-agent RL. We need to return a single scalar for each episode's return\n to monitor training in the multi-agent RL setting. This function specifies what is the desired metric,\n e.g., the return achieved by agent 1 or the average return over all agents.\n \"\"\"\n\n logger: BaseLogger | None = None\n \"\"\"\n the logger with which to log statistics during training/testing/updating. To not log anything, use None.\n\n Relevant step types for logger update intervals:\n * `update_interval`: update step\n * `training_interval`: env step\n * `test_interval`: env step\n \"\"\"\n\n verbose: bool = True\n \"\"\"\n whether to print status information to stdout.\n If set to False, status information will still be logged (provided that logging is enabled via the\n `logging` Python module).\n \"\"\"\n\n show_progress: bool = True\n \"\"\"\n whether to display a progress bars during training.\n \"\"\"\n\n def __setstate__(self, state: dict) -> None:\n setstate(TrainerParams, self, state, renamed_properties={\"train_fn\": \"training_fn\"})\n\n def __post_init__(self) -> None:\n if self.resume_from_log and self.logger is None:\n raise ValueError(\"Cannot resume from log without a logger being provided\")\n if self.test_collector is None:\n if self.stop_fn is not None:\n raise ValueError(\n \"stop_fn cannot be activated without test steps being enabled (test_collector being set)\"\n )\n if self.test_fn is not None:\n raise ValueError(\n \"test_fn is set while test steps are disabled (test_collector is None)\"\n )\n if self.save_best_fn is not None:\n raise ValueError(\n \"save_best_fn is set while test steps are disabled (test_collector is None)\"\n )\n else:\n if self.test_step_num_episodes < 1:\n raise ValueError(\n \"test_step_num_episodes must be positive if test steps are enabled \"\n \"(test_collector not None)\"\n )\n\n\n@dataclass(kw_only=True)\nclass OnlineTrainerParams(TrainerParams):\n training_collector: BaseCollector\n \"\"\"\n the collector with which to gather new data for training in each training step\n \"\"\"\n\n collection_step_num_env_steps: int | None = 2048\n \"\"\"\n the number of environment steps/transitions to collect in each collection step before the\n network update within each training step.\n\n This is mutually exclusive with :attr:`collection_step_num_episodes`, and one of the two must be set.\n\n Note that the exact number can be reached only if this is a multiple of the number of\n training environments being used, as each training environment will produce the same\n (non-zero) number of transitions.\n Specifically, if this is set to `n` and `m` training environments are used, then the total\n number of transitions collected per collection step is `ceil(n / m) * m =: c`.\n\n See :attr:`max_epochs` for information on the total number of environment steps being\n collected during training.\n \"\"\"\n\n collection_step_num_episodes: int | None = None\n \"\"\"\n the number of episodes to collect in each collection step before the network update within\n each training step. If this is set, the number of environment steps collected in each\n collection step is the sum of the lengths of the episodes collected.\n\n This is mutually exclusive with :attr:`collection_step_num_env_steps`, and one of the two must be set.\n \"\"\"\n\n test_in_training: bool = False\n \"\"\"\n Whether to apply a test step within a training step depending on the early stopping criterion\n (given by :attr:`stop_fn`) being satisfied based on the data collected within the training step.\n Specifically, after each collect step, we check whether the early stopping criterion (:attr:`stop_fn`)\n would be satisfied by data we collected (provided that at least one episode was indeed completed, such\n that we can evaluate returns, etc.). If the criterion is satisfied, we perform a full test step\n (collecting :attr:`test_step_num_episodes` episodes in order to evaluate performance), and if the early\n stopping criterion is also satisfied based on the test data, we stop training early.\n \"\"\"\n\n def __setstate__(self, state: dict) -> None:\n setstate(\n OnlineTrainerParams,\n self,\n state,\n renamed_properties={\n \"test_in_train\": \"test_in_training\",\n \"training_collector\": \"training_collector\",\n },\n )\n\n def __post_init__(self) -> None:\n super().__post_init__()\n if count_none(self.collection_step_num_env_steps, self.collection_step_num_episodes) != 1:\n raise ValueError(\n \"Exactly one of {collection_step_num_env_steps, collection_step_num_episodes} must be set\"\n )\n if self.test_in_training and (self.test_collector is None or self.stop_fn is None):\n raise ValueError(\"test_in_training requires test_collector and stop_fn to be set\")\n\n\n@dataclass(kw_only=True)\nclass OnPolicyTrainerParams(OnlineTrainerParams):\n batch_size: int | None = 64\n \"\"\"\n Use mini-batches of this size for gradient updates (causing the gradient to be less accurate,\n a form of regularization).\n Set ``batch_size=None`` for the full buffer that was collected within the training step to be\n used for the gradient update (no mini-batching).\n \"\"\"\n\n update_step_num_repetitions: int = 1\n \"\"\"\n controls, within one update step of an on-policy algorithm, the number of times\n the full collected data is applied for gradient updates, i.e. if the parameter is\n 5, then the collected data shall be used five times to update the policy within the same\n update step.\n \"\"\"\n\n\n@dataclass(kw_only=True)\nclass OffPolicyTrainerParams(OnlineTrainerParams):\n batch_size: int = 64\n \"\"\"\n the the number of environment steps/transitions to sample from the buffer for a gradient update.\n \"\"\"\n\n update_step_num_gradient_steps_per_sample: float = 1.0\n \"\"\"\n the number of gradient steps to perform per sample collected (see :attr:`collection_step_num_env_steps`).\n Specifically, if this is set to `u` and the number of samples collected in the preceding\n collection step is `n`, then `round(u * n)` gradient steps will be performed.\n \"\"\"\n\n\n@dataclass(kw_only=True)\nclass OfflineTrainerParams(TrainerParams):\n buffer: ReplayBuffer\n \"\"\"\n the replay buffer with environment steps to use as training data for offline learning.\n This buffer will be pre-processed using the RL algorithm's pre-processing\n function (if any) before training.\n \"\"\"\n\n batch_size: int = 64\n \"\"\"\n the number of environment steps/transitions to sample from the buffer for a gradient update.\n \"\"\"\n\n\nTTrainerParams = TypeVar(\"TTrainerParams\", bound=TrainerParams)\nTOnlineTrainerParams = TypeVar(\"TOnlineTrainerParams\", bound=OnlineTrainerParams)\nTAlgorithm = TypeVar(\"TAlgorithm\", bound=Algorithm)\n\n\nclass Trainer(Generic[TAlgorithm, TTrainerParams], ABC):\n \"\"\"\n Base class for trainers in Tianshou, which orchestrate the training process and call upon an RL algorithm's\n specific network updating logic to perform the actual gradient updates.\n\n The base class already implements the fundamental epoch logic and fully implements the test step\n logic, which is common to all trainers. The training step logic is left to be implemented by subclasses.\n \"\"\"\n\n def __init__(\n self,\n algorithm: TAlgorithm,\n params: TTrainerParams,\n ):\n self.algorithm = algorithm\n self.params = params\n\n self._logger = params.logger or LazyLogger()\n\n self._start_time = time.time()\n self._stat: defaultdict[str, MovAvg] = defaultdict(MovAvg)\n self._start_epoch = 0\n\n self._epoch = self._start_epoch\n\n # initialize stats on the best model found during a test step\n # NOTE: The values don't matter, as in the first test step (which is taken in reset()\n # at the beginning of the training process), these will all be updated\n self._best_score = 0.0\n self._best_reward = 0.0\n self._best_reward_std = 0.0\n self._best_epoch = self._start_epoch\n\n self._current_update_step = 0\n \"\"\"\n the current (1-based) update step/training step number (to be incremented before the actual step is taken)\n \"\"\"\n\n self._env_step = 0\n \"\"\"\n the step counter which is used to track progress of the training process.\n For online learning (i.e. on-policy and off-policy learning), this is the total number of\n environment steps collected, and for offline training, it is the total number of environment\n steps that have been sampled from the replay buffer to perform gradient updates.\n \"\"\"\n\n self._policy_update_time = 0.0\n\n self._compute_score_fn: Callable[[CollectStats], float] = (\n params.compute_score_fn or self._compute_score_fn_default\n )\n\n self._stop_fn_flag = False\n\n @staticmethod\n def _compute_score_fn_default(stat: CollectStats) -> float:\n \"\"\"\n The default score function, which returns the mean return/reward.\n\n :param stat: the collection stats\n :return: the mean return\n \"\"\"\n assert stat.returns_stat is not None # for mypy\n return stat.returns_stat.mean\n\n @property\n def _pbar(self) -> Callable[..., tqdm.tqdm]:\n \"\"\"Use as context manager or iterator, i.e., `with self._pbar(...) as t:` or `for _ in self._pbar(...):`.\"\"\"\n return partial(\n tqdm.tqdm,\n dynamic_ncols=True,\n ascii=True,\n disable=not self.params.show_progress,\n )\n\n def _reset_collectors(self, reset_buffer: bool = False) -> None:\n if self.params.test_collector is not None:\n self.params.test_collector.reset(reset_buffer=reset_buffer)\n\n def reset(self, reset_collectors: bool = True, reset_collector_buffers: bool = False) -> None:\n \"\"\"Initializes the training process.\n\n :param reset_collectors: whether to reset the collectors prior to starting the training process.\n Specifically, this will reset the environments in the collectors (starting new episodes),\n and the statistics stored in the collector. Whether the contained buffers will be reset/cleared\n is determined by the `reset_buffer` parameter.\n :param reset_collector_buffers: whether, for the case where the collectors are reset, to reset/clear the\n contained buffers as well.\n This has no effect if `reset_collectors` is False.\n \"\"\"\n TraceLogger.log(log, lambda: \"Trainer reset\")\n self._env_step = 0\n self._current_update_step = 0\n\n if self.params.resume_from_log:\n (\n self._start_epoch,\n self._env_step,\n self._current_update_step,\n ) = self._logger.restore_data()\n\n self._epoch = self._start_epoch\n\n self._start_time = time.time()\n\n if reset_collectors:\n self._reset_collectors(reset_buffer=reset_collector_buffers)\n\n # make an initial test step to determine the initial best model\n if self.params.test_collector is not None:\n assert self.params.test_step_num_episodes is not None\n assert not isinstance(self.params.test_collector, AsyncCollector) # Issue 700\n self._test_step(force_update_best=True, log_msg_prefix=\"Initial test step\")\n\n self._stop_fn_flag = False\n\n self._log_params(self.algorithm)\n\n def _log_params(self, module: torch.nn.Module) -> None:\n \"\"\"Logs the parameters of the module to the trace logger by subcomponent (if the trace logger is enabled).\"\"\"\n if not TraceLogger.is_enabled:\n return\n\n def module_has_params(m: torch.nn.Module) -> bool:\n return any(p.requires_grad for p in m.parameters())\n\n relevant_modules = {}\n\n def gather_modules(m: torch.nn.Module) -> None:\n for name, submodule in m.named_children():\n if name == \"policy\":\n gather_modules(submodule)\n else:\n if module_has_params(submodule):\n relevant_modules[name] = submodule\n\n gather_modules(module)\n\n for name, module in sorted(relevant_modules.items()):\n TraceLogger.log(\n log,\n lambda: f\"Params[{name}]: {torch_param_hash(module)}\",\n )\n\n class _TrainingStepResult(ABC):\n @abstractmethod\n def get_steps_in_epoch_advancement(self) -> int:\n \"\"\"\n :return: the number of steps that were done within the epoch, where the concrete semantics\n of what a step is depend on the type of algorithm. See docstring of `TrainerParams.epoch_num_steps`.\n \"\"\"\n\n @abstractmethod\n def get_collect_stats(self) -> CollectStats | None:\n pass\n\n @abstractmethod\n def get_training_stats(self) -> TrainingStats | None:\n pass\n\n @abstractmethod\n def is_training_done(self) -> bool:\n \"\"\":return: whether the early stopping criterion is satisfied and training shall stop.\"\"\"\n\n @abstractmethod\n def get_env_step_advancement(self) -> int:\n \"\"\"\n :return: the number of steps by which to advance the env_step counter in the trainer (see docstring\n of trainer attribute). The semantics depend on the type of the algorithm.\n \"\"\"\n\n @abstractmethod\n def _create_epoch_pbar_data_dict(\n self, training_step_result: _TrainingStepResult\n ) -> dict[str, str]:\n pass\n\n def _create_info_stats(\n self,\n ) -> InfoStats:\n test_collector = self.params.test_collector\n if isinstance(self.params, OnlineTrainerParams):\n training_collector = self.params.training_collector\n else:\n training_collector = None\n\n duration = max(0.0, time.time() - self._start_time)\n test_time = 0.0\n update_speed = 0.0\n train_time_collect = 0.0\n if test_collector is not None:\n test_time = test_collector.collect_time\n\n if training_collector is not None:\n train_time_collect = training_collector.collect_time\n update_speed = training_collector.collect_step / (duration - test_time)\n\n timing_stat = TimingStats(\n total_time=duration,\n train_time=duration - test_time,\n train_time_collect=train_time_collect,\n train_time_update=self._policy_update_time,\n test_time=test_time,\n update_speed=update_speed,\n )\n\n return InfoStats(\n update_step=self._current_update_step,\n best_score=self._best_score,\n best_reward=self._best_reward,\n best_reward_std=self._best_reward_std,\n train_step=training_collector.collect_step if training_collector is not None else 0,\n train_episode=training_collector.collect_episode\n if training_collector is not None\n else 0,\n test_step=test_collector.collect_step if test_collector is not None else 0,\n test_episode=test_collector.collect_episode if test_collector is not None else 0,\n timing=timing_stat,\n )\n\n def execute_epoch(self) -> EpochStats:\n self._epoch += 1\n TraceLogger.log(log, lambda: f\"Epoch #{self._epoch} start\")\n\n # perform the required number of steps for the epoch (`epoch_num_steps`)\n steps_done_in_this_epoch = 0\n train_collect_stats, training_stats = None, None\n with self._pbar(\n total=self.params.epoch_num_steps, desc=f\"Epoch #{self._epoch}\", position=1\n ) as t:\n while steps_done_in_this_epoch < self.params.epoch_num_steps and not self._stop_fn_flag:\n # perform a training step and update progress\n TraceLogger.log(log, lambda: \"Training step\")\n self._current_update_step += 1\n training_step_result = self._training_step()\n steps_done_in_this_epoch += training_step_result.get_steps_in_epoch_advancement()\n t.update(training_step_result.get_steps_in_epoch_advancement())\n self._stop_fn_flag = training_step_result.is_training_done()\n self._env_step += training_step_result.get_env_step_advancement()\n training_stats = training_step_result.get_training_stats()\n TraceLogger.log(\n log,\n lambda: f\"Training step complete: stats={training_stats.get_loss_stats_dict() if training_stats is not None else None}\",\n )\n self._log_params(self.algorithm)\n\n collect_stats = training_step_result.get_collect_stats()\n if collect_stats is not None:\n self._logger.log_training_data(asdict(collect_stats), self._env_step)\n\n pbar_data_dict = self._create_epoch_pbar_data_dict(training_step_result)\n pbar_data_dict = set_numerical_fields_to_precision(pbar_data_dict)\n pbar_data_dict[\"update_step\"] = str(self._current_update_step)\n t.set_postfix(**pbar_data_dict)\n\n test_collect_stats = None\n if not self._stop_fn_flag:\n self._logger.save_data(\n self._epoch,\n self._env_step,\n self._current_update_step,\n self.params.save_checkpoint_fn,\n )\n\n # test step\n if self.params.test_collector is not None:\n test_collect_stats, self._stop_fn_flag = self._test_step()\n\n info_stats = self._create_info_stats()\n\n self._logger.log_info_data(asdict(info_stats), self._epoch)\n\n return EpochStats(\n epoch=self._epoch,\n train_collect_stat=train_collect_stats,\n test_collect_stat=test_collect_stats,\n training_stat=training_stats,\n info_stat=info_stats,\n )\n\n def _should_stop_training_early(\n self, *, score: float | None = None, collect_stats: CollectStats | None = None\n ) -> bool:\n \"\"\"\n Determine whether, given the early stopping criterion stop_fn, training shall be stopped early\n based on the score achieved or the collection stats (from which the score could be computed).\n \"\"\"\n # If no stop criterion is defined, we can never stop training early\n if self.params.stop_fn is None:\n return False\n\n if score is None:\n if collect_stats is None:\n raise ValueError(\"Must provide collect_stats if score is not given\")\n\n # If no episodes were collected, we have no episode returns and thus cannot compute a score\n if collect_stats.n_collected_episodes == 0:\n return False\n\n score = self._compute_score_fn(collect_stats)\n\n return self.params.stop_fn(score)\n\n def _collect_test_episodes(\n self,\n ) -> CollectStats:\n assert self.params.test_collector is not None\n collector = self.params.test_collector\n collector.reset(reset_stats=False)\n if self.params.test_fn:\n self.params.test_fn(self._epoch, self._env_step)\n result = collector.collect(n_episode=self.params.test_step_num_episodes)\n if self.params.multi_agent_return_reduction:\n rew = self.params.multi_agent_return_reduction(result.returns)\n result.returns = rew\n result.returns_stat = SequenceSummaryStats.from_sequence(rew)\n if self._logger and self._env_step is not None:\n assert result.n_collected_episodes > 0\n self._logger.log_test_data(asdict(result), self._env_step)\n return result\n\n def _test_step(\n self, force_update_best: bool = False, log_msg_prefix: str | None = None\n ) -> tuple[CollectStats, bool]:\n \"\"\"Performs one test step.\n\n :param log_msg_prefix: a prefix to prepend to the log message, which is to establish the context within\n which the test step is being carried out\n :param force_update_best: whether to force updating of the best model stats (best score, reward, etc.)\n and call the `save_best_fn` callback\n \"\"\"\n assert self.params.test_step_num_episodes is not None\n assert self.params.test_collector is not None\n\n # collect test episodes\n test_stat = self._collect_test_episodes()\n assert test_stat.returns_stat is not None # for mypy\n\n # check whether we have a new best score and, if so, update stats and save the model\n # (or if forced)\n rew, rew_std = test_stat.returns_stat.mean, test_stat.returns_stat.std\n score = self._compute_score_fn(test_stat)\n if score > self._best_score or force_update_best:\n self._best_score = score\n self._best_epoch = self._epoch\n self._best_reward = float(rew)\n self._best_reward_std = rew_std\n if self.params.save_best_fn:\n self.params.save_best_fn(self.algorithm)\n\n # log results\n cur_info, best_info = \"\", \"\"\n if score != rew:\n cur_info, best_info = (\n f\", score: {score: .6f}\",\n f\", best_score: {self._best_score:.6f}\",\n )\n if log_msg_prefix is None:\n log_msg_prefix = f\"Epoch #{self._epoch}\"\n log_msg = (\n f\"{log_msg_prefix}: test_reward: {rew:.6f} ± {rew_std:.6f},{cur_info}\"\n f\" best_reward: {self._best_reward:.6f} ± \"\n f\"{self._best_reward_std:.6f}{best_info} in #{self._best_epoch}\"\n )\n log.info(log_msg)\n if self.params.verbose:\n print(log_msg, flush=True)\n\n # determine whether training shall be stopped early\n stop_fn_flag = self._should_stop_training_early(score=self._best_score)\n\n return test_stat, stop_fn_flag\n\n @abstractmethod\n def _training_step(self) -> _TrainingStepResult:\n \"\"\"Performs one training step.\"\"\"\n\n def _update_moving_avg_stats_and_log_update_data(self, update_stat: TrainingStats) -> None:\n \"\"\"Log losses, update moving average stats, and also modify the smoothed_loss in update_stat.\"\"\"\n cur_losses_dict = update_stat.get_loss_stats_dict()\n update_stat.smoothed_loss = self._update_moving_avg_stats_and_get_averaged_data(\n cur_losses_dict,\n )\n self._logger.log_update_data(asdict(update_stat), self._current_update_step)\n\n # TODO: seems convoluted, there should be a better way of dealing with the moving average stats\n def _update_moving_avg_stats_and_get_averaged_data(\n self,\n data: dict[str, float],\n ) -> dict[str, float]:\n \"\"\"Add entries to the moving average object in the trainer and retrieve the averaged results.\n\n :param data: any entries to be tracked in the moving average object.\n :return: A dictionary containing the averaged values of the tracked entries.\n\n \"\"\"\n smoothed_data = {}\n for key, loss_item in data.items():\n self._stat[key].add(loss_item)\n smoothed_data[key] = self._stat[key].get()\n return smoothed_data\n\n def run(\n self, reset_collectors: bool = True, reset_collector_buffers: bool = False\n ) -> InfoStats:\n \"\"\"Runs the training process with the configuration given at construction.\n\n :param reset_collectors: whether to reset the collectors prior to starting the training process.\n Specifically, this will reset the environments in the collectors (starting new episodes),\n and the statistics stored in the collector. Whether the contained buffers will be reset/cleared\n is determined by the `reset_buffer` parameter.\n :param reset_collector_buffers: whether, for the case where the collectors are reset, to reset/clear the\n contained buffers as well.\n This has no effect if `reset_collectors` is False.\n \"\"\"\n self.reset(\n reset_collectors=reset_collectors,\n reset_collector_buffers=reset_collector_buffers,\n )\n\n while self._epoch < self.params.max_epochs and not self._stop_fn_flag:\n self.execute_epoch()\n\n return self._create_info_stats()\n\n\nclass OfflineTrainer(Trainer[OfflineAlgorithm, OfflineTrainerParams]):\n \"\"\"An offline trainer, which samples mini-batches from a given buffer and passes them to\n the algorithm's update function.\n \"\"\"\n\n def __init__(\n self,\n algorithm: OfflineAlgorithm,\n params: OfflineTrainerParams,\n ):\n super().__init__(algorithm, params)\n self._buffer = algorithm.process_buffer(self.params.buffer)\n\n class _TrainingStepResult(Trainer._TrainingStepResult):\n def __init__(self, training_stats: TrainingStats, env_step_advancement: int):\n self._training_stats = training_stats\n self._env_step_advancement = env_step_advancement\n\n def get_steps_in_epoch_advancement(self) -> int:\n return 1\n\n def get_collect_stats(self) -> None:\n return None\n\n def get_training_stats(self) -> TrainingStats:\n return self._training_stats\n\n def is_training_done(self) -> bool:\n return False\n\n def get_env_step_advancement(self) -> int:\n return self._env_step_advancement\n\n def _training_step(self) -> _TrainingStepResult:\n with policy_within_training_step(self.algorithm.policy):\n # Note: since sample_size=batch_size, this will perform\n # exactly one gradient step. This is why we don't need to calculate the\n # number of gradient steps, like in the on-policy case.\n training_stats = self.algorithm.update(\n sample_size=self.params.batch_size, buffer=self._buffer\n )\n self._update_moving_avg_stats_and_log_update_data(training_stats)\n self._policy_update_time += training_stats.train_time\n return self._TrainingStepResult(\n training_stats=training_stats,\n env_step_advancement=self.params.batch_size,\n )\n\n def _create_epoch_pbar_data_dict(\n self, training_step_result: Trainer._TrainingStepResult\n ) -> dict[str, str]:\n return {}\n\n\nclass OnlineTrainer(\n Trainer[TAlgorithm, TOnlineTrainerParams],\n Generic[TAlgorithm, TOnlineTrainerParams],\n ABC,\n):\n \"\"\"\n An online trainer, which collects data from the environment in each training step and\n uses the collected data to perform an update step, the nature of which is to be defined\n in subclasses.\n \"\"\"\n\n def __init__(\n self,\n algorithm: TAlgorithm,\n params: TOnlineTrainerParams,\n ):\n super().__init__(algorithm, params)\n self._env_episode = 0\n \"\"\"\n the total number of episodes collected in the environment\n \"\"\"\n\n def _reset_collectors(self, reset_buffer: bool = False) -> None:\n super()._reset_collectors(reset_buffer=reset_buffer)\n self.params.training_collector.reset(reset_buffer=reset_buffer)\n\n def reset(self, reset_collectors: bool = True, reset_collector_buffers: bool = False) -> None:\n super().reset(\n reset_collectors=reset_collectors,\n reset_collector_buffers=reset_collector_buffers,\n )\n\n if (\n self.params.test_in_training\n and self.params.training_collector.policy is not self.algorithm.policy\n ):\n log.warning(\n \"The training data collector's policy is not the same as the one being trained, \"\n \"yet test_in_training is enabled. This may lead to unexpected results.\"\n )\n\n self._env_episode = 0\n\n class _TrainingStepResult(Trainer._TrainingStepResult):\n def __init__(\n self,\n collect_stats: CollectStats,\n training_stats: TrainingStats | None,\n is_training_done: bool,\n ):\n self._collect_stats = collect_stats\n self._training_stats = training_stats\n self._is_training_done = is_training_done\n\n def get_steps_in_epoch_advancement(self) -> int:\n return self.get_env_step_advancement()\n\n def get_collect_stats(self) -> CollectStats:\n return self._collect_stats\n\n def get_training_stats(self) -> TrainingStats | None:\n return self._training_stats\n\n def is_training_done(self) -> bool:\n return self._is_training_done\n\n def get_env_step_advancement(self) -> int:\n return self._collect_stats.n_collected_steps\n\n def _training_step(self) -> _TrainingStepResult:\n \"\"\"Perform one training step.\n\n For an online algorithm, a training step involves:\n * collecting data\n * for the case where `test_in_train` is activated,\n determining whether the stop condition has been reached\n (and returning without performing any actual training if so)\n * performing a gradient update step\n \"\"\"\n with policy_within_training_step(self.algorithm.policy):\n # collect data\n collect_stats = self._collect_training_data()\n\n # determine whether we should stop training based on the data collected\n should_stop_training = False\n if self.params.test_in_training:\n should_stop_training = self._test_in_train(collect_stats)\n\n # perform gradient update step (if not already done)\n training_stats: TrainingStats | None = None\n if not should_stop_training:\n training_stats = self._update_step(collect_stats)\n\n return self._TrainingStepResult(\n collect_stats=collect_stats,\n training_stats=training_stats,\n is_training_done=should_stop_training,\n )\n\n def _collect_training_data(self) -> CollectStats:\n \"\"\"Performs training data collection.\n\n :return: the data collection stats\n \"\"\"\n assert self.params.test_step_num_episodes is not None\n assert self.params.training_collector is not None\n\n if self.params.training_fn:\n self.params.training_fn(self._epoch, self._env_step)\n\n collect_stats = self.params.training_collector.collect(\n n_step=self.params.collection_step_num_env_steps,\n n_episode=self.params.collection_step_num_episodes,\n )\n TraceLogger.log(\n log,\n lambda: f\"Collected {collect_stats.n_collected_steps} steps, {collect_stats.n_collected_episodes} episodes\",\n )\n\n if self.params.training_collector.buffer.hasnull():\n from tianshou.data.collector import EpisodeRolloutHook\n from tianshou.env import DummyVectorEnv\n\n raise MalformedBufferError(\n f\"Encountered NaNs in buffer after {self._env_step} steps.\"\n f\"Such errors are usually caused by either a bug in the environment or by \"\n f\"problematic implementations {EpisodeRolloutHook.__class__.__name__}. \"\n f\"For debugging such issues it is recommended to run the training in a single process, \"\n f\"e.g., by using {DummyVectorEnv.__class__.__name__}.\",\n )\n\n if collect_stats.n_collected_episodes > 0:\n assert collect_stats.returns_stat is not None # for mypy\n assert collect_stats.lens_stat is not None # for mypy\n if self.params.multi_agent_return_reduction:\n rew = self.params.multi_agent_return_reduction(collect_stats.returns)\n collect_stats.returns = rew\n collect_stats.returns_stat = SequenceSummaryStats.from_sequence(rew)\n\n # update collection stats specific to this specialization\n self._env_episode += collect_stats.n_collected_episodes\n\n return collect_stats\n\n def _test_in_train(\n self,\n train_collect_stats: CollectStats,\n ) -> bool:\n \"\"\"\n Performs a test step if the data collected in the current training step suggests that performance\n is good enough to stop training early. If the test step confirms that performance is indeed good\n enough, returns True, and False otherwise.\n\n Specifically, applies the early stopping criterion to the data collected in the current training step,\n and if the criterion is satisfied, performs a test step which returns the relevant result.\n\n :param train_collect_stats: the data collection stats from the preceding collection step\n :return: flag indicating whether to stop training early\n \"\"\"\n should_stop_training = False\n\n # check whether the stop criterion is satisfied based on the data collected in the training step\n # (if any full episodes were indeed collected)\n if train_collect_stats.n_collected_episodes > 0 and self._should_stop_training_early(\n collect_stats=train_collect_stats\n ):\n # apply a test step, temporarily switching out of \"is_training_step\" semantics such that the policy can\n # be evaluated, in order to determine whether we should stop training\n with policy_within_training_step(self.algorithm.policy, enabled=False):\n _, should_stop_training = self._test_step(\n log_msg_prefix=f\"Test step triggered by train stats (env_step={self._env_step})\"\n )\n\n return should_stop_training\n\n @abstractmethod\n def _update_step(\n self,\n collect_stats: CollectStatsBase,\n ) -> TrainingStats:\n \"\"\"Performs a gradient update step, calling the algorithm's update method accordingly.\n\n :param collect_stats: provides info about the preceding data collection step.\n \"\"\"\n\n def _create_epoch_pbar_data_dict(\n self, training_step_result: Trainer._TrainingStepResult\n ) -> dict[str, str]:\n collect_stats = training_step_result.get_collect_stats()\n assert collect_stats is not None\n result = {\n \"env_step\": str(self._env_step),\n \"env_episode\": str(self._env_episode),\n \"n_ep\": str(collect_stats.n_collected_episodes),\n \"n_st\": str(collect_stats.n_collected_steps),\n }\n # return and episode length info is only available if at least one episode was completed\n if collect_stats.n_collected_episodes > 0:\n assert collect_stats.returns_stat is not None\n assert collect_stats.lens_stat is not None\n result.update(\n {\n \"rew\": f\"{collect_stats.returns_stat.mean:.2f}\",\n \"len\": str(int(collect_stats.lens_stat.mean)),\n }\n )\n return result\n\n\nclass OffPolicyTrainer(OnlineTrainer[OffPolicyAlgorithm, OffPolicyTrainerParams]):\n \"\"\"An off-policy trainer, which samples mini-batches from the buffer of collected data and passes them to\n algorithm's `update` function.\n\n The algorithm's `update` method is expected to not perform additional mini-batching but just update\n model parameters from the received mini-batch.\n \"\"\"\n\n def _update_step(\n self,\n collect_stats: CollectStatsBase,\n ) -> TrainingStats:\n \"\"\"Perform `update_step_num_gradient_steps_per_sample * n_collected_steps` gradient steps by sampling\n mini-batches from the buffer.\n\n :param collect_stats: the :class:`~TrainingStats` instance returned by the last gradient step. Some values\n in it will be replaced by their moving averages.\n \"\"\"\n assert self.params.training_collector is not None\n n_collected_steps = collect_stats.n_collected_steps\n n_gradient_steps = round(\n self.params.update_step_num_gradient_steps_per_sample * n_collected_steps\n )\n if n_gradient_steps == 0:\n raise ValueError(\n f\"n_gradient_steps is 0, n_collected_steps={n_collected_steps}, \"\n f\"update_step_num_gradient_steps_per_sample={self.params.update_step_num_gradient_steps_per_sample}\",\n )\n\n update_stat = None\n disable_pbar = n_gradient_steps < 20 # only show progress bar if there are many steps\n for _ in self._pbar(\n range(n_gradient_steps),\n desc=\"Offpolicy gradient update\",\n position=0,\n leave=False,\n disable=disable_pbar,\n ):\n update_stat = self._sample_and_update(self.params.training_collector.buffer)\n self._policy_update_time += update_stat.train_time\n\n # TODO: only the last update_stat is returned, should be improved\n assert update_stat is not None\n return update_stat\n\n def _sample_and_update(self, buffer: ReplayBuffer) -> TrainingStats:\n \"\"\"Sample a mini-batch, perform one gradient step, and update the _gradient_step counter.\"\"\"\n # Note: since sample_size=batch_size, this will perform\n # exactly one gradient step. This is why we don't need to calculate the\n # number of gradient steps, like in the on-policy case.\n update_stat = self.algorithm.update(sample_size=self.params.batch_size, buffer=buffer)\n self._update_moving_avg_stats_and_log_update_data(update_stat)\n return update_stat\n\n\nclass OnPolicyTrainer(OnlineTrainer[OnPolicyAlgorithm, OnPolicyTrainerParams]):\n \"\"\"An on-policy trainer, which passes the entire buffer to the algorithm's `update` methods and\n resets the buffer thereafter.\n\n Note that it is expected that the update method of the algorithm will perform\n batching when using this trainer.\n \"\"\"\n\n def _update_step(\n self,\n collect_stats: CollectStatsBase | None = None,\n ) -> TrainingStats:\n \"\"\"Perform one on-policy update by passing the entire buffer to the algorithm's update method.\"\"\"\n assert self.params.training_collector is not None\n log.debug(\n f\"Performing on-policy update on buffer of length {len(self.params.training_collector.buffer)}\",\n )\n training_stat = self.algorithm.update(\n buffer=self.params.training_collector.buffer,\n batch_size=self.params.batch_size,\n repeat=self.params.update_step_num_repetitions,\n )\n\n # just for logging, no functional role\n self._policy_update_time += training_stat.train_time\n\n # Note 2: in the policy-update we modify the buffer, which is not very clean.\n # currently the modification will erase previous samples but keep things like\n # _ep_rew and _ep_len (b/c keep_statistics=True). This is needed since the collection might have stopped\n # in the middle of an episode and in the next collect iteration we need these numbers to compute correct\n # return and episode length values. With the current code structure, this means that after an update and buffer reset\n # such quantities can no longer be computed\n # from samples still contained in the buffer, which is also not clean\n self.params.training_collector.reset_buffer(keep_statistics=True)\n\n # The step is the number of mini-batches used for the update, so essentially\n self._update_moving_avg_stats_and_log_update_data(training_stat)\n\n return training_stat\n" }, { "path": "tianshou/utils/__init__.py", "content": "\"\"\"Utils package.\"\"\"\n\nfrom tianshou.utils.logger.logger_base import BaseLogger, LazyLogger\nfrom tianshou.utils.logger.tensorboard import TensorboardLogger\nfrom tianshou.utils.logger.wandb import WandbLogger\nfrom tianshou.utils.progress_bar import DummyTqdm, tqdm_config\nfrom tianshou.utils.statistics import MovAvg, RunningMeanStd\nfrom tianshou.utils.warning import deprecation\n\n__all__ = [\n \"BaseLogger\",\n \"DummyTqdm\",\n \"LazyLogger\",\n \"MovAvg\",\n \"RunningMeanStd\",\n \"TensorboardLogger\",\n \"WandbLogger\",\n \"deprecation\",\n \"tqdm_config\",\n]\n" }, { "path": "tianshou/utils/conversion.py", "content": "from typing import overload\n\nimport torch\n\n\n@overload\ndef to_optional_float(x: torch.Tensor) -> float: ...\n\n\n@overload\ndef to_optional_float(x: float) -> float: ...\n\n\n@overload\ndef to_optional_float(x: None) -> None: ...\n\n\ndef to_optional_float(x: torch.Tensor | float | None) -> float | None:\n \"\"\"For the common case where one needs to extract a float from a scalar Tensor, which may be None.\"\"\"\n if isinstance(x, torch.Tensor):\n return x.item()\n return x\n" }, { "path": "tianshou/utils/determinism.py", "content": "import difflib\nimport inspect\nimport os\nimport re\nimport time\nfrom collections.abc import Callable, Sequence\nfrom dataclasses import dataclass\nfrom io import StringIO\nfrom pathlib import Path\nfrom typing import Self\n\nimport torch\nfrom sensai.util import logging\nfrom sensai.util.git import GitStatus, git_status\nfrom sensai.util.pickle import dump_pickle, load_pickle\n\n\ndef format_log_message(\n logger: logging.Logger,\n level: int,\n msg: str,\n formatter: logging.Formatter,\n stacklevel: int = 1,\n) -> str:\n \"\"\"\n Formats a log message as it would have been created by `logger.log(level, msg)` with the given formatter.\n\n :param logger: the logger\n :param level: the log level\n :param msg: the message\n :param formatter: the formatter\n :param stacklevel: the stack level of the function to report as the generator\n :return: the formatted log message (not including trailing newline)\n \"\"\"\n frame_info = inspect.stack()[stacklevel]\n pathname = frame_info.filename\n lineno = frame_info.lineno\n func = frame_info.function\n\n record = logger.makeRecord(\n name=logger.name,\n level=level,\n fn=pathname,\n lno=lineno,\n msg=msg,\n args=(),\n exc_info=None,\n func=func,\n extra=None,\n )\n record.created = time.time()\n record.asctime = time.strftime(\"%Y-%m-%d %H:%M:%S\", time.localtime(record.created))\n\n return formatter.format(record)\n\n\nclass TraceLogger:\n \"\"\"Supports the collection of behavioural trace logs, which can, in particular, be used for determinism tests.\"\"\"\n\n is_enabled = False\n \"\"\"\n whether the trace logger is enabled.\n\n NOTE: The preferred way to enable this is via the context manager.\n \"\"\"\n verbose = False\n \"\"\"\n whether to print trace log messages to stdout.\n \"\"\"\n MESSAGE_TAG = \"[TRACE]\"\n \"\"\"\n a tag which is added at the beginning of log messages generated by this logger\n \"\"\"\n LOG_LEVEL = logging.DEBUG\n log_buffer: StringIO | None = None\n log_formatter: logging.Formatter | None = None\n\n @classmethod\n def log(cls, logger: logging.Logger, message_generator: Callable[[], str]) -> None:\n \"\"\"\n Logs a message intended for tracing agent-env interaction, which is enabled via\n `TraceAgentEnvLoggerContext`.\n\n :param logger: the logger to use for the actual logging\n :param message_generator: function which generates the log message (which may be expensive);\n if logging is disabled, the function will not be called.\n \"\"\"\n if not cls.is_enabled:\n return\n\n msg = message_generator()\n msg = cls.MESSAGE_TAG + \" \" + msg\n\n # Log with caller's frame info\n logger.log(logging.DEBUG, msg, stacklevel=2)\n\n # If a dedicated memory buffer is configured, also store the message there\n if cls.log_buffer is not None:\n msg_formatted = format_log_message(\n logger,\n logging.DEBUG,\n msg,\n cls.log_formatter,\n stacklevel=2,\n )\n cls.log_buffer.write(msg_formatted + \"\\n\")\n if cls.verbose:\n print(msg_formatted)\n\n\n@dataclass\nclass TraceLog:\n log_lines: list[str]\n\n def save_log(self, path: str) -> None:\n with open(path, \"w\") as f:\n for line in self.log_lines:\n f.write(line + \"\\n\")\n\n def print_log(self) -> None:\n for line in self.log_lines:\n print(line)\n\n def get_full_log(self) -> str:\n return \"\\n\".join(self.log_lines)\n\n def reduce_log_to_messages(self) -> \"TraceLog\":\n \"\"\"\n Removes logger names and function names from the log entries, such that each log message\n contains only the main text message itself (starting with the content after the logger's tag).\n\n :return: the result with reduced log messages\n \"\"\"\n lines = []\n tag = re.escape(TraceLogger.MESSAGE_TAG)\n for line in self.log_lines:\n lines.append(re.sub(r\".*\" + tag, \"\", line))\n return TraceLog(lines)\n\n def filter_messages(\n self,\n required_messages: Sequence[str] = (),\n optional_messages: Sequence[str] = (),\n ignored_messages: Sequence[str] = (),\n ) -> \"TraceLog\":\n \"\"\"\n Applies inclusion and or exclusion filtering to the log messages.\n If either `required_messages` or `optional_messages` is empty, inclusion filtering is applied.\n If `ignored_messages` is empty, exclusion filtering is applied.\n If both inclusion and exclusion filtering are applied, the exclusion filtering takes precedence.\n\n :param required_messages: required message substrings to filter for; each message is required to appear at least once\n (triggering exception otherwise)\n :param optional_messages: additional messages fragments to filter for; these are not required\n :param ignored_messages: message fragments that result in exclusion; takes precedence over\n `required_messages` and `optional_messages`\n :return: the result with reduced log messages\n \"\"\"\n import numpy as np\n\n required_message_counters = np.zeros(len(required_messages))\n\n def retain_line(line: str) -> bool:\n for ignored_message in ignored_messages:\n if ignored_message in line:\n return False\n if required_messages or optional_messages:\n for i, main_message in enumerate(required_messages):\n if main_message in line:\n required_message_counters[i] += 1\n return True\n return any(add_message in line for add_message in optional_messages)\n else:\n return True\n\n lines = []\n for line in self.log_lines:\n if retain_line(line):\n lines.append(line)\n\n assert np.all(\n required_message_counters > 0,\n ), \"Not all types of required messages were found in the trace. Were log messages changed?\"\n\n return TraceLog(lines)\n\n\nclass TraceLoggerContext:\n \"\"\"\n A context manager which enables the trace logger.\n Apart from enabling the logging, it can optionally create a memory log buffer, such that\n getting the trace log is not strictly dependent on the logging system.\n \"\"\"\n\n def __init__(\n self,\n enable_log_buffer: bool = True,\n log_format: str = \"%(name)s:%(funcName)s - %(message)s\",\n ) -> None:\n \"\"\"\n :param enable_log_buffer: whether to enable the dedicated log buffer for trace logs, whose contents\n can, within the context of this manager, be accessed via method `get_log`.\n :param log_format: the logger format string to use for the dedicated log buffer\n \"\"\"\n self._enable_log_buffer = enable_log_buffer\n self._log_format: str = log_format\n self._log_buffer: StringIO | None = None\n\n def __enter__(self) -> Self:\n TraceLogger.is_enabled = True\n\n if self._enable_log_buffer:\n TraceLogger.log_buffer = StringIO()\n TraceLogger.log_formatter = logging.Formatter(self._log_format)\n self._log_buffer = TraceLogger.log_buffer\n\n return self\n\n def __exit__(self, exc_type, exc_val, exc_tb): # type: ignore\n TraceLogger.is_enabled = False\n TraceLogger.log_buffer = None\n TraceLogger.log_formatter = None\n\n def get_log(self) -> TraceLog:\n \"\"\":return: the full trace log that was captured if `enable_log_buffer` was enabled at construction\"\"\"\n if self._log_buffer is None:\n raise Exception(\n \"This method is only supported if the log buffer is enabled at construction\",\n )\n return TraceLog(log_lines=self._log_buffer.getvalue().split(\"\\n\"))\n\n\ndef torch_param_hash(module: torch.nn.Module) -> str:\n \"\"\"\n Computes a hash of the parameters of the given module; parameters not requiring gradients are ignored.\n\n :param module: a torch module\n :return: a hex digest of the parameters of the module\n \"\"\"\n import hashlib\n\n hasher = hashlib.sha1()\n for param in module.parameters():\n if param.requires_grad:\n np_array = param.detach().cpu().numpy()\n hasher.update(np_array.tobytes())\n return hasher.hexdigest()\n\n\nclass TraceDeterminismTest:\n def __init__(\n self,\n base_path: Path,\n core_messages: Sequence[str] = (),\n ignored_messages: Sequence[str] = (),\n log_filename: str | None = None,\n ) -> None:\n \"\"\"\n :param base_path: the directory where the reference results are stored (will be created if necessary)\n :param core_messages: message fragments that make up the core of a trace; if empty, all messages are considered core\n :param ignored_messages: message fragments to ignore in the trace log (if any); takes precedence over\n `core_messages`\n :param log_filename: the name of the log file to which results are to be written (if any)\n \"\"\"\n base_path.mkdir(parents=True, exist_ok=True)\n self.base_path = base_path\n self.core_messages = core_messages\n self.ignored_messages = ignored_messages\n self.log_filename = log_filename\n\n @dataclass(kw_only=True)\n class Result:\n git_status: GitStatus\n log: TraceLog\n\n def check(\n self,\n current_log: TraceLog,\n name: str,\n create_reference_result: bool = False,\n pass_if_core_messages_unchanged: bool = False,\n ) -> None:\n \"\"\"\n Checks the given log against the reference result for the given name.\n\n :param current_log: the result to check\n :param name: the name of the reference result; must be unique among all tests!\n :param create_reference_result: whether update the reference result with the given result\n \"\"\"\n import pytest\n\n reference_result_path = self.base_path / f\"{name}.pkl.bz2\"\n current_git_status = git_status()\n\n if create_reference_result:\n current_result = self.Result(git_status=current_git_status, log=current_log)\n dump_pickle(current_result, reference_result_path)\n\n reference_result: TraceDeterminismTest.Result = load_pickle(\n reference_result_path,\n )\n reference_log = reference_result.log\n\n current_log_reduced = current_log.reduce_log_to_messages().filter_messages(\n ignored_messages=self.ignored_messages,\n )\n reference_log_reduced = reference_log.reduce_log_to_messages().filter_messages(\n ignored_messages=self.ignored_messages,\n )\n\n results: list[tuple[TraceLog, str]] = [\n (reference_log_reduced, \"expected\"),\n (current_log_reduced, \"current\"),\n (reference_log, \"expected_full\"),\n (current_log, \"current_full\"),\n ]\n\n if self.core_messages:\n result_main_messages = current_log_reduced.filter_messages(\n required_messages=self.core_messages,\n )\n reference_result_main_messages = reference_log_reduced.filter_messages(\n required_messages=self.core_messages,\n )\n results.extend(\n [\n (reference_result_main_messages, \"expected_core\"),\n (result_main_messages, \"current_core\"),\n ],\n )\n else:\n result_main_messages = current_log_reduced\n reference_result_main_messages = reference_log_reduced\n\n logs_equivalent = current_log_reduced.get_full_log() == reference_log_reduced.get_full_log()\n if logs_equivalent:\n status_passed = True\n status_message = \"OK\"\n else:\n core_messages_unchanged = (\n len(self.core_messages) > 0\n and result_main_messages.get_full_log()\n == reference_result_main_messages.get_full_log()\n )\n status_passed = core_messages_unchanged and pass_if_core_messages_unchanged\n\n if status_passed:\n status_message = \"OK (core messages unchanged)\"\n else:\n # save files for comparison\n files = []\n for r, suffix in results:\n path = os.path.abspath(f\"determinism_{name}_{suffix}.txt\")\n r.save_log(path)\n files.append(path)\n\n paths_str = \"\\n\".join(files)\n main_message = (\n f\"Please inspect the changes by diffing the log files:\\n{paths_str}\\n\"\n f\"If the changes are OK, enable the `create_reference_result` flag temporarily, \"\n \"rerun the test and then commit the updated reference file.\\n\\nHere's the first part of the diff:\\n\"\n )\n\n # compute diff and add to message\n num_diff_lines_to_show = 30\n for i, line in enumerate(\n difflib.unified_diff(\n reference_log_reduced.log_lines,\n current_log_reduced.log_lines,\n fromfile=\"expected.txt\",\n tofile=\"current.txt\",\n lineterm=\"\",\n ),\n ):\n if i == num_diff_lines_to_show:\n break\n main_message += line + \"\\n\"\n\n if core_messages_unchanged:\n status_message = (\n \"The behaviour log has changed, but the core messages are still the same (so this \"\n f\"probably isn't an issue). {main_message}\"\n )\n else:\n status_message = f\"The behaviour log has changed; even the core messages are different. {main_message}\"\n\n # write log message\n if self.log_filename:\n with open(self.log_filename, \"a\") as f:\n hr = \"-\" * 100\n f.write(f\"\\n\\n{hr}\\nName: {name}\\n\")\n f.write(f\"Reference state: {reference_result.git_status}\\n\")\n f.write(f\"Current state: {current_git_status}\\n\")\n f.write(f\"Test result: {status_message}\\n\")\n\n if not status_passed:\n pytest.fail(status_message)\n" }, { "path": "tianshou/utils/lagged_network.py", "content": "from copy import deepcopy\r\nfrom dataclasses import dataclass\r\nfrom typing import Self\r\n\r\nimport torch\r\n\r\n\r\ndef polyak_parameter_update(tgt: torch.nn.Module, src: torch.nn.Module, tau: float) -> None:\r\n \"\"\"Softly updates the parameters of a target network `tgt` with the parameters of a source network `src`\r\n using Polyak averaging: `tau * src + (1 - tau) * tgt`.\r\n\r\n :param tgt: the target network that receives the parameter update\r\n :param src: the source network whose parameters are used for the update\r\n :param tau: the fraction with which to use the source network's parameters, the inverse `1-tau` being\r\n the fraction with which to retain the target network's parameters.\r\n \"\"\"\r\n for tgt_param, src_param in zip(tgt.parameters(), src.parameters(), strict=True):\r\n tgt_param.data.copy_(tau * src_param.data + (1 - tau) * tgt_param.data)\r\n\r\n\r\nclass EvalModeModuleWrapper(torch.nn.Module):\r\n \"\"\"\r\n A wrapper around a torch.nn.Module that forces the module to eval mode.\r\n\r\n The wrapped module supports only the forward method, attribute access is not supported.\r\n **NOTE**: It is *not* recommended to support attribute/method access beyond this via `__getattr__`,\r\n because torch.nn.Module already heavily relies on `__getattr__` to provides its own attribute access.\r\n Overriding it naively will cause problems!\r\n But it's also not necessary for our use cases; forward is enough.\r\n \"\"\"\r\n\r\n def __init__(self, m: torch.nn.Module):\r\n super().__init__()\r\n m.eval()\r\n self.module = m\r\n\r\n def forward(self, *args, **kwargs): # type: ignore\r\n self.module.eval()\r\n return self.module(*args, **kwargs)\r\n\r\n def train(self, mode: bool = True) -> Self:\r\n super().train(mode=mode)\r\n self.module.eval() # force eval mode\r\n return self\r\n\r\n\r\n@dataclass\r\nclass LaggedNetworkPair:\r\n target: torch.nn.Module\r\n source: torch.nn.Module\r\n\r\n\r\nclass LaggedNetworkCollection:\r\n def __init__(self) -> None:\r\n self._lagged_network_pairs: list[LaggedNetworkPair] = []\r\n\r\n def add_lagged_network(self, source: torch.nn.Module) -> EvalModeModuleWrapper:\r\n \"\"\"\r\n Adds a lagged network to the collection, returning the target network, which\r\n is forced to eval mode. The target network is a copy of the source network,\r\n which, however, supports only the forward method (hence the type torch.nn.Module);\r\n attribute access is not supported.\r\n\r\n :param source: the source network whose parameters are to be copied to the target network\r\n :return: the target network, which supports only the forward method and is forced to eval mode\r\n \"\"\"\r\n target = deepcopy(source)\r\n self._lagged_network_pairs.append(LaggedNetworkPair(target, source))\r\n return EvalModeModuleWrapper(target)\r\n\r\n def polyak_parameter_update(self, tau: float) -> None:\r\n \"\"\"Softly updates the parameters of each target network `tgt` with the parameters of a source network `src`\r\n using Polyak averaging: `tau * src + (1 - tau) * tgt`.\r\n\r\n :param tau: the fraction with which to use the source network's parameters, the inverse `1-tau` being\r\n the fraction with which to retain the target network's parameters.\r\n \"\"\"\r\n for pair in self._lagged_network_pairs:\r\n polyak_parameter_update(pair.target, pair.source, tau)\r\n\r\n def full_parameter_update(self) -> None:\r\n \"\"\"Fully updates the target networks with the source networks' parameters (exact copy).\"\"\"\r\n for pair in self._lagged_network_pairs:\r\n for tgt_param, src_param in zip(\r\n pair.target.parameters(), pair.source.parameters(), strict=True\r\n ):\r\n tgt_param.data.copy_(src_param.data)\r\n" }, { "path": "tianshou/utils/logger/__init__.py", "content": "" }, { "path": "tianshou/utils/logger/logger_base.py", "content": "import typing\nfrom abc import ABC, abstractmethod\nfrom collections.abc import Callable\nfrom enum import StrEnum\nfrom numbers import Number\n\nimport numpy as np\n\nVALID_LOG_VALS_TYPE = int | Number | np.number | np.ndarray | float\n# It's unfortunate, but we can't use Union type in isinstance, hence we resort to this\nVALID_LOG_VALS = typing.get_args(VALID_LOG_VALS_TYPE)\n\nTRestoredData = dict[str, np.ndarray | dict[str, \"TRestoredData\"]]\n\n\nclass DataScope(StrEnum):\n TRAINING = \"training\"\n TEST = \"test\"\n UPDATE = \"update\"\n INFO = \"info\"\n\n\nclass BaseLogger(ABC):\n \"\"\"The base class for any logger which is compatible with trainer.\n\n :param training_interval: the interval size (in env steps) after which log_training_data() will be called.\n :param test_interval: the interval size (in env steps) after which log_test_data() will be called.\n :param update_interval: the interval size (in env steps) after which log_update_data() will be called.\n :param info_interval: the interval size (in env steps) after which the method log_info() will be called.\n :param save_interval: the interval size (in env steps) after which the checkpoint and end\n of epoch related logs will be saved.\n \"\"\"\n\n def __init__(\n self,\n training_interval: int = 1000,\n test_interval: int = 1,\n update_interval: int = 1000,\n info_interval: int = 1,\n save_interval: int | None = None,\n exclude_arrays: bool = True,\n ) -> None:\n super().__init__()\n self.training_interval = training_interval\n self.test_interval = test_interval\n self.update_interval = update_interval\n self.info_interval = info_interval\n self.save_interval = save_interval\n self.exclude_arrays = exclude_arrays\n self.last_log_training_step = -1\n self.last_log_test_step = -1\n self.last_log_update_step = -1\n self.last_log_info_step = -1\n\n @abstractmethod\n def write(self, step_type: str, step: int, data: dict[str, VALID_LOG_VALS_TYPE]) -> None:\n \"\"\"Specify how the writer is used to log data.\n\n :param str step_type: namespace which the data dict belongs to.\n :param step: stands for the ordinate of the data dict.\n :param data: the data to write with format ``{key: value}``.\n \"\"\"\n\n @abstractmethod\n def prepare_dict_for_logging(self, log_data: dict) -> dict[str, VALID_LOG_VALS_TYPE]:\n \"\"\"Prepare the dict for logging by filtering out invalid data types.\n\n If necessary, reformulate the dict to be compatible with the writer.\n\n :param log_data: the dict to be prepared for logging.\n :return: the prepared dict.\n \"\"\"\n\n @abstractmethod\n def finalize(self) -> None:\n \"\"\"Finalize the logger, e.g., close writers and connections.\"\"\"\n\n def log_training_data(self, log_data: dict, step: int) -> None:\n \"\"\"Use writer to log statistics generated during training.\n\n :param log_data: a dict containing the information returned by the collector during the train step.\n :param step: stands for the timestep the collector result is logged.\n \"\"\"\n # TODO: move interval check to calling method\n if step - self.last_log_training_step >= self.training_interval:\n log_data = self.prepare_dict_for_logging(log_data)\n self.write(f\"{DataScope.TRAINING}/env_step\", step, log_data)\n self.last_log_training_step = step\n\n def log_test_data(self, log_data: dict, step: int) -> None:\n \"\"\"Use writer to log statistics generated during evaluating.\n\n :param log_data:a dict containing the information returned by the collector during the evaluation step.\n :param step: stands for the timestep the collector result is logged.\n \"\"\"\n # TODO: move interval check to calling method (stupid because log_test_data is only called from function in utils.py, not from BaseTrainer)\n if step - self.last_log_test_step >= self.test_interval:\n log_data = self.prepare_dict_for_logging(log_data)\n self.write(f\"{DataScope.TEST}/env_step\", step, log_data)\n self.last_log_test_step = step\n\n def log_update_data(self, log_data: dict, step: int) -> None:\n \"\"\"Use writer to log statistics generated during updating.\n\n :param log_data:a dict containing the information returned during the policy update step.\n :param step: stands for the timestep the policy training data is logged.\n \"\"\"\n # TODO: move interval check to calling method\n if step - self.last_log_update_step >= self.update_interval:\n log_data = self.prepare_dict_for_logging(log_data)\n self.write(f\"{DataScope.UPDATE}/update_step\", step, log_data)\n self.last_log_update_step = step\n\n def log_info_data(self, log_data: dict, step: int) -> None:\n \"\"\"Use writer to log global statistics.\n\n :param log_data: a dict containing information of data collected at the end of an epoch.\n :param step: stands for the timestep the training info is logged.\n \"\"\"\n if (\n step - self.last_log_info_step >= self.info_interval\n ): # TODO: move interval check to calling method\n log_data = self.prepare_dict_for_logging(log_data)\n self.write(f\"{DataScope.INFO}/epoch\", step, log_data)\n self.last_log_info_step = step\n\n @abstractmethod\n def save_data(\n self,\n epoch: int,\n env_step: int,\n update_step: int,\n save_checkpoint_fn: Callable[[int, int, int], str] | None = None,\n ) -> None:\n \"\"\"Use writer to log metadata when calling ``save_checkpoint_fn`` in trainer.\n\n :param epoch: the epoch in trainer.\n :param env_step: the env_step in trainer.\n :param update_step: the update step count in the trainer.\n :param function save_checkpoint_fn: a hook defined by user, see trainer\n documentation for detail.\n \"\"\"\n\n @abstractmethod\n def restore_data(self) -> tuple[int, int, int]:\n \"\"\"Restore internal data if present and return the metadata from existing log for continuation of training.\n\n If it finds nothing or an error occurs during the recover process, it will\n return the default parameters.\n\n :return: epoch, env_step, update_step.\n \"\"\"\n\n @staticmethod\n @abstractmethod\n def restore_logged_data(\n log_path: str,\n ) -> TRestoredData:\n \"\"\"Load the logged data from disk for post-processing.\n\n :return: a dict containing the logged data.\n \"\"\"\n\n\nclass LazyLogger(BaseLogger):\n \"\"\"A logger that does nothing. Used as the placeholder in trainer.\"\"\"\n\n def __init__(self) -> None:\n super().__init__()\n\n def prepare_dict_for_logging(\n self,\n data: dict[str, VALID_LOG_VALS_TYPE],\n ) -> dict[str, VALID_LOG_VALS_TYPE]:\n return data\n\n def write(self, step_type: str, step: int, data: dict[str, VALID_LOG_VALS_TYPE]) -> None:\n \"\"\"The LazyLogger writes nothing.\"\"\"\n\n def finalize(self) -> None:\n pass\n\n def save_data(\n self,\n epoch: int,\n env_step: int,\n update_step: int,\n save_checkpoint_fn: Callable[[int, int, int], str] | None = None,\n ) -> None:\n pass\n\n def restore_data(self) -> tuple[int, int, int]:\n return 0, 0, 0\n\n @staticmethod\n def restore_logged_data(log_path: str) -> dict:\n return {}\n" }, { "path": "tianshou/utils/logger/tensorboard.py", "content": "from collections.abc import Callable\nfrom typing import Any\n\nimport numpy as np\nfrom matplotlib.figure import Figure\nfrom tensorboard.backend.event_processing import event_accumulator\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom tianshou.utils.logger.logger_base import (\n VALID_LOG_VALS,\n VALID_LOG_VALS_TYPE,\n BaseLogger,\n TRestoredData,\n)\n\n\nclass TensorboardLogger(BaseLogger):\n \"\"\"A logger that relies on tensorboard SummaryWriter by default to visualize and log statistics.\n\n :param SummaryWriter writer: the writer to log data.\n :param training_interval: the interval size (in env steps) after which log_training_data() will be called.\n :param test_interval: the interval size (in env steps) after which log_test_data() will be called.\n :param update_interval: the interval size (in env steps) after which log_update_data() will be called.\n :param info_interval: the interval size (in env steps) after which the method log_info() will be called.\n :param save_interval: the interval size (in env steps) after which the checkpoint and end of epoch related logs will be saved.\n :param write_flush: whether to flush tensorboard result after each\n add_scalar operation. Default to True.\n \"\"\"\n\n def __init__(\n self,\n writer: SummaryWriter,\n training_interval: int = 1000,\n test_interval: int = 1,\n update_interval: int = 1000,\n info_interval: int = 1,\n save_interval: int | None = None,\n write_flush: bool = True,\n ) -> None:\n super().__init__(\n training_interval, test_interval, update_interval, info_interval, save_interval\n )\n self.write_flush = write_flush\n self.last_save_step = -1\n self.writer = writer\n\n def prepare_dict_for_logging(\n self,\n input_dict: dict[str, Any],\n parent_key: str = \"\",\n delimiter: str = \"/\",\n exclude_arrays: bool = True,\n ) -> dict[str, VALID_LOG_VALS_TYPE]:\n \"\"\"Flattens and filters a nested dictionary by recursively traversing all levels and compressing the keys.\n\n Filtering is performed with respect to valid logging data types.\n\n :param input_dict: The nested dictionary to be flattened and filtered.\n :param parent_key: The parent key used as a prefix before the input_dict keys.\n :param delimiter: The delimiter used to separate the keys.\n :param exclude_arrays: Whether to exclude numpy arrays from the output.\n :return: A flattened dictionary where the keys are compressed and values are filtered.\n \"\"\"\n result = {}\n\n def add_to_result(\n cur_dict: dict,\n prefix: str = \"\",\n ) -> None:\n for key, value in cur_dict.items():\n if exclude_arrays and isinstance(value, np.ndarray):\n continue\n\n new_key = prefix + delimiter + str(key)\n new_key = new_key.lstrip(delimiter)\n\n if isinstance(value, dict):\n add_to_result(\n value,\n new_key,\n )\n elif isinstance(value, VALID_LOG_VALS):\n result[new_key] = value\n\n add_to_result(input_dict, prefix=parent_key)\n return result\n\n def write(self, step_type: str, step: int, data: dict[str, Any]) -> None:\n scope, step_name = step_type.split(\"/\")\n self.writer.add_scalar(step_type, step, global_step=step)\n for k, v in data.items():\n scope_key = f\"{scope}/{k}\"\n if isinstance(v, np.ndarray):\n self.writer.add_histogram(scope_key, v, global_step=step, bins=\"auto\")\n elif isinstance(v, Figure):\n self.writer.add_figure(scope_key, v, global_step=step)\n else:\n self.writer.add_scalar(scope_key, v, global_step=step)\n if self.write_flush: # issue 580\n self.writer.flush() # issue #482\n\n def finalize(self) -> None:\n self.writer.close()\n\n def save_data(\n self,\n epoch: int,\n env_step: int,\n update_step: int,\n save_checkpoint_fn: Callable[[int, int, int], str] | None = None,\n ) -> None:\n if (\n self.save_interval is not None\n and save_checkpoint_fn is not None\n and epoch - self.last_save_step >= self.save_interval\n ):\n self.last_save_step = epoch\n save_checkpoint_fn(epoch, env_step, update_step)\n self.write(\"save/epoch\", epoch, {\"save/epoch\": epoch})\n self.write(\"save/env_step\", env_step, {\"save/env_step\": env_step})\n self.write(\n \"save/gradient_step\",\n update_step,\n {\"save/gradient_step\": update_step},\n )\n\n def restore_data(self) -> tuple[int, int, int]:\n ea = event_accumulator.EventAccumulator(self.writer.log_dir)\n ea.Reload()\n\n try: # epoch / gradient_step\n epoch = ea.scalars.Items(\"save/epoch\")[-1].step\n self.last_save_step = self.last_log_test_step = epoch\n gradient_step = ea.scalars.Items(\"save/gradient_step\")[-1].step\n self.last_log_update_step = gradient_step\n except KeyError:\n epoch, gradient_step = 0, 0\n try: # offline trainer doesn't have env_step\n env_step = ea.scalars.Items(\"save/env_step\")[-1].step\n self.last_log_train_step = env_step\n except KeyError:\n env_step = 0\n\n return epoch, env_step, gradient_step\n\n @staticmethod\n def restore_logged_data(\n log_path: str,\n ) -> TRestoredData:\n \"\"\"Restores the logged data from the tensorboard log directory.\n\n The result is a nested dictionary where the keys are the tensorboard keys\n and the values are the corresponding numpy arrays. The keys in each level\n form a nested structure, where the hierarchy is represented by the slashes\n in the tensorboard key-strings.\n \"\"\"\n ea = event_accumulator.EventAccumulator(log_path)\n ea.Reload()\n\n def add_value_to_innermost_nested_dict(\n data_dict: dict[str, Any],\n key_string: str,\n value: Any,\n ) -> None:\n \"\"\"A particular logic, walking through the keys in the\n `key_string` and adding the value to the `data_dict` in a nested manner,\n creating nested dictionaries on the fly if necessary, or updating existing ones.\n The value is added only to the innermost-nested dictionary.\n\n\n Example:\n -------\n >>> data_dict = {}\n >>> add_value_to_innermost_nested_dict(data_dict, \"a/b/c\", 1)\n >>> data_dict\n {\"a\": {\"b\": {\"c\": 1}}}\n\n \"\"\"\n keys = key_string.split(\"/\")\n\n cur_nested_dict = data_dict\n # walk through the intermediate keys to reach the innermost-nested dict,\n # creating nested dictionaries on the fly if necessary\n for k in keys[:-1]:\n cur_nested_dict = cur_nested_dict.setdefault(k, {})\n # After the loop above,\n # this is the innermost-nested dict, where the value is finally set\n # for the last key in the key_string\n cur_nested_dict[keys[-1]] = value\n\n restored_data: dict[str, np.ndarray | dict] = {}\n for key_string in ea.scalars.Keys():\n add_value_to_innermost_nested_dict(\n restored_data,\n key_string,\n np.array([s.value for s in ea.scalars.Items(key_string)]),\n )\n\n return restored_data\n" }, { "path": "tianshou/utils/logger/wandb.py", "content": "import argparse\nimport logging\nimport os\nfrom collections.abc import Callable\n\nfrom torch.utils.tensorboard import SummaryWriter\n\nfrom tianshou.utils import BaseLogger, TensorboardLogger\nfrom tianshou.utils.logger.logger_base import VALID_LOG_VALS_TYPE, TRestoredData\n\nlog = logging.getLogger(__name__)\n\n\nclass WandbLogger(BaseLogger):\n \"\"\"Weights and Biases logger that sends data to https://wandb.ai/.\n\n This logger creates three panels with plots: train, test, and update.\n Make sure to select the correct access for each panel in weights and biases:\n\n Example of usage:\n ::\n\n logger = WandbLogger()\n logger.load(SummaryWriter(log_path))\n\n :param training_interval: the log interval in log_training_data().\n :param test_interval: the log interval in log_test_data().\n :param update_interval: the log interval in log_update_data().\n :param info_interval: the log interval in log_info_data().\n :param save_interval: the save interval in save_data(). Default to 1 (save at\n the end of each epoch).\n :param write_flush: whether to flush tensorboard result after each\n add_scalar operation. Default to True.\n :param str project: W&B project name. Default to \"tianshou\".\n :param str name: W&B run name. Default to None. If None, random name is assigned.\n :param str entity: W&B team/organization name. Default to None.\n :param str run_id: run id of W&B run to be resumed. Default to None.\n :param argparse.Namespace config: experiment configurations. Default to None.\n \"\"\"\n\n def __init__(\n self,\n training_interval: int = 1000,\n test_interval: int = 1,\n update_interval: int = 1000,\n info_interval: int = 1,\n save_interval: int | None = None,\n write_flush: bool = True,\n project: str | None = None,\n name: str | None = None,\n entity: str | None = None,\n run_id: str | None = None,\n group: str | None = None,\n job_type: str | None = None,\n config: argparse.Namespace | dict | None = None,\n monitor_gym: bool = True,\n disable_stats: bool = False,\n log_dir: str | None = None,\n ) -> None:\n import wandb\n\n super().__init__(\n training_interval, test_interval, update_interval, info_interval, save_interval\n )\n self.last_save_step = -1\n self.write_flush = write_flush\n self.restored = False\n if project is None:\n project = os.getenv(\"WANDB_PROJECT\", \"tianshou\")\n\n wandb_run = (\n wandb.init(\n project=project,\n group=group,\n job_type=job_type,\n name=name,\n id=run_id,\n resume=\"allow\",\n entity=entity,\n sync_tensorboard=True,\n # monitor_gym=monitor_gym, # currently disabled until gymnasium version is bumped to >1.0.0 https://github.com/wandb/wandb/issues/7047\n dir=log_dir,\n config=config, # type: ignore\n settings=wandb.Settings(x_disable_stats=disable_stats),\n )\n if not wandb.run\n else wandb.run\n )\n assert wandb_run is not None\n self.wandb_run = wandb_run\n self.wandb_run._label(repo=\"tianshou\")\n self.tensorboard_logger: TensorboardLogger | None = None\n self.writer: SummaryWriter | None = None\n\n def prepare_dict_for_logging(self, log_data: dict) -> dict[str, VALID_LOG_VALS_TYPE]:\n if self.tensorboard_logger is None:\n raise Exception(\n \"`logger` needs to load the Tensorboard Writer before \"\n \"preparing data for logging. Try `logger.load(SummaryWriter(log_path))`\",\n )\n return self.tensorboard_logger.prepare_dict_for_logging(log_data)\n\n def load(self, writer: SummaryWriter) -> None:\n self.writer = writer\n self.tensorboard_logger = TensorboardLogger(\n writer,\n self.training_interval,\n self.test_interval,\n self.update_interval,\n self.info_interval,\n self.save_interval,\n self.write_flush,\n )\n\n def write(self, step_type: str, step: int, data: dict[str, VALID_LOG_VALS_TYPE]) -> None:\n if self.tensorboard_logger is None:\n raise RuntimeError(\n \"`logger` needs to load the Tensorboard Writer before \"\n \"writing data. Try `logger.load(SummaryWriter(log_path))`\",\n )\n self.tensorboard_logger.write(step_type, step, data)\n\n def finalize(self) -> None:\n if self.wandb_run is not None:\n self.wandb_run.finish()\n if self.tensorboard_logger is not None:\n self.tensorboard_logger.finalize()\n\n def save_data(\n self,\n epoch: int,\n env_step: int,\n update_step: int,\n save_checkpoint_fn: Callable[[int, int, int], str] | None = None,\n ) -> None:\n \"\"\"Use writer to log metadata when calling ``save_checkpoint_fn`` in trainer.\n\n :param epoch: the epoch in trainer.\n :param env_step: the env_step in trainer.\n :param update_step: the gradient_step in trainer.\n :param function save_checkpoint_fn: a hook defined by user, see trainer\n documentation for detail.\n \"\"\"\n import wandb\n\n if (\n self.save_interval is not None\n and save_checkpoint_fn\n and epoch - self.last_save_step >= self.save_interval\n ):\n self.last_save_step = epoch\n checkpoint_path = save_checkpoint_fn(epoch, env_step, update_step)\n\n checkpoint_artifact = wandb.Artifact(\n \"run_\" + self.wandb_run.id + \"_checkpoint\",\n type=\"model\",\n metadata={\n \"save/epoch\": epoch,\n \"save/env_step\": env_step,\n \"save/gradient_step\": update_step,\n \"checkpoint_path\": str(checkpoint_path),\n },\n )\n checkpoint_artifact.add_file(str(checkpoint_path))\n self.wandb_run.log_artifact(checkpoint_artifact)\n\n def restore_data(self) -> tuple[int, int, int]:\n checkpoint_artifact = self.wandb_run.use_artifact(\n f\"run_{self.wandb_run.id}_checkpoint:latest\",\n )\n assert checkpoint_artifact is not None, \"W&B dataset artifact doesn't exist\"\n\n checkpoint_artifact.download(\n os.path.dirname(checkpoint_artifact.metadata[\"checkpoint_path\"]),\n )\n\n try: # epoch / gradient_step\n epoch = checkpoint_artifact.metadata[\"save/epoch\"]\n self.last_save_step = self.last_log_test_step = epoch\n gradient_step = checkpoint_artifact.metadata[\"save/gradient_step\"]\n self.last_log_update_step = gradient_step\n except KeyError:\n epoch, gradient_step = 0, 0\n try: # offline trainer doesn't have env_step\n env_step = checkpoint_artifact.metadata[\"save/env_step\"]\n self.last_log_train_step = env_step\n except KeyError:\n env_step = 0\n return epoch, env_step, gradient_step\n\n @staticmethod\n def restore_logged_data(log_path: str) -> TRestoredData:\n log.warning(\n \"Logging data directly from W&B is not yet implemented, will use the \"\n \"TensorboardLogger to restore it from disc instead.\",\n )\n return TensorboardLogger.restore_logged_data(log_path)\n" }, { "path": "tianshou/utils/logging.py", "content": "from typing import Any\n\n\ndef set_numerical_fields_to_precision(data: dict[str, Any], precision: int = 3) -> dict[str, Any]:\n \"\"\"Returns a copy of the given dictionary with all numerical values rounded to the given precision.\n\n Note: does not recurse into nested dictionaries.\n\n :param data: a dictionary\n :param precision: the precision to be used\n \"\"\"\n result = {}\n for k, v in data.items():\n if isinstance(v, float):\n v = round(v, precision)\n result[k] = v\n return result\n" }, { "path": "tianshou/utils/net/__init__.py", "content": "" }, { "path": "tianshou/utils/net/common.py", "content": "from abc import ABC, abstractmethod\nfrom collections.abc import Callable, Sequence\nfrom typing import Any, Generic, TypeAlias, TypeVar, cast, no_type_check\n\nimport numpy as np\nimport torch\nfrom gymnasium import spaces\nfrom torch import nn\n\nfrom tianshou.data.batch import Batch\nfrom tianshou.data.types import RecurrentStateBatch, TObs\nfrom tianshou.utils.space_info import ActionSpaceInfo\nfrom tianshou.utils.torch_utils import torch_device\n\nModuleType = type[nn.Module]\nArgsType = tuple[Any, ...] | dict[Any, Any] | Sequence[tuple[Any, ...]] | Sequence[dict[Any, Any]]\nTActionShape: TypeAlias = Sequence[int] | int | np.int64\nTLinearLayer: TypeAlias = Callable[[int, int], nn.Module]\nT = TypeVar(\"T\")\n\n\ndef miniblock(\n input_size: int,\n output_size: int = 0,\n norm_layer: ModuleType | None = None,\n norm_args: tuple[Any, ...] | dict[Any, Any] | None = None,\n activation: ModuleType | None = None,\n act_args: tuple[Any, ...] | dict[Any, Any] | None = None,\n linear_layer: TLinearLayer = nn.Linear,\n) -> list[nn.Module]:\n \"\"\"Construct a miniblock with given input/output-size, norm layer and activation.\"\"\"\n layers: list[nn.Module] = [linear_layer(input_size, output_size)]\n if norm_layer is not None:\n if isinstance(norm_args, tuple):\n layers += [norm_layer(output_size, *norm_args)]\n elif isinstance(norm_args, dict):\n layers += [norm_layer(output_size, **norm_args)]\n else:\n layers += [norm_layer(output_size)]\n if activation is not None:\n if isinstance(act_args, tuple):\n layers += [activation(*act_args)]\n elif isinstance(act_args, dict):\n layers += [activation(**act_args)]\n else:\n layers += [activation()]\n return layers\n\n\nclass ModuleWithVectorOutput(nn.Module):\n \"\"\"\n A module that outputs a vector of a known size.\n\n Use `from_module` to adapt a module to this interface.\n \"\"\"\n\n def __init__(self, output_dim: int) -> None:\n \"\"\":param output_dim: the dimension of the output vector.\"\"\"\n super().__init__()\n self.output_dim = output_dim\n\n @staticmethod\n def from_module(module: nn.Module, output_dim: int) -> \"ModuleWithVectorOutput\":\n \"\"\"\n :param module: the module to adapt.\n :param output_dim: dimension of the output vector produced by the module.\n \"\"\"\n return ModuleWithVectorOutputAdapter(module, output_dim)\n\n def get_output_dim(self) -> int:\n \"\"\":return: the dimension of the output vector.\"\"\"\n return self.output_dim\n\n\nclass ModuleWithVectorOutputAdapter(ModuleWithVectorOutput):\n \"\"\"Adapts a module with vector output to provide the :class:`ModuleWithVectorOutput` interface.\"\"\"\n\n def __init__(self, module: nn.Module, output_dim: int) -> None:\n \"\"\"\n :param module: the module to adapt.\n :param output_dim: the dimension of the output vector produced by the module.\n \"\"\"\n super().__init__(output_dim)\n self.module = module\n\n def forward(self, *args: Any, **kwargs: Any) -> Any:\n return self.module(*args, **kwargs)\n\n\nclass MLP(ModuleWithVectorOutput):\n \"\"\"Simple MLP backbone.\"\"\"\n\n def __init__(\n self,\n *,\n input_dim: int,\n output_dim: int = 0,\n hidden_sizes: Sequence[int] = (),\n norm_layer: ModuleType | Sequence[ModuleType] | None = None,\n norm_args: ArgsType | None = None,\n activation: ModuleType | Sequence[ModuleType] | None = nn.ReLU,\n act_args: ArgsType | None = None,\n linear_layer: TLinearLayer = nn.Linear,\n flatten_input: bool = True,\n ) -> None:\n \"\"\"\n :param input_dim: dimension of the input vector.\n :param output_dim: dimension of the output vector. If set to 0, there\n is no explicit final linear layer and the output dimension is the last hidden layer's dimension.\n :param hidden_sizes: shape of MLP passed in as a list, not including\n input_dim and output_dim.\n :param norm_layer: use which normalization before activation, e.g.,\n ``nn.LayerNorm`` and ``nn.BatchNorm1d``. Default to no normalization.\n You can also pass a list of normalization modules with the same length\n of hidden_sizes, to use different normalization module in different\n layers. Default to no normalization.\n :param activation: which activation to use after each layer, can be both\n the same activation for all layers if passed in nn.Module, or different\n activation for different Modules if passed in a list. Default to\n nn.ReLU.\n :param linear_layer: use this module as linear layer. Default to nn.Linear.\n :param flatten_input: whether to flatten input data. Default to True.\n \"\"\"\n if norm_layer:\n if isinstance(norm_layer, list):\n assert len(norm_layer) == len(hidden_sizes)\n norm_layer_list = norm_layer\n if isinstance(norm_args, list):\n assert len(norm_args) == len(hidden_sizes)\n norm_args_list = norm_args\n else:\n norm_args_list = [norm_args for _ in range(len(hidden_sizes))]\n else:\n norm_layer_list = [norm_layer for _ in range(len(hidden_sizes))]\n norm_args_list = [norm_args for _ in range(len(hidden_sizes))]\n else:\n norm_layer_list = [None] * len(hidden_sizes)\n norm_args_list = [None] * len(hidden_sizes)\n if activation:\n if isinstance(activation, list):\n assert len(activation) == len(hidden_sizes)\n activation_list = activation\n if isinstance(act_args, list):\n assert len(act_args) == len(hidden_sizes)\n act_args_list = act_args\n else:\n act_args_list = [act_args for _ in range(len(hidden_sizes))]\n else:\n activation_list = [activation for _ in range(len(hidden_sizes))]\n act_args_list = [act_args for _ in range(len(hidden_sizes))]\n else:\n activation_list = [None] * len(hidden_sizes)\n act_args_list = [None] * len(hidden_sizes)\n hidden_sizes = [input_dim, *list(hidden_sizes)]\n model = []\n for in_dim, out_dim, norm, norm_args, activ, act_args in zip(\n hidden_sizes[:-1],\n hidden_sizes[1:],\n norm_layer_list,\n norm_args_list,\n activation_list,\n act_args_list,\n strict=True,\n ):\n model += miniblock(in_dim, out_dim, norm, norm_args, activ, act_args, linear_layer)\n if output_dim > 0:\n model += [linear_layer(hidden_sizes[-1], output_dim)]\n super().__init__(output_dim or hidden_sizes[-1])\n self.model = nn.Sequential(*model)\n self.flatten_input = flatten_input\n\n @no_type_check\n def forward(self, obs: np.ndarray | torch.Tensor) -> torch.Tensor:\n device = torch_device(self)\n obs = torch.as_tensor(obs, device=device, dtype=torch.float32)\n if self.flatten_input:\n obs = obs.flatten(1)\n return self.model(obs)\n\n\nTRecurrentState = TypeVar(\"TRecurrentState\", bound=Any)\n\n\nclass ActionReprNet(Generic[TRecurrentState], nn.Module, ABC):\n \"\"\"Abstract base class for neural networks used to compute action-related\n representations from environment observations, which defines the\n signature of the forward method.\n\n An action-related representation can be a number of things, including:\n * a distribution over actions in a discrete action space in the form of a vector of\n unnormalized log probabilities (called \"logits\" in PyTorch jargon)\n * the Q-values of all actions in a discrete action space\n * the parameters of a distribution (e.g., mean and std. dev. for a Gaussian distribution)\n over actions in a continuous action space\n \"\"\"\n\n @abstractmethod\n def forward(\n self,\n obs: TObs,\n state: TRecurrentState | None = None,\n info: dict[str, Any] | None = None,\n ) -> tuple[torch.Tensor | Sequence[torch.Tensor], TRecurrentState | None]:\n \"\"\"\n The main method for tianshou to compute action representations (such as actions, inputs of distributions, Q-values, etc)\n from env observations.\n Implementations will always make use of the preprocess_net as the first processing step.\n\n :param obs: the observations from the environment as retrieved from `ObsBatchProtocol.obs`.\n If the environment is a dict env, this will be an instance of Batch, otherwise it will be an array (or tensor if your\n env returns tensors).\n :param state: the hidden state of the RNN, if applicable\n :param info: the info object from the environment step\n :return: a tuple (action_repr, hidden_state), where action_repr is either an actual action for the environment or\n a representation from which it can be retrieved/sampled (e.g., mean and std for a Gaussian distribution),\n and hidden_state is the new hidden state of the RNN, if applicable.\n \"\"\"\n\n\nclass ActionReprNetWithVectorOutput(Generic[T], ActionReprNet[T], ModuleWithVectorOutput):\n \"\"\"A neural network for the computation of action-related representations which outputs\n a vector of a known size.\n \"\"\"\n\n def __init__(self, output_dim: int) -> None:\n super().__init__(output_dim)\n\n\nclass Actor(Generic[T], ActionReprNetWithVectorOutput[T], ABC):\n @abstractmethod\n def get_preprocess_net(self) -> ModuleWithVectorOutput:\n \"\"\"Returns the network component that is used for pre-processing, i.e.\n the component which produces a latent representation, which then is transformed\n into the final output.\n This is, therefore, the first part of the network which processes the input.\n For example, a CNN is often used in Atari examples.\n\n We need this method to be able to share latent representation computations with\n other networks (e.g. critics) within an algorithm.\n\n Actors that do not have a pre-processing stage can return nn.Identity()\n (see :class:`RandomActor` for an example).\n \"\"\"\n\n\nclass Net(ActionReprNetWithVectorOutput[Any]):\n \"\"\"A multi-layer perceptron which outputs an action-related representation.\n\n :param state_shape: int or a sequence of int of the shape of state.\n :param action_shape: int or a sequence of int of the shape of action.\n :param hidden_sizes: shape of MLP passed in as a list.\n :param norm_layer: use which normalization before activation, e.g.,\n ``nn.LayerNorm`` and ``nn.BatchNorm1d``. Default to no normalization.\n You can also pass a list of normalization modules with the same length\n of hidden_sizes, to use different normalization module in different\n layers. Default to no normalization.\n :param activation: which activation to use after each layer, can be both\n the same activation for all layers if passed in nn.Module, or different\n activation for different Modules if passed in a list. Default to\n nn.ReLU.\n :param softmax: whether to apply a softmax layer over the last layer's\n output.\n :param concat: whether the input shape is concatenated by state_shape\n and action_shape. If it is True, ``action_shape`` is not the output\n shape, but affects the input shape only.\n :param num_atoms: in order to expand to the net of distributional RL.\n Default to 1 (not use).\n :param dueling_param: whether to use dueling network to calculate Q\n values (for Dueling DQN). If you want to use dueling option, you should\n pass a tuple of two dict (first for Q and second for V) stating\n self-defined arguments as stated in\n class:`~tianshou.utils.net.common.MLP`. Default to None.\n :param linear_layer: use this module constructor, which takes the input\n and output dimension as input, as linear layer. Default to nn.Linear.\n\n .. seealso::\n\n Please refer to :class:`~tianshou.utils.net.common.MLP` for more\n detailed explanation on the usage of activation, norm_layer, etc.\n\n You can also refer to :class:`~tianshou.utils.net.continuous.Actor`,\n :class:`~tianshou.utils.net.continuous.Critic`, etc, to see how it's\n suggested be used.\n \"\"\"\n\n def __init__(\n self,\n *,\n state_shape: int | Sequence[int],\n action_shape: TActionShape = 0,\n hidden_sizes: Sequence[int] = (),\n norm_layer: ModuleType | Sequence[ModuleType] | None = None,\n norm_args: ArgsType | None = None,\n activation: ModuleType | Sequence[ModuleType] | None = nn.ReLU,\n act_args: ArgsType | None = None,\n softmax: bool = False,\n concat: bool = False,\n num_atoms: int = 1,\n dueling_param: tuple[dict[str, Any], dict[str, Any]] | None = None,\n linear_layer: TLinearLayer = nn.Linear,\n ) -> None:\n input_dim = int(np.prod(state_shape))\n action_dim = int(np.prod(action_shape)) * num_atoms\n if concat:\n input_dim += action_dim\n use_dueling = dueling_param is not None\n model = MLP(\n input_dim=input_dim,\n output_dim=action_dim if not use_dueling and not concat else 0,\n hidden_sizes=hidden_sizes,\n norm_layer=norm_layer,\n norm_args=norm_args,\n activation=activation,\n act_args=act_args,\n linear_layer=linear_layer,\n )\n Q: MLP | None = None\n V: MLP | None = None\n if use_dueling: # dueling DQN\n assert dueling_param is not None\n kwargs_update = {\n \"input_dim\": model.output_dim,\n }\n # Important: don't change the original dict (e.g., don't use .update())\n q_kwargs = {**dueling_param[0], **kwargs_update}\n v_kwargs = {**dueling_param[1], **kwargs_update}\n\n q_kwargs[\"output_dim\"] = 0 if concat else action_dim\n v_kwargs[\"output_dim\"] = 0 if concat else num_atoms\n Q, V = MLP(**q_kwargs), MLP(**v_kwargs)\n output_dim = Q.output_dim\n else:\n output_dim = model.output_dim\n\n super().__init__(output_dim)\n self.use_dueling = use_dueling\n self.softmax = softmax\n self.num_atoms = num_atoms\n self.model = model\n self.Q = Q\n self.V = V\n\n def forward(\n self,\n obs: TObs,\n state: T | None = None,\n info: dict[str, Any] | None = None,\n ) -> tuple[torch.Tensor, T | Any]:\n \"\"\"Mapping: obs -> flatten (inside MLP)-> logits.\n\n :param obs:\n :param state: unused and returned as is\n :param info: unused\n \"\"\"\n logits = self.model(obs)\n batch_size = logits.shape[0]\n if self.use_dueling: # Dueling DQN\n assert self.Q is not None\n assert self.V is not None\n q, v = self.Q(logits), self.V(logits)\n if self.num_atoms > 1:\n q = q.view(batch_size, -1, self.num_atoms)\n v = v.view(batch_size, -1, self.num_atoms)\n logits = q - q.mean(dim=1, keepdim=True) + v\n elif self.num_atoms > 1:\n logits = logits.view(batch_size, -1, self.num_atoms)\n if self.softmax:\n logits = torch.softmax(logits, dim=-1)\n return logits, state\n\n\nclass Recurrent(ActionReprNetWithVectorOutput[RecurrentStateBatch]):\n \"\"\"Simple Recurrent network based on LSTM.\"\"\"\n\n def __init__(\n self,\n *,\n layer_num: int,\n state_shape: int | Sequence[int],\n action_shape: TActionShape,\n hidden_layer_size: int = 128,\n ) -> None:\n output_dim = int(np.prod(action_shape))\n super().__init__(output_dim)\n self.nn = nn.LSTM(\n input_size=hidden_layer_size,\n hidden_size=hidden_layer_size,\n num_layers=layer_num,\n batch_first=True,\n )\n self.fc1 = nn.Linear(int(np.prod(state_shape)), hidden_layer_size)\n self.fc2 = nn.Linear(hidden_layer_size, output_dim)\n\n def get_preprocess_net(self) -> ModuleWithVectorOutput:\n return ModuleWithVectorOutput.from_module(nn.Identity(), self.output_dim)\n\n def forward(\n self,\n obs: TObs,\n state: RecurrentStateBatch | None = None,\n info: dict[str, Any] | None = None,\n ) -> tuple[torch.Tensor, RecurrentStateBatch]:\n \"\"\"Mapping: obs -> flatten -> logits.\n\n In the evaluation mode, `obs` should be with shape ``[bsz, dim]``; in the\n training mode, `obs` should be with shape ``[bsz, len, dim]``. See the code\n and comment for more detail.\n\n :param obs:\n :param state: either None or a dict with keys 'hidden' and 'cell'\n :param info: unused\n :return: predicted action, next state as dict with keys 'hidden' and 'cell'\n \"\"\"\n # Note: the original type of state is Batch but it might also be a dict\n # If it is a Batch, .issubset(state) will not work. However,\n # issubset(state.keys()) always works\n if state is not None and not {\"hidden\", \"cell\"}.issubset(state.keys()):\n raise ValueError(\n f\"Expected to find keys 'hidden' and 'cell' but instead found {state.keys()}\",\n )\n\n device = torch_device(self)\n obs = torch.as_tensor(obs, device=device, dtype=torch.float32)\n # obs [bsz, len, dim] (training) or [bsz, dim] (evaluation)\n # In short, the tensor's shape in training phase is longer than which\n # in evaluation phase.\n if len(obs.shape) == 2:\n obs = obs.unsqueeze(-2)\n obs = self.fc1(obs)\n self.nn.flatten_parameters()\n if state is None:\n obs, (hidden, cell) = self.nn(obs)\n else:\n # we store the stack data in [bsz, len, ...] format\n # but pytorch rnn needs [len, bsz, ...]\n obs, (hidden, cell) = self.nn(\n obs,\n (\n state[\"hidden\"].transpose(0, 1).contiguous(),\n state[\"cell\"].transpose(0, 1).contiguous(),\n ),\n )\n obs = self.fc2(obs[:, -1])\n # please ensure the first dim is batch size: [bsz, len, ...]\n rnn_state_batch = cast(\n RecurrentStateBatch,\n Batch(\n {\n \"hidden\": hidden.transpose(0, 1).detach(),\n \"cell\": cell.transpose(0, 1).detach(),\n },\n ),\n )\n return obs, rnn_state_batch\n\n\nclass ActorCritic(nn.Module):\n \"\"\"An actor-critic network for parsing parameters.\n\n Using ``actor_critic.parameters()`` instead of set.union or list+list to avoid\n issue #449.\n\n :param nn.Module actor: the actor network.\n :param nn.Module critic: the critic network.\n \"\"\"\n\n def __init__(self, actor: nn.Module, critic: nn.Module) -> None:\n super().__init__()\n self.actor = actor\n self.critic = critic\n\n\nclass DataParallelNet(nn.Module):\n \"\"\"DataParallel wrapper for training agent with multi-GPU.\n\n This class does only the conversion of input data type, from numpy array to torch's\n Tensor. If the input is a nested dictionary, the user should create a similar class\n to do the same thing.\n\n :param net: the network to be distributed in different GPUs.\n \"\"\"\n\n def __init__(self, net: nn.Module) -> None:\n super().__init__()\n self.net = nn.DataParallel(net)\n\n def forward(\n self,\n obs: TObs,\n *args: Any,\n **kwargs: Any,\n ) -> tuple[Any, Any]:\n if not isinstance(obs, torch.Tensor):\n obs = torch.as_tensor(obs, dtype=torch.float32)\n obs = obs.cuda()\n return self.net(obs, *args, **kwargs)\n\n\n# The same functionality as DataParallelNet\n# The duplication is worth it because the ActionReprNet abstraction is so important\nclass ActionReprNetDataParallelWrapper(ActionReprNet):\n def __init__(self, net: ActionReprNet) -> None:\n super().__init__()\n self.net = nn.DataParallel(net)\n\n def forward(\n self,\n obs: TObs,\n state: TRecurrentState | None = None,\n info: dict[str, Any] | None = None,\n ) -> tuple[torch.Tensor, TRecurrentState | None]:\n if not isinstance(obs, torch.Tensor):\n obs = torch.as_tensor(obs, dtype=torch.float32)\n obs = obs.cuda()\n return self.net(obs, state=state, info=info)\n\n\nclass EnsembleLinear(nn.Module):\n \"\"\"Linear Layer of Ensemble network.\n\n :param ensemble_size: Number of subnets in the ensemble.\n :param in_feature: dimension of the input vector.\n :param out_feature: dimension of the output vector.\n :param bias: whether to include an additive bias, default to be True.\n \"\"\"\n\n def __init__(\n self,\n ensemble_size: int,\n in_feature: int,\n out_feature: int,\n bias: bool = True,\n ) -> None:\n super().__init__()\n\n # To be consistent with PyTorch default initializer\n k = np.sqrt(1.0 / in_feature)\n weight_data = torch.rand((ensemble_size, in_feature, out_feature)) * 2 * k - k\n self.weight = nn.Parameter(weight_data, requires_grad=True)\n\n self.bias_weights: nn.Parameter | None = None\n if bias:\n bias_data = torch.rand((ensemble_size, 1, out_feature)) * 2 * k - k\n self.bias_weights = nn.Parameter(bias_data, requires_grad=True)\n\n def forward(self, x: torch.Tensor) -> torch.Tensor:\n x = torch.matmul(x, self.weight)\n if self.bias_weights is not None:\n x = x + self.bias_weights\n return x\n\n\nclass BranchingNet(ActionReprNet):\n \"\"\"Branching dual Q network.\n\n Network for the BranchingDQNPolicy, it uses a common network module, a value module\n and action \"branches\" one for each dimension. It allows for a linear scaling\n of Q-value the output w.r.t. the number of dimensions in the action space.\n\n This network architecture efficiently handles environments with multiple independent\n action dimensions by using a branching structure. Instead of representing all action\n combinations (which grows exponentially), it represents each action dimension separately\n (linear scaling).\n For example, if there are 3 actions with 3 possible values each, then we would normally\n need to consider 3^4 = 81 unique actions, whereas with this architecture, we can instead\n use 3 branches with 4 actions per dimension, resulting in 3 * 4 = 12 values to be considered.\n\n Common use cases include multi-joint robotic control tasks, where each joint can be controlled\n independently.\n\n For more information, please refer to: arXiv:1711.08946.\n \"\"\"\n\n def __init__(\n self,\n *,\n state_shape: int | Sequence[int],\n num_branches: int = 0,\n action_per_branch: int = 2,\n common_hidden_sizes: list[int] | None = None,\n value_hidden_sizes: list[int] | None = None,\n action_hidden_sizes: list[int] | None = None,\n norm_layer: ModuleType | None = None,\n norm_args: ArgsType | None = None,\n activation: ModuleType | None = nn.ReLU,\n act_args: ArgsType | None = None,\n ) -> None:\n \"\"\"\n :param state_shape: int or a sequence of int of the shape of state.\n :param num_branches: number of action dimensions in the environment.\n Each branch represents one independent action dimension.\n For example, in a robot with 7 joints, you would set this to 7.\n :param action_per_branch: Number of possible discrete values for each action dimension.\n For example, if each joint can have 3 positions (left, center, right),\n you would set this to 3.\n :param common_hidden_sizes: shape of the common MLP network passed in as a list.\n :param value_hidden_sizes: shape of the value MLP network passed in as a list.\n :param action_hidden_sizes: shape of the action MLP network passed in as a list.\n :param norm_layer: use which normalization before activation, e.g.,\n ``nn.LayerNorm`` and ``nn.BatchNorm1d``. Default to no normalization.\n You can also pass a list of normalization modules with the same length\n of hidden_sizes, to use different normalization module in different\n layers. Default to no normalization.\n :param activation: which activation to use after each layer, can be both\n the same activation for all layers if passed in nn.Module, or different\n activation for different Modules if passed in a list. Default to\n nn.ReLU.\n \"\"\"\n super().__init__()\n common_hidden_sizes = common_hidden_sizes or []\n value_hidden_sizes = value_hidden_sizes or []\n action_hidden_sizes = action_hidden_sizes or []\n\n self.num_branches = num_branches\n self.action_per_branch = action_per_branch\n # common network\n common_input_dim = int(np.prod(state_shape))\n common_output_dim = 0\n self.common = MLP(\n input_dim=common_input_dim,\n output_dim=common_output_dim,\n hidden_sizes=common_hidden_sizes,\n norm_layer=norm_layer,\n norm_args=norm_args,\n activation=activation,\n act_args=act_args,\n )\n # value network\n value_input_dim = common_hidden_sizes[-1]\n value_output_dim = 1\n self.value = MLP(\n input_dim=value_input_dim,\n output_dim=value_output_dim,\n hidden_sizes=value_hidden_sizes,\n norm_layer=norm_layer,\n norm_args=norm_args,\n activation=activation,\n act_args=act_args,\n )\n # action branching network\n action_input_dim = common_hidden_sizes[-1]\n action_output_dim = action_per_branch\n self.branches = nn.ModuleList(\n [\n MLP(\n input_dim=action_input_dim,\n output_dim=action_output_dim,\n hidden_sizes=action_hidden_sizes,\n norm_layer=norm_layer,\n norm_args=norm_args,\n activation=activation,\n act_args=act_args,\n )\n for _ in range(self.num_branches)\n ],\n )\n\n def forward(\n self,\n obs: TObs,\n state: T | None = None,\n info: dict[str, Any] | None = None,\n ) -> tuple[torch.Tensor, T | None]:\n \"\"\"Mapping: obs -> model -> logits.\"\"\"\n common_out = self.common(obs)\n value_out = self.value(common_out)\n value_out = torch.unsqueeze(value_out, 1)\n action_out = []\n for b in self.branches:\n action_out.append(b(common_out))\n action_scores = torch.stack(action_out, 1)\n action_scores = action_scores - torch.mean(action_scores, 2, keepdim=True)\n logits = value_out + action_scores\n return logits, state\n\n\ndef get_dict_state_decorator(\n state_shape: dict[str, int | Sequence[int]],\n keys: Sequence[str],\n) -> tuple[Callable, int]:\n \"\"\"A helper function to make Net or equivalent classes (e.g. Actor, Critic) applicable to dict state.\n\n The first return item, ``decorator_fn``, will alter the implementation of forward\n function of the given class by preprocessing the observation. The preprocessing is\n basically flatten the observation and concatenate them based on the ``keys`` order.\n The batch dimension is preserved if presented. The result observation shape will\n be equal to ``new_state_shape``, the second return item.\n\n :param state_shape: A dictionary indicating each state's shape\n :param keys: A list of state's keys. The flatten observation will be according to\n this list order.\n :returns: a 2-items tuple ``decorator_fn`` and ``new_state_shape``\n \"\"\"\n original_shape = state_shape\n flat_state_shapes = []\n for k in keys:\n flat_state_shapes.append(int(np.prod(state_shape[k])))\n new_state_shape = sum(flat_state_shapes)\n\n def preprocess_obs(obs: Batch | dict | torch.Tensor | np.ndarray) -> torch.Tensor:\n if isinstance(obs, dict) or (isinstance(obs, Batch) and keys[0] in obs):\n if original_shape[keys[0]] == obs[keys[0]].shape:\n # No batch dim\n new_obs = torch.Tensor([obs[k] for k in keys]).flatten()\n # new_obs = torch.Tensor([obs[k] for k in keys]).reshape(1, -1)\n else:\n bsz = obs[keys[0]].shape[0]\n new_obs = torch.cat([torch.Tensor(obs[k].reshape(bsz, -1)) for k in keys], dim=1)\n else:\n new_obs = torch.Tensor(obs)\n return new_obs\n\n @no_type_check\n def decorator_fn(net_class):\n class new_net_class(net_class):\n def forward(self, obs: TObs, *args, **kwargs) -> Any:\n return super().forward(preprocess_obs(obs), *args, **kwargs)\n\n return new_net_class\n\n return decorator_fn, new_state_shape\n\n\nclass AbstractContinuousActorProbabilistic(Actor, ABC):\n \"\"\"Type bound for probabilistic actors which output distribution parameters for continuous action spaces.\"\"\"\n\n\nclass AbstractDiscreteActor(Actor, ABC):\n \"\"\"\n Type bound for discrete actors.\n\n For on-policy algos like Reinforce, this typically directly outputs unnormalized log\n probabilities, which can be interpreted as \"logits\" in conjunction with a\n `torch.distributions.Categorical` instance.\n\n In Tianshou, discrete actors are also used for computing action distributions within\n Q-learning type algorithms (e.g., DQN). In this case, the observations are mapped\n to a vector of Q-values (one for each action). In other words, the component is actually\n a critic, not an actor in the traditional sense.\n Note that when sampling actions, the Q-values can be interpreted as inputs for\n a `torch.distributions.Categorical` instance, similar to the on-policy case mentioned\n above.\n \"\"\"\n\n\nclass RandomActor(AbstractContinuousActorProbabilistic, AbstractDiscreteActor):\n \"\"\"An actor that returns random actions.\n\n For continuous action spaces, forward returns a batch of random actions sampled from the action space.\n For discrete action spaces, forward returns a batch of n-dimensional arrays corresponding to the\n uniform distribution over the n possible actions (same interface as in :class:`~.net.discrete.Actor`).\n \"\"\"\n\n def __init__(self, action_space: spaces.Box | spaces.Discrete) -> None:\n if isinstance(action_space, spaces.Discrete):\n output_dim = action_space.n\n else:\n output_dim = np.prod(action_space.shape)\n super().__init__(int(output_dim))\n self._action_space = action_space\n self._space_info = ActionSpaceInfo.from_space(action_space)\n\n @property\n def action_space(self) -> spaces.Box | spaces.Discrete:\n return self._action_space\n\n @property\n def space_info(self) -> ActionSpaceInfo:\n return self._space_info\n\n def get_preprocess_net(self) -> ModuleWithVectorOutput:\n return ModuleWithVectorOutput.from_module(nn.Identity(), self.output_dim)\n\n def get_output_dim(self) -> int:\n return self.space_info.action_dim\n\n @property\n def is_discrete(self) -> bool:\n return isinstance(self.action_space, spaces.Discrete)\n\n def forward(\n self,\n obs: TObs,\n state: T | None = None,\n info: dict[str, Any] | None = None,\n ) -> tuple[torch.Tensor, T | None]:\n batch_size = len(obs)\n if isinstance(self.action_space, spaces.Box):\n action = np.stack([self.action_space.sample() for _ in range(batch_size)])\n else:\n # Discrete Actors currently return an n-dimensional array of probabilities for each action\n action = 1 / self.action_space.n * np.ones((batch_size, self.action_space.n))\n return torch.Tensor(action), state\n\n def compute_action_batch(self, obs: TObs) -> torch.Tensor:\n if self.is_discrete:\n # Different from forward which returns discrete probabilities, see comment there\n assert isinstance(self.action_space, spaces.Discrete) # for mypy\n return torch.Tensor(np.random.randint(low=0, high=self.action_space.n, size=len(obs)))\n else:\n return self.forward(obs)[0]\n" }, { "path": "tianshou/utils/net/continuous.py", "content": "import warnings\nfrom abc import ABC, abstractmethod\nfrom collections.abc import Sequence\nfrom typing import Any, TypeVar\n\nimport numpy as np\nimport torch\nfrom sensai.util.pickle import setstate\nfrom torch import nn\n\nfrom tianshou.data.types import TObs\nfrom tianshou.utils.net.common import (\n MLP,\n AbstractContinuousActorProbabilistic,\n Actor,\n ModuleWithVectorOutput,\n TActionShape,\n TLinearLayer,\n)\nfrom tianshou.utils.torch_utils import torch_device\n\nSIGMA_MIN = -20\nSIGMA_MAX = 2\n\nT = TypeVar(\"T\")\n\n\nclass AbstractContinuousActorDeterministic(Actor, ABC):\n \"\"\"Marker interface for continuous deterministic actors (DDPG like).\"\"\"\n\n\nclass ContinuousActorDeterministic(AbstractContinuousActorDeterministic):\n \"\"\"Actor network that directly outputs actions for continuous action space.\n Used primarily in DDPG and its variants.\n\n It will create an actor operated in continuous action space with structure of preprocess_net ---> action_shape.\n\n :param preprocess_net: first part of input processing.\n :param action_shape: a sequence of int for the shape of action.\n :param hidden_sizes: a sequence of int for constructing the MLP after\n preprocess_net.\n :param max_action: the scale for the final action.\n \"\"\"\n\n def __init__(\n self,\n *,\n preprocess_net: ModuleWithVectorOutput,\n action_shape: TActionShape,\n hidden_sizes: Sequence[int] = (),\n max_action: float = 1.0,\n ) -> None:\n output_dim = int(np.prod(action_shape))\n super().__init__(output_dim)\n self.preprocess = preprocess_net\n input_dim = preprocess_net.get_output_dim()\n self.last = MLP(\n input_dim=input_dim,\n output_dim=self.output_dim,\n hidden_sizes=hidden_sizes,\n )\n self.max_action = max_action\n\n def get_preprocess_net(self) -> ModuleWithVectorOutput:\n return self.preprocess\n\n def get_output_dim(self) -> int:\n return self.output_dim\n\n def forward(\n self,\n obs: TObs,\n state: T | None = None,\n info: dict[str, Any] | None = None,\n ) -> tuple[torch.Tensor, T | None]:\n \"\"\"Mapping: s_B -> action_values_BA, hidden_state_BH | None.\n\n Returns a tensor representing the actions directly, i.e, of shape\n `(n_actions, )`, and a hidden state (which may be None).\n The hidden state is only not None if a recurrent net is used as part of the\n learning algorithm (support for RNNs is currently experimental).\n \"\"\"\n action_BA, hidden_BH = self.preprocess(obs, state)\n action_BA = self.max_action * torch.tanh(self.last(action_BA))\n return action_BA, hidden_BH\n\n\nclass AbstractContinuousCritic(ModuleWithVectorOutput, ABC):\n @abstractmethod\n def forward(\n self,\n obs: np.ndarray | torch.Tensor,\n act: np.ndarray | torch.Tensor | None = None,\n info: dict[str, Any] | None = None,\n ) -> torch.Tensor:\n \"\"\"Mapping: (s_B, a_B) -> Q(s, a)_B.\"\"\"\n\n\nclass ContinuousCritic(AbstractContinuousCritic):\n \"\"\"Simple critic network.\n\n It will create an actor operated in continuous action space with structure of preprocess_net ---> 1(q value).\n\n :param preprocess_net: the pre-processing network, which returns a vector of a known dimension.\n Typically, an instance of :class:`~tianshou.utils.net.common.Net`.\n :param hidden_sizes: a sequence of int for constructing the MLP after\n preprocess_net.\n :param linear_layer: use this module as linear layer.\n :param flatten_input: whether to flatten input data for the last layer.\n :param apply_preprocess_net_to_obs_only: whether to apply `preprocess_net` to the observations only (before\n concatenating with the action) - and without the observations being modified in any way beforehand.\n This allows the actor's preprocessing network to be reused for the critic.\n \"\"\"\n\n def __init__(\n self,\n *,\n preprocess_net: ModuleWithVectorOutput,\n hidden_sizes: Sequence[int] = (),\n linear_layer: TLinearLayer = nn.Linear,\n flatten_input: bool = True,\n apply_preprocess_net_to_obs_only: bool = False,\n ) -> None:\n super().__init__(output_dim=1)\n self.preprocess = preprocess_net\n self.apply_preprocess_net_to_obs_only = apply_preprocess_net_to_obs_only\n input_dim = preprocess_net.get_output_dim()\n self.last = MLP(\n input_dim=input_dim,\n output_dim=1,\n hidden_sizes=hidden_sizes,\n linear_layer=linear_layer,\n flatten_input=flatten_input,\n )\n\n def __setstate__(self, state: dict) -> None:\n setstate(\n ContinuousCritic,\n self,\n state,\n new_default_properties={\"apply_preprocess_net_to_obs_only\": False},\n )\n\n def forward(\n self,\n obs: np.ndarray | torch.Tensor,\n act: np.ndarray | torch.Tensor | None = None,\n info: dict[str, Any] | None = None,\n ) -> torch.Tensor:\n \"\"\"Mapping: (s_B, a_B) -> Q(s, a)_B.\"\"\"\n device = torch_device(self)\n obs = torch.as_tensor(\n obs,\n device=device,\n dtype=torch.float32,\n )\n if self.apply_preprocess_net_to_obs_only:\n obs, _ = self.preprocess(obs)\n obs = obs.flatten(1)\n if act is not None:\n act = torch.as_tensor(\n act,\n device=device,\n dtype=torch.float32,\n ).flatten(1)\n obs = torch.cat([obs, act], dim=1)\n if not self.apply_preprocess_net_to_obs_only:\n obs, _ = self.preprocess(obs)\n return self.last(obs)\n\n\nclass ContinuousActorProbabilistic(AbstractContinuousActorProbabilistic):\n \"\"\"Simple actor network that outputs `mu` and `sigma` to be used as input for a `dist_fn` (typically, a Gaussian).\n\n Used primarily in SAC, PPO and variants thereof. For deterministic policies, see :class:`~Actor`.\n\n :param preprocess_net: the pre-processing network, which returns a vector of a known dimension.\n :param action_shape: a sequence of int for the shape of action.\n :param hidden_sizes: a sequence of int for constructing the MLP after\n preprocess_net.\n :param max_action: the scale for the final action logits.\n :param unbounded: whether to apply tanh activation on final logits.\n :param conditioned_sigma: True when sigma is calculated from the\n input, False when sigma is an independent parameter.\n \"\"\"\n\n def __init__(\n self,\n *,\n preprocess_net: ModuleWithVectorOutput,\n action_shape: TActionShape,\n hidden_sizes: Sequence[int] = (),\n max_action: float = 1.0,\n unbounded: bool = False,\n conditioned_sigma: bool = False,\n ) -> None:\n output_dim = int(np.prod(action_shape))\n super().__init__(output_dim)\n if unbounded and not np.isclose(max_action, 1.0):\n warnings.warn(\"Note that max_action input will be discarded when unbounded is True.\")\n max_action = 1.0\n self.preprocess = preprocess_net\n input_dim = preprocess_net.get_output_dim()\n self.mu = MLP(input_dim=input_dim, output_dim=output_dim, hidden_sizes=hidden_sizes)\n self._c_sigma = conditioned_sigma\n if conditioned_sigma:\n self.sigma = MLP(\n input_dim=input_dim,\n output_dim=output_dim,\n hidden_sizes=hidden_sizes,\n )\n else:\n self.sigma_param = nn.Parameter(torch.zeros(output_dim, 1))\n self.max_action = max_action\n self._unbounded = unbounded\n\n def get_preprocess_net(self) -> ModuleWithVectorOutput:\n return self.preprocess\n\n def forward(\n self,\n obs: TObs,\n state: T | None = None,\n info: dict[str, Any] | None = None,\n ) -> tuple[tuple[torch.Tensor, torch.Tensor], T | None]:\n if info is None:\n info = {}\n logits, hidden = self.preprocess(obs, state)\n mu = self.mu(logits)\n if not self._unbounded:\n mu = self.max_action * torch.tanh(mu)\n if self._c_sigma:\n sigma = torch.clamp(self.sigma(logits), min=SIGMA_MIN, max=SIGMA_MAX).exp()\n else:\n shape = [1] * len(mu.shape)\n shape[1] = -1\n sigma = (self.sigma_param.view(shape) + torch.zeros_like(mu)).exp()\n return (mu, sigma), state\n\n\nclass RecurrentActorProb(nn.Module):\n \"\"\"Recurrent version of ActorProb.\"\"\"\n\n def __init__(\n self,\n *,\n layer_num: int,\n state_shape: Sequence[int],\n action_shape: Sequence[int],\n hidden_layer_size: int = 128,\n max_action: float = 1.0,\n unbounded: bool = False,\n conditioned_sigma: bool = False,\n ) -> None:\n super().__init__()\n if unbounded and not np.isclose(max_action, 1.0):\n warnings.warn(\"Note that max_action input will be discarded when unbounded is True.\")\n max_action = 1.0\n self.nn = nn.LSTM(\n input_size=int(np.prod(state_shape)),\n hidden_size=hidden_layer_size,\n num_layers=layer_num,\n batch_first=True,\n )\n output_dim = int(np.prod(action_shape))\n self.mu = nn.Linear(hidden_layer_size, output_dim)\n self._c_sigma = conditioned_sigma\n if conditioned_sigma:\n self.sigma = nn.Linear(hidden_layer_size, output_dim)\n else:\n self.sigma_param = nn.Parameter(torch.zeros(output_dim, 1))\n self.max_action = max_action\n self._unbounded = unbounded\n\n def forward(\n self,\n obs: np.ndarray | torch.Tensor,\n state: dict[str, torch.Tensor] | None = None,\n info: dict[str, Any] | None = None,\n ) -> tuple[tuple[torch.Tensor, torch.Tensor], dict[str, torch.Tensor]]:\n \"\"\"Almost the same as :class:`~tianshou.utils.net.common.Recurrent`.\"\"\"\n if info is None:\n info = {}\n device = torch_device(self)\n obs = torch.as_tensor(\n obs,\n device=device,\n dtype=torch.float32,\n )\n # obs [bsz, len, dim] (training) or [bsz, dim] (evaluation)\n # In short, the tensor's shape in training phase is longer than which\n # in evaluation phase.\n if len(obs.shape) == 2:\n obs = obs.unsqueeze(-2)\n self.nn.flatten_parameters()\n if state is None:\n obs, (hidden, cell) = self.nn(obs)\n else:\n # we store the stack data in [bsz, len, ...] format\n # but pytorch rnn needs [len, bsz, ...]\n obs, (hidden, cell) = self.nn(\n obs,\n (\n state[\"hidden\"].transpose(0, 1).contiguous(),\n state[\"cell\"].transpose(0, 1).contiguous(),\n ),\n )\n logits = obs[:, -1]\n mu = self.mu(logits)\n if not self._unbounded:\n mu = self.max_action * torch.tanh(mu)\n if self._c_sigma:\n sigma = torch.clamp(self.sigma(logits), min=SIGMA_MIN, max=SIGMA_MAX).exp()\n else:\n shape = [1] * len(mu.shape)\n shape[1] = -1\n sigma = (self.sigma_param.view(shape) + torch.zeros_like(mu)).exp()\n # please ensure the first dim is batch size: [bsz, len, ...]\n return (mu, sigma), {\n \"hidden\": hidden.transpose(0, 1).detach(),\n \"cell\": cell.transpose(0, 1).detach(),\n }\n\n\nclass RecurrentCritic(nn.Module):\n \"\"\"Recurrent version of Critic.\"\"\"\n\n def __init__(\n self,\n layer_num: int,\n state_shape: Sequence[int],\n action_shape: Sequence[int] = (0,),\n hidden_layer_size: int = 128,\n ) -> None:\n super().__init__()\n self.state_shape = state_shape\n self.action_shape = action_shape\n self.nn = nn.LSTM(\n input_size=int(np.prod(state_shape)),\n hidden_size=hidden_layer_size,\n num_layers=layer_num,\n batch_first=True,\n )\n self.fc2 = nn.Linear(hidden_layer_size + int(np.prod(action_shape)), 1)\n\n def forward(\n self,\n obs: np.ndarray | torch.Tensor,\n act: np.ndarray | torch.Tensor | None = None,\n info: dict[str, Any] | None = None,\n ) -> torch.Tensor:\n \"\"\"Almost the same as :class:`~tianshou.utils.net.common.Recurrent`.\"\"\"\n if info is None:\n info = {}\n device = torch_device(self)\n obs = torch.as_tensor(\n obs,\n device=device,\n dtype=torch.float32,\n )\n # obs [bsz, len, dim] (training) or [bsz, dim] (evaluation)\n # In short, the tensor's shape in training phase is longer than which\n # in evaluation phase.\n assert len(obs.shape) == 3\n self.nn.flatten_parameters()\n obs, (hidden, cell) = self.nn(obs)\n obs = obs[:, -1]\n if act is not None:\n act = torch.as_tensor(\n act,\n device=device,\n dtype=torch.float32,\n )\n obs = torch.cat([obs, act], dim=1)\n return self.fc2(obs)\n\n\nclass Perturbation(nn.Module):\n \"\"\"Implementation of perturbation network in BCQ algorithm.\n\n Given a state and action, it can generate perturbed action.\n\n :param preprocess_net: a self-defined preprocess_net which output a\n flattened hidden state.\n :param max_action: the maximum value of each dimension of action.\n :param device: which device to create this model on.\n :param phi: max perturbation parameter for BCQ.\n\n .. seealso::\n\n You can refer to `examples/offline/offline_bcq.py` to see how to use it.\n \"\"\"\n\n def __init__(\n self,\n *,\n preprocess_net: nn.Module,\n max_action: float,\n phi: float = 0.05,\n ):\n # preprocess_net: input_dim=state_dim+action_dim, output_dim=action_dim\n super().__init__()\n self.preprocess_net = preprocess_net\n self.max_action = max_action\n self.phi = phi\n\n def forward(self, state: torch.Tensor, action: torch.Tensor) -> torch.Tensor:\n # preprocess_net\n logits = self.preprocess_net(torch.cat([state, action], -1))[0]\n noise = self.phi * self.max_action * torch.tanh(logits)\n # clip to [-max_action, max_action]\n return (noise + action).clamp(-self.max_action, self.max_action)\n\n\nclass VAE(nn.Module):\n \"\"\"Implementation of VAE.\n\n It models the distribution of action. Given a state, it can generate actions similar to those in batch.\n It is used in BCQ algorithm.\n\n :param encoder: the encoder in VAE. Its input_dim must be\n state_dim + action_dim, and output_dim must be hidden_dim.\n :param decoder: the decoder in VAE. Its input_dim must be\n state_dim + latent_dim, and output_dim must be action_dim.\n :param hidden_dim: the size of the last linear-layer in encoder.\n :param latent_dim: the size of latent layer.\n :param max_action: the maximum value of each dimension of action.\n :param device: which device to create this model on.\n\n .. seealso::\n\n You can refer to `examples/offline/offline_bcq.py` to see how to use it.\n \"\"\"\n\n def __init__(\n self,\n *,\n encoder: nn.Module,\n decoder: nn.Module,\n hidden_dim: int,\n latent_dim: int,\n max_action: float,\n ):\n super().__init__()\n self.encoder = encoder\n\n self.mean = nn.Linear(hidden_dim, latent_dim)\n self.log_std = nn.Linear(hidden_dim, latent_dim)\n\n self.decoder = decoder\n\n self.max_action = max_action\n self.latent_dim = latent_dim\n\n def forward(\n self,\n state: torch.Tensor,\n action: torch.Tensor,\n ) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor]:\n # [state, action] -> z , [state, z] -> action\n latent_z = self.encoder(torch.cat([state, action], -1))\n # shape of z: (state.shape[:-1], hidden_dim)\n\n mean = self.mean(latent_z)\n # Clamped for numerical stability\n log_std = self.log_std(latent_z).clamp(-4, 15)\n std = torch.exp(log_std)\n # shape of mean, std: (state.shape[:-1], latent_dim)\n\n latent_z = mean + std * torch.randn_like(std) # (state.shape[:-1], latent_dim)\n\n reconstruction = self.decode(state, latent_z) # (state.shape[:-1], action_dim)\n return reconstruction, mean, std\n\n def decode(\n self,\n state: torch.Tensor,\n latent_z: torch.Tensor | None = None,\n ) -> torch.Tensor:\n # decode(state) -> action\n if latent_z is None:\n # state.shape[0] may be batch_size\n # latent vector clipped to [-0.5, 0.5]\n device = torch_device(self)\n latent_z = (\n torch.randn(state.shape[:-1] + (self.latent_dim,)).to(device).clamp(-0.5, 0.5)\n )\n\n # decode z with state!\n return self.max_action * torch.tanh(self.decoder(torch.cat([state, latent_z], -1)))\n" }, { "path": "tianshou/utils/net/discrete.py", "content": "from collections.abc import Sequence\nfrom typing import Any, TypeVar\n\nimport numpy as np\nimport torch\nimport torch.nn.functional as F\nfrom torch import nn\n\nfrom tianshou.data import Batch, to_torch\nfrom tianshou.data.types import TObs\nfrom tianshou.utils.net.common import (\n MLP,\n AbstractDiscreteActor,\n ModuleWithVectorOutput,\n TActionShape,\n)\nfrom tianshou.utils.torch_utils import torch_device\n\nT = TypeVar(\"T\")\n\n\ndef dist_fn_categorical_from_logits(\n logits: torch.Tensor,\n) -> torch.distributions.Categorical:\n \"\"\"Default distribution function for categorical actors.\"\"\"\n return torch.distributions.Categorical(logits=logits)\n\n\nclass DiscreteActor(AbstractDiscreteActor):\n \"\"\"\n Generic discrete actor which uses a preprocessing network to generate a latent representation\n which is subsequently passed to an MLP to compute the output.\n\n For common output semantics, see :class:`DiscreteActorInterface`.\n \"\"\"\n\n def __init__(\n self,\n *,\n preprocess_net: ModuleWithVectorOutput,\n action_shape: TActionShape,\n hidden_sizes: Sequence[int] = (),\n softmax_output: bool = True,\n ) -> None:\n \"\"\"\n :param preprocess_net: the preprocessing network, which outputs a vector of a known dimension;\n typically an instance of :class:`~tianshou.utils.net.common.Net`.\n :param action_shape: a sequence of int for the shape of action.\n :param hidden_sizes: a sequence of int for constructing the MLP after\n preprocess_net. Default to empty sequence (where the MLP now contains\n only a single linear layer).\n :param softmax_output: whether to apply a softmax layer over the last\n layer's output.\n \"\"\"\n output_dim = int(np.prod(action_shape))\n super().__init__(output_dim)\n self.preprocess = preprocess_net\n input_dim = preprocess_net.get_output_dim()\n self.last = MLP(\n input_dim=input_dim,\n output_dim=self.output_dim,\n hidden_sizes=hidden_sizes,\n )\n self.softmax_output = softmax_output\n\n def get_preprocess_net(self) -> ModuleWithVectorOutput:\n return self.preprocess\n\n def forward(\n self,\n obs: TObs,\n state: T | None = None,\n info: dict[str, Any] | None = None,\n ) -> tuple[torch.Tensor, T | None]:\n r\"\"\"Mapping: (s_B, ...) -> action_values_BA, hidden_state_BH | None.\n\n\n Returns a tensor representing the values of each action, i.e, of shape\n `(n_actions, )` (see class docstring for more info on the meaning of that), and\n a hidden state (which may be None). If `self.softmax_output` is True, they are the\n probabilities for taking each action. Otherwise, they will be action values.\n The hidden state is only\n not None if a recurrent net is used as part of the learning algorithm.\n \"\"\"\n x, hidden_BH = self.preprocess(obs, state)\n x = self.last(x)\n if self.softmax_output:\n x = F.softmax(x, dim=-1)\n # If we computed softmax, output is probabilities, otherwise it's the non-normalized action values\n output_BA = x\n return output_BA, hidden_BH\n\n\nclass DiscreteCritic(ModuleWithVectorOutput):\n \"\"\"Simple critic network for discrete action spaces.\n\n :param preprocess_net: the preprocessing network, which outputs a vector of a known dimension;\n typically an instance of :class:`~tianshou.utils.net.common.Net`.\n :param hidden_sizes: a sequence of int for constructing the MLP after\n preprocess_net. Default to empty sequence (where the MLP now contains\n only a single linear layer).\n :param last_size: the output dimension of Critic network.\n \"\"\"\n\n def __init__(\n self,\n *,\n preprocess_net: ModuleWithVectorOutput,\n hidden_sizes: Sequence[int] = (),\n last_size: int = 1,\n ) -> None:\n super().__init__(output_dim=last_size)\n self.preprocess = preprocess_net\n input_dim = preprocess_net.get_output_dim()\n self.last = MLP(input_dim=input_dim, output_dim=last_size, hidden_sizes=hidden_sizes)\n\n def forward(\n self, obs: TObs, state: T | None = None, info: dict[str, Any] | None = None\n ) -> torch.Tensor:\n \"\"\"Mapping: s_B -> V(s)_B.\"\"\"\n # TODO: don't use this mechanism for passing state\n logits, _ = self.preprocess(obs, state=state)\n return self.last(logits)\n\n\nclass CosineEmbeddingNetwork(nn.Module):\n \"\"\"Cosine embedding network for IQN. Convert a scalar in [0, 1] to a list of n-dim vectors.\n\n :param num_cosines: the number of cosines used for the embedding.\n :param embedding_dim: the dimension of the embedding/output.\n\n .. note::\n\n From https://github.com/ku2482/fqf-iqn-qrdqn.pytorch/blob/master\n /fqf_iqn_qrdqn/network.py .\n \"\"\"\n\n def __init__(self, num_cosines: int, embedding_dim: int) -> None:\n super().__init__()\n self.net = nn.Sequential(nn.Linear(num_cosines, embedding_dim), nn.ReLU())\n self.num_cosines = num_cosines\n self.embedding_dim = embedding_dim\n\n def forward(self, taus: torch.Tensor) -> torch.Tensor:\n batch_size = taus.shape[0]\n N = taus.shape[1]\n # Calculate i * \\pi (i=1,...,N).\n i_pi = np.pi * torch.arange(\n start=1,\n end=self.num_cosines + 1,\n dtype=taus.dtype,\n device=taus.device,\n ).view(1, 1, self.num_cosines)\n # Calculate cos(i * \\pi * \\tau).\n cosines = torch.cos(taus.view(batch_size, N, 1) * i_pi).view(\n batch_size * N,\n self.num_cosines,\n )\n # Calculate embeddings of taus.\n return self.net(cosines).view(batch_size, N, self.embedding_dim)\n\n\nclass ImplicitQuantileNetwork(DiscreteCritic):\n \"\"\"Implicit Quantile Network.\n\n :param preprocess_net: a self-defined preprocess_net which output a\n flattened hidden state.\n :param action_shape: a sequence of int for the shape of action.\n :param hidden_sizes: a sequence of int for constructing the MLP after\n preprocess_net. Default to empty sequence (where the MLP now contains\n only a single linear layer).\n :param num_cosines: the number of cosines to use for cosine embedding.\n Default to 64.\n\n .. note::\n\n Although this class inherits Critic, it is actually a quantile Q-Network\n with output shape (batch_size, action_dim, sample_size).\n\n The second item of the first return value is tau vector.\n \"\"\"\n\n def __init__(\n self,\n *,\n preprocess_net: ModuleWithVectorOutput,\n action_shape: TActionShape,\n hidden_sizes: Sequence[int] = (),\n num_cosines: int = 64,\n ) -> None:\n last_size = int(np.prod(action_shape))\n super().__init__(\n preprocess_net=preprocess_net,\n hidden_sizes=hidden_sizes,\n last_size=last_size,\n )\n self.input_dim = preprocess_net.get_output_dim()\n self.embed_model = CosineEmbeddingNetwork(num_cosines, self.input_dim)\n\n def forward( # type: ignore\n self,\n obs: np.ndarray | torch.Tensor,\n sample_size: int,\n **kwargs: Any,\n ) -> tuple[Any, torch.Tensor]:\n r\"\"\"Mapping: s -> Q(s, \\*).\"\"\"\n logits, hidden = self.preprocess(obs, state=kwargs.get(\"state\"))\n # Sample fractions.\n batch_size = logits.size(0)\n taus = torch.rand(batch_size, sample_size, dtype=logits.dtype, device=logits.device)\n embedding = (logits.unsqueeze(1) * self.embed_model(taus)).view(\n batch_size * sample_size,\n -1,\n )\n out = self.last(embedding).view(batch_size, sample_size, -1).transpose(1, 2)\n return (out, taus), hidden\n\n\nclass FractionProposalNetwork(nn.Module):\n \"\"\"Fraction proposal network for FQF.\n\n :param num_fractions: the number of factions to propose.\n :param embedding_dim: the dimension of the embedding/input.\n\n .. note::\n\n Adapted from https://github.com/ku2482/fqf-iqn-qrdqn.pytorch/blob/master\n /fqf_iqn_qrdqn/network.py .\n \"\"\"\n\n def __init__(self, num_fractions: int, embedding_dim: int) -> None:\n super().__init__()\n self.net = nn.Linear(embedding_dim, num_fractions)\n torch.nn.init.xavier_uniform_(self.net.weight, gain=0.01)\n torch.nn.init.constant_(self.net.bias, 0)\n self.num_fractions = num_fractions\n self.embedding_dim = embedding_dim\n\n def forward(\n self,\n obs_embeddings: torch.Tensor,\n ) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor]:\n # Calculate (log of) probabilities q_i in the paper.\n dist = torch.distributions.Categorical(logits=self.net(obs_embeddings))\n taus_1_N = torch.cumsum(dist.probs, dim=1)\n # Calculate \\tau_i (i=0,...,N).\n taus = F.pad(taus_1_N, (1, 0))\n # Calculate \\hat \\tau_i (i=0,...,N-1).\n tau_hats = (taus[:, :-1] + taus[:, 1:]).detach() / 2.0\n # Calculate entropies of value distributions.\n entropies = dist.entropy()\n return taus, tau_hats, entropies\n\n\nclass FullQuantileFunction(ImplicitQuantileNetwork):\n \"\"\"Full(y parameterized) Quantile Function.\n\n :param preprocess_net: a self-defined preprocess_net which output a\n flattened hidden state.\n :param action_shape: a sequence of int for the shape of action.\n :param hidden_sizes: a sequence of int for constructing the MLP after\n preprocess_net. Default to empty sequence (where the MLP now contains\n only a single linear layer).\n :param num_cosines: the number of cosines to use for cosine embedding.\n Default to 64.\n\n .. note::\n\n The first return value is a tuple of (quantiles, fractions, quantiles_tau),\n where fractions is a Batch(taus, tau_hats, entropies).\n \"\"\"\n\n def __init__(\n self,\n *,\n preprocess_net: ModuleWithVectorOutput,\n action_shape: TActionShape,\n hidden_sizes: Sequence[int] = (),\n num_cosines: int = 64,\n ) -> None:\n super().__init__(\n preprocess_net=preprocess_net,\n action_shape=action_shape,\n hidden_sizes=hidden_sizes,\n num_cosines=num_cosines,\n )\n\n def _compute_quantiles(self, obs: torch.Tensor, taus: torch.Tensor) -> torch.Tensor:\n batch_size, sample_size = taus.shape\n embedding = (obs.unsqueeze(1) * self.embed_model(taus)).view(batch_size * sample_size, -1)\n return self.last(embedding).view(batch_size, sample_size, -1).transpose(1, 2)\n\n def forward( # type: ignore\n self,\n obs: np.ndarray | torch.Tensor,\n propose_model: FractionProposalNetwork,\n fractions: Batch | None = None,\n **kwargs: Any,\n ) -> tuple[Any, torch.Tensor]:\n r\"\"\"Mapping: s -> Q(s, \\*).\"\"\"\n logits, hidden = self.preprocess(obs, state=kwargs.get(\"state\"))\n # Propose fractions\n if fractions is None:\n taus, tau_hats, entropies = propose_model(logits.detach())\n fractions = Batch(taus=taus, tau_hats=tau_hats, entropies=entropies)\n else:\n taus, tau_hats = fractions.taus, fractions.tau_hats\n quantiles = self._compute_quantiles(logits, tau_hats)\n # Calculate quantiles_tau for computing fraction grad\n quantiles_tau = None\n if self.training:\n with torch.no_grad():\n quantiles_tau = self._compute_quantiles(logits, taus[:, 1:-1])\n return (quantiles, fractions, quantiles_tau), hidden\n\n\nclass NoisyLinear(nn.Module):\n \"\"\"Implementation of Noisy Networks. arXiv:1706.10295.\n\n :param in_features: the number of input features.\n :param out_features: the number of output features.\n :param noisy_std: initial standard deviation of noisy linear layers.\n\n .. note::\n\n Adapted from https://github.com/ku2482/fqf-iqn-qrdqn.pytorch/blob/master\n /fqf_iqn_qrdqn/network.py .\n \"\"\"\n\n def __init__(self, in_features: int, out_features: int, noisy_std: float = 0.5) -> None:\n super().__init__()\n\n # Learnable parameters.\n self.mu_W = nn.Parameter(torch.FloatTensor(out_features, in_features))\n self.sigma_W = nn.Parameter(torch.FloatTensor(out_features, in_features))\n self.mu_bias = nn.Parameter(torch.FloatTensor(out_features))\n self.sigma_bias = nn.Parameter(torch.FloatTensor(out_features))\n\n # Factorized noise parameters.\n self.eps_p = nn.Parameter(torch.FloatTensor(in_features), requires_grad=False)\n self.eps_q = nn.Parameter(torch.FloatTensor(out_features), requires_grad=False)\n\n self.in_features = in_features\n self.out_features = out_features\n self.sigma = noisy_std\n\n self.reset()\n self.sample()\n\n def reset(self) -> None:\n bound = 1 / np.sqrt(self.in_features)\n self.mu_W.data.uniform_(-bound, bound)\n self.mu_bias.data.uniform_(-bound, bound)\n self.sigma_W.data.fill_(self.sigma / np.sqrt(self.in_features))\n self.sigma_bias.data.fill_(self.sigma / np.sqrt(self.in_features))\n\n def f(self, x: torch.Tensor) -> torch.Tensor:\n x = torch.randn(x.size(0), device=x.device)\n return x.sign().mul_(x.abs().sqrt_())\n\n # TODO: rename or change functionality? Usually sample is not an inplace operation...\n def sample(self) -> None:\n self.eps_p.copy_(self.f(self.eps_p))\n self.eps_q.copy_(self.f(self.eps_q))\n\n def forward(self, x: torch.Tensor) -> torch.Tensor:\n if self.training:\n weight = self.mu_W + self.sigma_W * (self.eps_q.ger(self.eps_p))\n bias = self.mu_bias + self.sigma_bias * self.eps_q.clone()\n else:\n weight = self.mu_W\n bias = self.mu_bias\n\n return F.linear(x, weight, bias)\n\n\nclass IntrinsicCuriosityModule(nn.Module):\n \"\"\"Implementation of Intrinsic Curiosity Module. arXiv:1705.05363.\n\n :param feature_net: a self-defined feature_net which output a\n flattened hidden state.\n :param feature_dim: input dimension of the feature net.\n :param action_dim: dimension of the action space.\n :param hidden_sizes: hidden layer sizes for forward and inverse models.\n \"\"\"\n\n def __init__(\n self,\n *,\n feature_net: nn.Module,\n feature_dim: int,\n action_dim: int,\n hidden_sizes: Sequence[int] = (),\n ) -> None:\n super().__init__()\n self.feature_net = feature_net\n self.forward_model = MLP(\n input_dim=feature_dim + action_dim,\n output_dim=feature_dim,\n hidden_sizes=hidden_sizes,\n )\n self.inverse_model = MLP(\n input_dim=feature_dim * 2,\n output_dim=action_dim,\n hidden_sizes=hidden_sizes,\n )\n self.feature_dim = feature_dim\n self.action_dim = action_dim\n\n def forward(\n self,\n s1: np.ndarray | torch.Tensor,\n act: np.ndarray | torch.Tensor,\n s2: np.ndarray | torch.Tensor,\n **kwargs: Any,\n ) -> tuple[torch.Tensor, torch.Tensor]:\n r\"\"\"Mapping: s1, act, s2 -> mse_loss, act_hat.\"\"\"\n device = torch_device(self)\n s1 = to_torch(s1, dtype=torch.float32, device=device)\n s2 = to_torch(s2, dtype=torch.float32, device=device)\n phi1, phi2 = self.feature_net(s1), self.feature_net(s2)\n act = to_torch(act, dtype=torch.long, device=device)\n phi2_hat = self.forward_model(\n torch.cat([phi1, F.one_hot(act, num_classes=self.action_dim)], dim=1),\n )\n mse_loss = 0.5 * F.mse_loss(phi2_hat, phi2, reduction=\"none\").sum(1)\n act_hat = self.inverse_model(torch.cat([phi1, phi2], dim=1))\n return mse_loss, act_hat\n" }, { "path": "tianshou/utils/print.py", "content": "import pprint\nfrom collections.abc import Sequence\nfrom dataclasses import asdict, dataclass\n\n\n@dataclass\nclass DataclassPPrintMixin:\n def pprint_asdict(self, exclude_fields: Sequence[str] | None = None, indent: int = 4) -> None:\n \"\"\"Pretty-print the object as a dict, excluding specified fields.\n\n :param exclude_fields: A sequence of field names to exclude from the output.\n If None, no fields are excluded.\n :param indent: The indentation to use when pretty-printing.\n \"\"\"\n print(self.pprints_asdict(exclude_fields=exclude_fields, indent=indent))\n\n def pprints_asdict(self, exclude_fields: Sequence[str] | None = None, indent: int = 4) -> str:\n \"\"\"String corresponding to pretty-print of the object as a dict, excluding specified fields.\n\n :param exclude_fields: A sequence of field names to exclude from the output.\n If None, no fields are excluded.\n :param indent: The indentation to use when pretty-printing.\n \"\"\"\n prefix = f\"{self.__class__.__name__}\\n----------------------------------------\\n\"\n print_dict = asdict(self)\n exclude_fields = exclude_fields or []\n for field in exclude_fields:\n print_dict.pop(field, None)\n return prefix + pprint.pformat(print_dict, indent=indent)\n" }, { "path": "tianshou/utils/progress_bar.py", "content": "from typing import Any\n\ntqdm_config = {\n \"dynamic_ncols\": True,\n \"ascii\": True,\n}\n\n\nclass DummyTqdm:\n \"\"\"A dummy tqdm class that keeps stats but without progress bar.\n\n It supports ``__enter__`` and ``__exit__``, update and a dummy\n ``set_postfix``, which is the interface that trainers use.\n\n .. note::\n\n Using ``disable=True`` in tqdm config results in infinite loop, thus\n this class is created. See the discussion at #641 for details.\n \"\"\"\n\n def __init__(self, total: int, **kwargs: Any):\n self.total = total\n self.n = 0\n\n def set_postfix(self, **kwargs: Any) -> None:\n pass\n\n def update(self, n: int = 1) -> None:\n self.n += n\n\n def __enter__(self) -> \"DummyTqdm\":\n return self\n\n def __exit__(self, *args: Any, **kwargs: Any) -> None:\n pass\n" }, { "path": "tianshou/utils/space_info.py", "content": "from collections.abc import Sequence\nfrom dataclasses import dataclass\nfrom typing import Any, Self\n\nimport gymnasium as gym\nimport numpy as np\nfrom gymnasium import spaces\nfrom sensai.util.string import ToStringMixin\n\n\n@dataclass(kw_only=True)\nclass ActionSpaceInfo(ToStringMixin):\n \"\"\"A data structure for storing the different attributes of the action space.\"\"\"\n\n action_shape: int | Sequence[int]\n \"\"\"The shape of the action space.\"\"\"\n min_action: float\n \"\"\"The smallest allowable action or in the continuous case the lower bound for allowable action value.\"\"\"\n max_action: float\n \"\"\"The largest allowable action or in the continuous case the upper bound for allowable action value.\"\"\"\n\n @property\n def action_dim(self) -> int:\n \"\"\"Return the number of distinct actions (must be greater than zero) an agent can take it its action space.\"\"\"\n if isinstance(self.action_shape, int):\n return self.action_shape\n else:\n return int(np.prod(self.action_shape))\n\n @classmethod\n def from_space(cls, space: spaces.Space) -> Self:\n \"\"\"Instantiate the `ActionSpaceInfo` object from a `Space`, supported spaces are Box and Discrete.\"\"\"\n if isinstance(space, spaces.Box):\n return cls(\n action_shape=space.shape,\n min_action=float(np.min(space.low)),\n max_action=float(np.max(space.high)),\n )\n elif isinstance(space, spaces.Discrete):\n return cls(\n action_shape=int(space.n),\n min_action=float(space.start),\n max_action=float(space.start + space.n - 1),\n )\n else:\n raise ValueError(\n f\"Unsupported space type: {space.__class__}. Currently supported types are Discrete and Box.\",\n )\n\n def _tostring_additional_entries(self) -> dict[str, Any]:\n return {\"action_dim\": self.action_dim}\n\n\n@dataclass(kw_only=True)\nclass ObservationSpaceInfo(ToStringMixin):\n \"\"\"A data structure for storing the different attributes of the observation space.\"\"\"\n\n obs_shape: int | Sequence[int]\n \"\"\"The shape of the observation space.\"\"\"\n\n @property\n def obs_dim(self) -> int:\n \"\"\"Return the number of distinct features (must be greater than zero) or dimensions in the observation space.\"\"\"\n if isinstance(self.obs_shape, int):\n return self.obs_shape\n else:\n return int(np.prod(self.obs_shape))\n\n @classmethod\n def from_space(cls, space: spaces.Space) -> Self:\n \"\"\"Instantiate the `ObservationSpaceInfo` object from a `Space`, supported spaces are Box and Discrete.\"\"\"\n if isinstance(space, spaces.Box):\n return cls(\n obs_shape=space.shape,\n )\n elif isinstance(space, spaces.Discrete):\n return cls(\n obs_shape=int(space.n),\n )\n else:\n raise ValueError(\n f\"Unsupported space type: {space.__class__}. Currently supported types are Discrete and Box.\",\n )\n\n def _tostring_additional_entries(self) -> dict[str, Any]:\n return {\"obs_dim\": self.obs_dim}\n\n\n@dataclass(kw_only=True)\nclass SpaceInfo(ToStringMixin):\n \"\"\"A data structure for storing the attributes of both the action and observation space.\"\"\"\n\n action_info: ActionSpaceInfo\n \"\"\"Stores the attributes of the action space.\"\"\"\n observation_info: ObservationSpaceInfo\n \"\"\"Stores the attributes of the observation space.\"\"\"\n\n @classmethod\n def from_env(cls, env: gym.Env) -> Self:\n \"\"\"Instantiate the `SpaceInfo` object from `gym.Env.action_space` and `gym.Env.observation_space`.\"\"\"\n return cls.from_spaces(env.action_space, env.observation_space)\n\n @classmethod\n def from_spaces(cls, action_space: spaces.Space, observation_space: spaces.Space) -> Self:\n \"\"\"Instantiate the `SpaceInfo` object from `ActionSpaceInfo` and `ObservationSpaceInfo`.\"\"\"\n action_info = ActionSpaceInfo.from_space(action_space)\n observation_info = ObservationSpaceInfo.from_space(observation_space)\n\n return cls(\n action_info=action_info,\n observation_info=observation_info,\n )\n" }, { "path": "tianshou/utils/statistics.py", "content": "from numbers import Number\n\nimport numpy as np\nimport torch\n\n\nclass MovAvg:\n \"\"\"Class for moving average.\n\n It will automatically exclude the infinity and NaN. Usage:\n ::\n\n >>> stat = MovAvg(size=66)\n >>> stat.add(torch.tensor(5))\n 5.0\n >>> stat.add(float('inf')) # which will not add to stat\n 5.0\n >>> stat.add([6, 7, 8])\n 6.5\n >>> stat.get()\n 6.5\n >>> print(f'{stat.mean():.2f}±{stat.std():.2f}')\n 6.50±1.12\n \"\"\"\n\n def __init__(self, size: int = 100) -> None:\n super().__init__()\n self.size = size\n self.cache: list[np.number] = []\n self.banned = [np.inf, np.nan, -np.inf]\n\n def add(\n self,\n data_array: Number | float | np.number | list | np.ndarray | torch.Tensor,\n ) -> float:\n \"\"\"Add a scalar into :class:`MovAvg`.\n\n You can add ``torch.Tensor`` with only one element, a python scalar, or\n a list of python scalar.\n \"\"\"\n if isinstance(data_array, torch.Tensor):\n data_array = data_array.flatten().cpu().numpy()\n if np.isscalar(data_array):\n data_array = [data_array]\n for number in data_array: # type: ignore\n if number not in self.banned:\n self.cache.append(number)\n if self.size > 0 and len(self.cache) > self.size:\n self.cache = self.cache[-self.size :]\n return self.get()\n\n def get(self) -> float:\n \"\"\"Get the average.\"\"\"\n if len(self.cache) == 0:\n return 0.0\n return float(np.mean(self.cache)) # type: ignore\n\n def mean(self) -> float:\n \"\"\"Get the average. Same as :meth:`get`.\"\"\"\n return self.get()\n\n def std(self) -> float:\n \"\"\"Get the standard deviation.\"\"\"\n if len(self.cache) == 0:\n return 0.0\n return float(np.std(self.cache)) # type: ignore\n\n\nclass RunningMeanStd:\n \"\"\"Calculates the running mean and std of a data stream.\n\n https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Parallel_algorithm\n\n :param mean: the initial mean estimation for data array. Default to 0.\n :param std: the initial standard error estimation for data array.\n :param clip_max: the maximum absolute value for data array. Default to\n 10.0.\n :param epsilon: To avoid division by zero.\n \"\"\"\n\n def __init__(\n self,\n mean: float | np.ndarray = 0.0,\n std: float | np.ndarray = 1.0,\n clip_max: float | None = 10.0,\n epsilon: float = np.finfo(np.float32).eps.item(),\n ) -> None:\n self.mean, self.var = mean, std\n self.clip_max = clip_max\n self.count = 0\n self.eps = epsilon\n\n def norm(self, data_array: float | np.ndarray) -> float | np.ndarray:\n data_array = (data_array - self.mean) / np.sqrt(self.var + self.eps)\n if self.clip_max:\n data_array = np.clip(data_array, -self.clip_max, self.clip_max)\n return data_array\n\n def update(self, data_array: np.ndarray) -> None:\n \"\"\"Add a batch of item into RMS with the same shape, modify mean/var/count.\"\"\"\n batch_mean, batch_var = np.mean(data_array, axis=0), np.var(data_array, axis=0)\n batch_count = len(data_array)\n\n delta = batch_mean - self.mean\n total_count = self.count + batch_count\n\n new_mean = self.mean + delta * batch_count / total_count\n m_a = self.var * self.count\n m_b = batch_var * batch_count\n m_2 = m_a + m_b + delta**2 * self.count * batch_count / total_count\n new_var = m_2 / total_count\n\n self.mean, self.var = new_mean, new_var\n self.count = total_count\n" }, { "path": "tianshou/utils/torch_utils.py", "content": "from collections.abc import Iterator\nfrom contextlib import contextmanager\nfrom typing import TYPE_CHECKING, overload\n\nimport torch\nimport torch.distributions as dist\nfrom gymnasium import spaces\nfrom torch import nn\n\nif TYPE_CHECKING:\n from tianshou.algorithm import algorithm_base\n\n\n@contextmanager\ndef torch_train_mode(module: nn.Module, enabled: bool = True) -> Iterator[None]:\n \"\"\"Temporarily switch to `module.training=enabled`, affecting things like `BatchNormalization`.\"\"\"\n original_mode = module.training\n try:\n module.train(enabled)\n yield\n finally:\n module.train(original_mode)\n\n\n@contextmanager\ndef policy_within_training_step(\n policy: \"algorithm_base.Policy\", enabled: bool = True\n) -> Iterator[None]:\n \"\"\"Temporarily switch to `policy.is_within_training_step=enabled`.\n\n Enabling this ensures that the policy is able to adapt its behavior,\n allowing it to differentiate between training and inference/evaluation,\n e.g., to sample actions instead of using the most probable action (where applicable)\n Note that for rollout, which also happens within a training step, one would usually want\n the wrapped torch module to be in evaluation mode, which can be achieved using\n `with torch_train_mode(policy, False)`. For subsequent gradient updates, the policy should be both\n within training step and in torch train mode.\n \"\"\"\n original_mode = policy.is_within_training_step\n try:\n policy.is_within_training_step = enabled\n yield\n finally:\n policy.is_within_training_step = original_mode\n\n\n@overload\ndef create_uniform_action_dist(action_space: spaces.Box, batch_size: int = 1) -> dist.Uniform: ...\n\n\n@overload\ndef create_uniform_action_dist(\n action_space: spaces.Discrete,\n batch_size: int = 1,\n) -> dist.Categorical: ...\n\n\ndef create_uniform_action_dist(\n action_space: spaces.Box | spaces.Discrete,\n batch_size: int = 1,\n) -> dist.Uniform | dist.Categorical:\n \"\"\"Create a Distribution such that sampling from it is equivalent to sampling a batch with `action_space.sample()`.\n\n :param action_space: the environment's action_space.\n :param batch_size: The number of environments or batch size for sampling.\n :return: A PyTorch distribution for sampling actions.\n \"\"\"\n if isinstance(action_space, spaces.Box):\n low = torch.FloatTensor(action_space.low).unsqueeze(0).repeat(batch_size, 1)\n high = torch.FloatTensor(action_space.high).unsqueeze(0).repeat(batch_size, 1)\n return dist.Uniform(low, high)\n\n elif isinstance(action_space, spaces.Discrete):\n return dist.Categorical(torch.ones(batch_size, int(action_space.n)))\n\n else:\n raise ValueError(f\"Unsupported action space type: {type(action_space)}\")\n\n\ndef torch_device(module: torch.nn.Module) -> torch.device:\n \"\"\"Gets the device of a torch module by retrieving the device of the parameters.\n\n If parameters are empty, it returns the CPU device as a fallback.\n \"\"\"\n try:\n return next(module.parameters()).device\n except StopIteration:\n return torch.device(\"cpu\")\n" }, { "path": "tianshou/utils/warning.py", "content": "import warnings\n\nwarnings.simplefilter(\"once\", DeprecationWarning)\n\n\ndef deprecation(msg: str) -> None:\n \"\"\"Deprecation warning wrapper.\"\"\"\n warnings.warn(msg, category=DeprecationWarning, stacklevel=2)\n" } ]