Showing preview only (632K chars total). Download the full file or copy to clipboard to get everything.
Repository: iterative/dvclive
Branch: main
Commit: 5ced79f3938f
Files: 80
Total size: 602.4 KB
Directory structure:
gitextract_j98cv22l/
├── .cruft.json
├── .gitattributes
├── .github/
│ ├── PULL_REQUEST_TEMPLATE.md
│ ├── codecov.yml
│ ├── dependabot.yml
│ └── workflows/
│ ├── release.yml
│ ├── tests.yml
│ └── update-template.yaml
├── .gitignore
├── .pre-commit-config.yaml
├── CODE_OF_CONDUCT.rst
├── CONTRIBUTING.rst
├── LICENSE
├── README.md
├── examples/
│ ├── DVCLive-Evidently.ipynb
│ ├── DVCLive-Fabric.ipynb
│ ├── DVCLive-HuggingFace.ipynb
│ ├── DVCLive-PyTorch-Lightning.ipynb
│ ├── DVCLive-Quickstart.ipynb
│ ├── DVCLive-YOLO.ipynb
│ └── DVCLive-scikit-learn.ipynb
├── noxfile.py
├── pyproject.toml
├── src/
│ └── dvclive/
│ ├── __init__.py
│ ├── dvc.py
│ ├── env.py
│ ├── error.py
│ ├── fabric.py
│ ├── fastai.py
│ ├── huggingface.py
│ ├── keras.py
│ ├── lgbm.py
│ ├── lightning.py
│ ├── live.py
│ ├── monitor_system.py
│ ├── optuna.py
│ ├── plots/
│ │ ├── __init__.py
│ │ ├── base.py
│ │ ├── custom.py
│ │ ├── image.py
│ │ ├── metric.py
│ │ ├── sklearn.py
│ │ └── utils.py
│ ├── py.typed
│ ├── report.py
│ ├── serialize.py
│ ├── studio.py
│ ├── utils.py
│ ├── vscode.py
│ └── xgb.py
└── tests/
├── __init__.py
├── conftest.py
├── frameworks/
│ ├── test_fabric.py
│ ├── test_fastai.py
│ ├── test_huggingface.py
│ ├── test_keras.py
│ ├── test_lgbm.py
│ ├── test_lightning.py
│ ├── test_optuna.py
│ └── test_xgboost.py
├── plots/
│ ├── test_custom.py
│ ├── test_image.py
│ ├── test_metric.py
│ └── test_sklearn.py
├── test_cleanup.py
├── test_context_manager.py
├── test_dvc.py
├── test_log_artifact.py
├── test_log_metric.py
├── test_log_param.py
├── test_logging.py
├── test_make_dvcyaml.py
├── test_make_report.py
├── test_make_summary.py
├── test_monitor_system.py
├── test_post_to_studio.py
├── test_resume.py
├── test_step.py
├── test_utils.py
└── test_vscode.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .cruft.json
================================================
{
"template": "https://github.com/iterative/py-template",
"commit": "e4ec95f4cfd03d4af0a8604d462ee11d07d63b42",
"checkout": null,
"context": {
"cookiecutter": {
"project_name": "dvclive",
"package_name": "dvclive",
"friendly_name": "dvclive",
"author": "Iterative",
"email": "support@dvc.org",
"github_user": "iterative",
"version": "0.0.0",
"copyright_year": "2022",
"license": "Apache-2.0",
"docs": "False",
"short_description": "Metric logger for ML projects.",
"development_status": "Development Status :: 4 - Beta",
"_template": "https://github.com/iterative/py-template"
}
},
"directory": null
}
================================================
FILE: .gitattributes
================================================
* text=auto eol=lf
================================================
FILE: .github/PULL_REQUEST_TEMPLATE.md
================================================
- [ ] ❗ I have followed the [Contributing to DVCLive](https://github.com/iterative/dvclive/blob/main/CONTRIBUTING.rst) guide.
- [ ] 📖 If this PR requires [documentation](https://dvc.org/doc) updates, I have created a separate PR (or issue, at least) in [dvc.org](https://github.com/iterative/dvc.org) and linked it here.
Thank you for the contribution - we'll try to review it as soon as possible. 🙏
================================================
FILE: .github/codecov.yml
================================================
coverage:
status:
project:
default:
# auto compares coverage to the previous base commit
target: auto
# adjust accordingly based on how flaky your tests are
# this allows a 10% drop from the previous base commit coverage
threshold: 10%
# non-blocking status checks
informational: false
================================================
FILE: .github/dependabot.yml
================================================
version: 2
updates:
- directory: "/"
package-ecosystem: "pip"
schedule:
interval: "weekly"
labels:
- "maintenance"
- directory: "/"
package-ecosystem: "github-actions"
schedule:
interval: "weekly"
labels:
- "maintenance"
================================================
FILE: .github/workflows/release.yml
================================================
name: Release
on:
release:
types: [published]
workflow_dispatch:
env:
FORCE_COLOR: "1"
jobs:
release:
environment: pypi
permissions:
contents: read
id-token: write
runs-on: ubuntu-latest
steps:
- name: Check out the repository
uses: actions/checkout@v5
with:
fetch-depth: 0
- name: Set up Python 3.14
uses: actions/setup-python@v6
with:
python-version: '3.14'
- uses: astral-sh/setup-uv@v7
- name: Install nox
run: uv pip install --system nox --upgrade
- name: Build package
run: nox -s build
- name: Upload package
if: github.event_name == 'release'
uses: pypa/gh-action-pypi-publish@release/v1
================================================
FILE: .github/workflows/tests.yml
================================================
name: Tests
on:
push:
branches: [main]
pull_request:
workflow_dispatch:
env:
FORCE_COLOR: "1"
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true
jobs:
lint:
runs-on: ubuntu-latest
steps:
- name: Check out the repository
uses: actions/checkout@v5
with:
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.13"
- uses: astral-sh/setup-uv@v7
with:
enable-cache: false
- name: Install nox
run: uv pip install --system nox --upgrade
- uses: actions/cache@v5
with:
path: ~/.cache/pre-commit/
key: pre-commit-4|${{ env.pythonLocation }}|${{ hashFiles('.pre-commit-config.yaml') }}
- name: Lint code
run: nox -s lint
tests:
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
pyv: ["3.9", "3.10", "3.11", "3.12", "3.13", "3.14"]
type: [core_tests]
include:
- os: ubuntu-latest
pyv: "3.13"
type: tests
exclude:
- os: ubuntu-latest
pyv: "3.13"
type: core_tests
runs-on: ${{ matrix.os }}
steps:
- name: Check out the repository
uses: actions/checkout@v5
with:
fetch-depth: 0
- name: Set up Python ${{ matrix.pyv }}
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.pyv }}
- uses: astral-sh/setup-uv@v7
with:
enable-cache: false
- name: Install nox
run: uv pip install --system nox --upgrade
- name: Run tests
run: nox -s ${{ matrix.type }}-${{ matrix.pyv }} -- --cov-report=xml
- name: Build package
run: nox -s build
- name: Upload coverage report
uses: codecov/codecov-action@v5
check:
if: always()
needs: [lint, tests]
runs-on: ubuntu-latest
steps:
- uses: re-actors/alls-green@release/v1
with:
jobs: ${{ toJSON(needs) }}
================================================
FILE: .github/workflows/update-template.yaml
================================================
name: Update template
on:
schedule:
- cron: '5 1 * * *' # every day at 01:05
workflow_dispatch:
jobs:
update:
runs-on: ubuntu-latest
steps:
- name: Check out the repository
uses: actions/checkout@v5
- name: Update template
uses: iterative/py-template@main
================================================
FILE: .gitignore
================================================
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
.pybuilder/
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version
# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock
# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/
# Celery stuff
celerybeat-schedule
celerybeat.pid
# SageMath parsed files
*.sage.py
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
# pytype static type analyzer
.pytype/
# Cython debug symbols
cython_debug/
# Editors
.idea
.vscode
.dvc/
.dvcignore
src/dvclive/_dvclive_version.py
================================================
FILE: .pre-commit-config.yaml
================================================
default_language_version:
python: python3
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v6.0.0
hooks:
- id: check-added-large-files
- id: check-case-conflict
- id: check-docstring-first
- id: check-executables-have-shebangs
- id: check-json
- id: check-merge-conflict
args: ["--assume-in-merge"]
- id: check-toml
- id: check-yaml
- id: debug-statements
- id: end-of-file-fixer
- id: mixed-line-ending
args: ["--fix=lf"]
- id: sort-simple-yaml
- id: trailing-whitespace
- repo: https://github.com/codespell-project/codespell
rev: v2.4.1
hooks:
- id: codespell
additional_dependencies: ["tomli"]
exclude: >
(?x)^(
.*\.ipynb
)$
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: "v0.14.3"
hooks:
- id: ruff-check
args: [--fix, --exit-non-zero-on-fix]
- id: ruff-format
- repo: https://github.com/macisamuele/language-formatters-pre-commit-hooks
rev: v2.15.0
hooks:
- id: pretty-format-toml
args: [--autofix, --no-sort]
- id: pretty-format-yaml
args: [--autofix, --indent, '2', '--offset', '2', --preserve-quotes]
================================================
FILE: CODE_OF_CONDUCT.rst
================================================
Contributor Covenant Code of Conduct
====================================
Our Pledge
----------
We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socioeconomic status, nationality, personal appearance, race, religion, or sexual identity and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.
Our Standards
-------------
Examples of behavior that contributes to a positive environment for our community include:
- Demonstrating empathy and kindness toward other people
- Being respectful of differing opinions, viewpoints, and experiences
- Giving and gracefully accepting constructive feedback
- Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience
- Focusing on what is best not just for us as individuals, but for the overall community
Examples of unacceptable behavior include:
- The use of sexualized language or imagery, and sexual attention or
advances of any kind
- Trolling, insulting or derogatory comments, and personal or political attacks
- Public or private harassment
- Publishing others' private information, such as a physical or email
address, without their explicit permission
- Other conduct which could reasonably be considered inappropriate in a
professional setting
Enforcement Responsibilities
----------------------------
Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful.
Community leaders have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, and will communicate reasons for moderation decisions when appropriate.
Scope
-----
This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces. Examples of representing our community include using an official e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event.
Enforcement
-----------
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement at support@dvc.org. All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the reporter of any incident.
Enforcement Guidelines
----------------------
Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of this Code of Conduct:
1. Correction
~~~~~~~~~~~~~
**Community Impact**: Use of inappropriate language or other behavior deemed unprofessional or unwelcome in the community.
**Consequence**: A private, written warning from community leaders, providing clarity around the nature of the violation and an explanation of why the behavior was inappropriate. A public apology may be requested.
2. Warning
~~~~~~~~~~
**Community Impact**: A violation through a single incident or series of actions.
**Consequence**: A warning with consequences for continued behavior. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban.
3. Temporary Ban
~~~~~~~~~~~~~~~~
**Community Impact**: A serious violation of community standards, including sustained inappropriate behavior.
**Consequence**: A temporary ban from any sort of interaction or public communication with the community for a specified period of time. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban.
4. Permanent Ban
~~~~~~~~~~~~~~~~
**Community Impact**: Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals.
**Consequence**: A permanent ban from any sort of public interaction within the community.
Attribution
-----------
This Code of Conduct is adapted from the `Contributor Covenant <homepage_>`__, version 2.0,
available at https://www.contributor-covenant.org/version/2/0/code_of_conduct/.
Community Impact Guidelines were inspired by `Mozilla’s code of conduct enforcement ladder <https://github.com/mozilla/inclusion>`__.
.. _homepage: https://www.contributor-covenant.org
For answers to common questions about this code of conduct, see the FAQ at
https://www.contributor-covenant.org/faq. Translations are available at https://www.contributor-covenant.org/translations.
================================================
FILE: CONTRIBUTING.rst
================================================
Contributor Guide
=================
Thank you for your interest in improving this project.
This project is open-source under the `Apache 2.0 license`_ and
welcomes contributions in the form of bug reports, feature requests, and pull requests.
Here is a list of important resources for contributors:
- `Source Code`_
- `Issue Tracker`_
- `Code of Conduct`_
.. _Apache 2.0 license: https://opensource.org/licenses/Apache-2.0
.. _Source Code: https://github.com/iterative/dvclive
.. _Issue Tracker: https://github.com/iterative/dvclive/issues
How to report a bug
-------------------
Report bugs on the `Issue Tracker`_.
When filing an issue, make sure to answer these questions:
- Which operating system and Python version are you using?
- Which version of this project are you using?
- What did you do?
- What did you expect to see?
- What did you see instead?
The best way to get your bug fixed is to provide a test case,
and/or steps to reproduce the issue.
How to request a feature
------------------------
Request features on the `Issue Tracker`_.
How to set up your development environment
------------------------------------------
You need Python 3.9+.
- Clone the repository:
.. code:: console
$ git clone https://github.com/iterative/dvclive
$ cd dvclive
- Set up a virtual environment:
.. code:: console
$ python -m venv .venv
$ source .venv/bin/activate
Install in editable mode including development dependencies:
.. code:: console
$ pip install -e .[tests]
If you need to test against a specific framework, you can install it separately:
.. code:: console
$ pip install -e .[tests,tf]
$ pip install -e .[tests,optuna]
How to test the project
-----------------------
Run the full test suite:
.. code:: console
$ pytest -v tests
Tests are located in the ``tests`` directory,
and are written using the pytest_ testing framework.
.. _pytest: https://pytest.readthedocs.io/
How to submit changes
---------------------
Open a `pull request`_ to submit changes to this project.
Your pull request needs to meet the following guidelines for acceptance:
- The test suite must pass without errors and warnings.
- Include unit tests.
- If your changes add functionality, update the documentation accordingly.
Feel free to submit early, though—we can always iterate on this.
To run linting and code formatting checks, you can use `pre-commit`:
.. code:: console
$ pre-commit run --all-files
It is recommended to open an issue before starting work on anything.
This will allow a chance to talk it over with the owners and validate your approach.
.. _pull request: https://github.com/iterative/dvclive/pulls
.. github-only
.. _Code of Conduct: CODE_OF_CONDUCT.rst
================================================
FILE: LICENSE
================================================
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "{}"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2022 Iterative.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
================================================
FILE: README.md
================================================
# DVCLive
[](https://pypi.org/project/dvclive/)
[](https://pypi.org/project/dvclive/)
[](https://pypi.org/project/dvclive)
[](https://opensource.org/licenses/Apache-2.0)
[](https://github.com/iterative/dvclive/actions?workflow=Tests)
[](https://app.codecov.io/gh/iterative/dvclive)
[](https://github.com/pre-commit/pre-commit)
[](https://github.com/psf/black)
DVCLive is a Python library for logging machine learning metrics and other
metadata in simple file formats, which is fully compatible with DVC.
# [Documentation](https://dvc.org/doc/dvclive)
- [Get Started](https://dvc.org/doc/start/experiments)
- [How it Works](https://dvc.org/doc/dvclive/how-it-works)
- [API Reference](https://dvc.org/doc/dvclive/live)
- [Integrations](https://dvc.org/doc/dvclive/ml-frameworks)
______________________________________________________________________
# Quickstart
| Python API Overview | PyTorch Lightning | Scikit-learn | Ultralytics YOLO v8 |
|--------|--------|--------|--------|
| <a href="https://colab.research.google.com/github/iterative/dvclive/blob/main/examples/DVCLive-Quickstart.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" /></a> | <a href="https://colab.research.google.com/github/iterative/dvclive/blob/main/examples/DVCLive-PyTorch-Lightning.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" /></a> | <a href="https://colab.research.google.com/github/iterative/dvclive/blob/main/examples/DVCLive-scikit-learn.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" /></a> | <a href="https://colab.research.google.com/github/iterative/dvclive/blob/main/examples/DVCLive-YOLO.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" /></a> |
## Install *dvclive*
```console
$ pip install dvclive
```
## Initialize DVC Repository
```console
$ git init
$ dvc init
$ git commit -m "DVC init"
```
## Example code
Copy the snippet below into `train.py` for a basic API usage example:
```python
import time
import random
from dvclive import Live
params = {"learning_rate": 0.002, "optimizer": "Adam", "epochs": 20}
with Live() as live:
# log a parameters
for param in params:
live.log_param(param, params[param])
# simulate training
offset = random.uniform(0.2, 0.1)
for epoch in range(1, params["epochs"]):
fuzz = random.uniform(0.01, 0.1)
accuracy = 1 - (2 ** - epoch) - fuzz - offset
loss = (2 ** - epoch) + fuzz + offset
# log metrics to studio
live.log_metric("accuracy", accuracy)
live.log_metric("loss", loss)
live.next_step()
time.sleep(0.2)
```
See [Integrations](https://dvc.org/doc/dvclive/ml-frameworks) for examples using
DVCLive alongside different ML Frameworks.
## Running
Run this a couple of times to simulate multiple experiments:
```console
$ python train.py
$ python train.py
$ python train.py
...
```
## Comparing
DVCLive outputs can be rendered in different ways:
### DVC CLI
You can use [dvc exp show](https://dvc.org/doc/command-reference/exp/show) and
[dvc plots](https://dvc.org/doc/command-reference/plots) to compare and
visualize metrics, parameters and plots across experiments:
```console
$ dvc exp show
```
```
─────────────────────────────────────────────────────────────────────────────────────────────────────────────
Experiment Created train.accuracy train.loss val.accuracy val.loss step epochs
─────────────────────────────────────────────────────────────────────────────────────────────────────────────
workspace - 6.0109 0.23311 6.062 0.24321 6 7
master 08:50 PM - - - - - -
├── 4475845 [aulic-chiv] 08:56 PM 6.0109 0.23311 6.062 0.24321 6 7
├── 7d4cef7 [yarer-tods] 08:56 PM 4.8551 0.82012 4.5555 0.033533 4 5
└── d503f8e [curst-chad] 08:56 PM 4.9768 0.070585 4.0773 0.46639 4 5
─────────────────────────────────────────────────────────────────────────────────────────────────────────────
```
```console
$ dvc plots diff $(dvc exp list --names-only) --open
```
### DVC Extension for VS Code
Inside the
[DVC Extension for VS Code](https://marketplace.visualstudio.com/items?itemName=Iterative.dvc),
you can compare and visualize results using the
[Experiments](https://github.com/iterative/vscode-dvc/blob/main/extension/resources/walkthrough/experiments-table.md)
and
[Plots](https://github.com/iterative/vscode-dvc/blob/main/extension/resources/walkthrough/plots.md)
views:
While experiments are running, live updates will be displayed in both views.
### DVC Studio
If you push the results to [DVC Studio](https://dvc.org/doc/studio), you can
compare experiments against the entire repo history:
You can enable
[Studio Live Experiments](https://dvc.org/doc/studio/user-guide/projects-and-experiments/live-metrics-and-plots)
to see live updates while experiments are running.
______________________________________________________________________
# Comparison to related technologies
**DVCLive** is an *ML Logger*, similar to:
- [MLFlow](https://mlflow.org/)
- [Weights & Biases](https://wandb.ai/site)
- [Neptune](https://neptune.ai/)
The main differences with those *ML Loggers* are:
- **DVCLive** does not **require** any additional services or servers to run.
- **DVCLive** metrics, parameters, and plots are
[stored as plain text files](https://dvc.org/doc/dvclive/how-it-works#directory-structure)
that can be versioned by tools like Git or tracked as pointers to files in DVC
storage.
- **DVCLive** can save experiments or runs as
[hidden Git commits](https://dvc.org/doc/dvclive/how-it-works#track-the-results).
You can then use different [options](#comparing) to visualize the metrics,
parameters, and plots across experiments.
______________________________________________________________________
# Contributing
Contributions are very welcome. To learn more, see the
[Contributor Guide](CONTRIBUTING.rst).
# License
Distributed under the terms of the
[Apache 2.0 license](https://opensource.org/licenses/Apache-2.0), *dvclive* is
free and open source software.
================================================
FILE: examples/DVCLive-Evidently.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "WpfOFaqHcnAt"
},
"source": [
"# Install Evidently and DVC with DVCLive"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"executionInfo": {
"elapsed": 2337,
"status": "ok",
"timestamp": 1697468096427,
"user": {
"displayName": "Francesco Motoko",
"userId": "00974636158007469548"
},
"user_tz": -120
},
"id": "BqWpagFPZ45W"
},
"outputs": [],
"source": [
"!pip uninstall -q -y sqlalchemy pyarrow ipython-sql pandas-gbq"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"executionInfo": {
"elapsed": 33615,
"status": "ok",
"timestamp": 1697468130037,
"user": {
"displayName": "Francesco Motoko",
"userId": "00974636158007469548"
},
"user_tz": -120
},
"id": "DijzqeokW595"
},
"outputs": [],
"source": [
"%%capture\n",
"!pip install -q dvc==3.25.0 dvclive==3.0.1 evidently==0.4.5 pandas==1.5.3"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ZyZ2sX8GcvMU"
},
"source": [
"# Load the data"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"executionInfo": {
"elapsed": 1772,
"status": "ok",
"timestamp": 1697468131788,
"user": {
"displayName": "Francesco Motoko",
"userId": "00974636158007469548"
},
"user_tz": -120
},
"id": "ZUrB0D59XMDD",
"outputId": "9f6f5a3c-f856-4d56-a8fb-ec4483ec6127"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--2023-10-16 14:55:29-- https://archive.ics.uci.edu/static/public/275/bike+sharing+dataset.zip\n",
"Resolving archive.ics.uci.edu (archive.ics.uci.edu)... 128.195.10.252\n",
"Connecting to archive.ics.uci.edu (archive.ics.uci.edu)|128.195.10.252|:443... connected.\n",
"HTTP request sent, awaiting response... 200 OK\n",
"Length: unspecified\n",
"Saving to: ‘bike+sharing+dataset.zip’\n",
"\n",
"bike+sharing+datase [ <=> ] 273.43K 443KB/s in 0.6s \n",
"\n",
"2023-10-16 14:55:30 (443 KB/s) - ‘bike+sharing+dataset.zip’ saved [279992]\n",
"\n",
"Archive: bike+sharing+dataset.zip\n",
" inflating: Readme.txt \n",
" inflating: day.csv \n",
" inflating: hour.csv \n"
]
}
],
"source": [
"!mkdir raw_data && \\\n",
" cd raw_data && \\\n",
" wget https://archive.ics.uci.edu/static/public/275/bike+sharing+dataset.zip && \\\n",
" unzip bike+sharing+dataset.zip"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"executionInfo": {
"elapsed": 357,
"status": "ok",
"timestamp": 1697468132141,
"user": {
"displayName": "Francesco Motoko",
"userId": "00974636158007469548"
},
"user_tz": -120
},
"id": "P3XXcUrQY1EQ"
},
"outputs": [],
"source": [
"import pandas as pd"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 206
},
"executionInfo": {
"elapsed": 9,
"status": "ok",
"timestamp": 1697468132141,
"user": {
"displayName": "Francesco Motoko",
"userId": "00974636158007469548"
},
"user_tz": -120
},
"id": "MDK0xkdbYCWg",
"outputId": "ec8d2605-144d-45ff-b442-70ba858a44a3"
},
"outputs": [
{
"data": {
"text/html": [
"\n",
" <div id=\"df-f57eeeda-2494-42a9-8d15-33081c4de93d\" class=\"colab-df-container\">\n",
" <div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>instant</th>\n",
" <th>dteday</th>\n",
" <th>season</th>\n",
" <th>yr</th>\n",
" <th>mnth</th>\n",
" <th>holiday</th>\n",
" <th>weekday</th>\n",
" <th>workingday</th>\n",
" <th>weathersit</th>\n",
" <th>temp</th>\n",
" <th>atemp</th>\n",
" <th>hum</th>\n",
" <th>windspeed</th>\n",
" <th>casual</th>\n",
" <th>registered</th>\n",
" <th>cnt</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>1</td>\n",
" <td>2011-01-01</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>6</td>\n",
" <td>0</td>\n",
" <td>2</td>\n",
" <td>0.344167</td>\n",
" <td>0.363625</td>\n",
" <td>0.805833</td>\n",
" <td>0.160446</td>\n",
" <td>331</td>\n",
" <td>654</td>\n",
" <td>985</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>2</td>\n",
" <td>2011-01-02</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>2</td>\n",
" <td>0.363478</td>\n",
" <td>0.353739</td>\n",
" <td>0.696087</td>\n",
" <td>0.248539</td>\n",
" <td>131</td>\n",
" <td>670</td>\n",
" <td>801</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>3</td>\n",
" <td>2011-01-03</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>1</td>\n",
" <td>1</td>\n",
" <td>0.196364</td>\n",
" <td>0.189405</td>\n",
" <td>0.437273</td>\n",
" <td>0.248309</td>\n",
" <td>120</td>\n",
" <td>1229</td>\n",
" <td>1349</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>4</td>\n",
" <td>2011-01-04</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>2</td>\n",
" <td>1</td>\n",
" <td>1</td>\n",
" <td>0.200000</td>\n",
" <td>0.212122</td>\n",
" <td>0.590435</td>\n",
" <td>0.160296</td>\n",
" <td>108</td>\n",
" <td>1454</td>\n",
" <td>1562</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>5</td>\n",
" <td>2011-01-05</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>3</td>\n",
" <td>1</td>\n",
" <td>1</td>\n",
" <td>0.226957</td>\n",
" <td>0.229270</td>\n",
" <td>0.436957</td>\n",
" <td>0.186900</td>\n",
" <td>82</td>\n",
" <td>1518</td>\n",
" <td>1600</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>\n",
" <div class=\"colab-df-buttons\">\n",
"\n",
" <div class=\"colab-df-container\">\n",
" <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-f57eeeda-2494-42a9-8d15-33081c4de93d')\"\n",
" title=\"Convert this dataframe to an interactive table.\"\n",
" style=\"display:none;\">\n",
"\n",
" <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
" <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
" </svg>\n",
" </button>\n",
"\n",
" <style>\n",
" .colab-df-container {\n",
" display:flex;\n",
" gap: 12px;\n",
" }\n",
"\n",
" .colab-df-convert {\n",
" background-color: #E8F0FE;\n",
" border: none;\n",
" border-radius: 50%;\n",
" cursor: pointer;\n",
" display: none;\n",
" fill: #1967D2;\n",
" height: 32px;\n",
" padding: 0 0 0 0;\n",
" width: 32px;\n",
" }\n",
"\n",
" .colab-df-convert:hover {\n",
" background-color: #E2EBFA;\n",
" box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
" fill: #174EA6;\n",
" }\n",
"\n",
" .colab-df-buttons div {\n",
" margin-bottom: 4px;\n",
" }\n",
"\n",
" [theme=dark] .colab-df-convert {\n",
" background-color: #3B4455;\n",
" fill: #D2E3FC;\n",
" }\n",
"\n",
" [theme=dark] .colab-df-convert:hover {\n",
" background-color: #434B5C;\n",
" box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
" filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
" fill: #FFFFFF;\n",
" }\n",
" </style>\n",
"\n",
" <script>\n",
" const buttonEl =\n",
" document.querySelector('#df-f57eeeda-2494-42a9-8d15-33081c4de93d button.colab-df-convert');\n",
" buttonEl.style.display =\n",
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
"\n",
" async function convertToInteractive(key) {\n",
" const element = document.querySelector('#df-f57eeeda-2494-42a9-8d15-33081c4de93d');\n",
" const dataTable =\n",
" await google.colab.kernel.invokeFunction('convertToInteractive',\n",
" [key], {});\n",
" if (!dataTable) return;\n",
"\n",
" const docLinkHtml = 'Like what you see? Visit the ' +\n",
" '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
" + ' to learn more about interactive tables.';\n",
" element.innerHTML = '';\n",
" dataTable['output_type'] = 'display_data';\n",
" await google.colab.output.renderOutput(dataTable, element);\n",
" const docLink = document.createElement('div');\n",
" docLink.innerHTML = docLinkHtml;\n",
" element.appendChild(docLink);\n",
" }\n",
" </script>\n",
" </div>\n",
"\n",
"\n",
"<div id=\"df-1a8f4885-5b57-4a9a-bfc7-8d551bb879a7\">\n",
" <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-1a8f4885-5b57-4a9a-bfc7-8d551bb879a7')\"\n",
" title=\"Suggest charts.\"\n",
" style=\"display:none;\">\n",
"\n",
"<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
" width=\"24px\">\n",
" <g>\n",
" <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
" </g>\n",
"</svg>\n",
" </button>\n",
"\n",
"<style>\n",
" .colab-df-quickchart {\n",
" --bg-color: #E8F0FE;\n",
" --fill-color: #1967D2;\n",
" --hover-bg-color: #E2EBFA;\n",
" --hover-fill-color: #174EA6;\n",
" --disabled-fill-color: #AAA;\n",
" --disabled-bg-color: #DDD;\n",
" }\n",
"\n",
" [theme=dark] .colab-df-quickchart {\n",
" --bg-color: #3B4455;\n",
" --fill-color: #D2E3FC;\n",
" --hover-bg-color: #434B5C;\n",
" --hover-fill-color: #FFFFFF;\n",
" --disabled-bg-color: #3B4455;\n",
" --disabled-fill-color: #666;\n",
" }\n",
"\n",
" .colab-df-quickchart {\n",
" background-color: var(--bg-color);\n",
" border: none;\n",
" border-radius: 50%;\n",
" cursor: pointer;\n",
" display: none;\n",
" fill: var(--fill-color);\n",
" height: 32px;\n",
" padding: 0;\n",
" width: 32px;\n",
" }\n",
"\n",
" .colab-df-quickchart:hover {\n",
" background-color: var(--hover-bg-color);\n",
" box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
" fill: var(--button-hover-fill-color);\n",
" }\n",
"\n",
" .colab-df-quickchart-complete:disabled,\n",
" .colab-df-quickchart-complete:disabled:hover {\n",
" background-color: var(--disabled-bg-color);\n",
" fill: var(--disabled-fill-color);\n",
" box-shadow: none;\n",
" }\n",
"\n",
" .colab-df-spinner {\n",
" border: 2px solid var(--fill-color);\n",
" border-color: transparent;\n",
" border-bottom-color: var(--fill-color);\n",
" animation:\n",
" spin 1s steps(1) infinite;\n",
" }\n",
"\n",
" @keyframes spin {\n",
" 0% {\n",
" border-color: transparent;\n",
" border-bottom-color: var(--fill-color);\n",
" border-left-color: var(--fill-color);\n",
" }\n",
" 20% {\n",
" border-color: transparent;\n",
" border-left-color: var(--fill-color);\n",
" border-top-color: var(--fill-color);\n",
" }\n",
" 30% {\n",
" border-color: transparent;\n",
" border-left-color: var(--fill-color);\n",
" border-top-color: var(--fill-color);\n",
" border-right-color: var(--fill-color);\n",
" }\n",
" 40% {\n",
" border-color: transparent;\n",
" border-right-color: var(--fill-color);\n",
" border-top-color: var(--fill-color);\n",
" }\n",
" 60% {\n",
" border-color: transparent;\n",
" border-right-color: var(--fill-color);\n",
" }\n",
" 80% {\n",
" border-color: transparent;\n",
" border-right-color: var(--fill-color);\n",
" border-bottom-color: var(--fill-color);\n",
" }\n",
" 90% {\n",
" border-color: transparent;\n",
" border-bottom-color: var(--fill-color);\n",
" }\n",
" }\n",
"</style>\n",
"\n",
" <script>\n",
" async function quickchart(key) {\n",
" const quickchartButtonEl =\n",
" document.querySelector('#' + key + ' button');\n",
" quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n",
" quickchartButtonEl.classList.add('colab-df-spinner');\n",
" try {\n",
" const charts = await google.colab.kernel.invokeFunction(\n",
" 'suggestCharts', [key], {});\n",
" } catch (error) {\n",
" console.error('Error during call to suggestCharts:', error);\n",
" }\n",
" quickchartButtonEl.classList.remove('colab-df-spinner');\n",
" quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
" }\n",
" (() => {\n",
" let quickchartButtonEl =\n",
" document.querySelector('#df-1a8f4885-5b57-4a9a-bfc7-8d551bb879a7 button');\n",
" quickchartButtonEl.style.display =\n",
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
" })();\n",
" </script>\n",
"</div>\n",
" </div>\n",
" </div>\n"
],
"text/plain": [
" instant dteday season yr mnth holiday weekday workingday \\\n",
"0 1 2011-01-01 1 0 1 0 6 0 \n",
"1 2 2011-01-02 1 0 1 0 0 0 \n",
"2 3 2011-01-03 1 0 1 0 1 1 \n",
"3 4 2011-01-04 1 0 1 0 2 1 \n",
"4 5 2011-01-05 1 0 1 0 3 1 \n",
"\n",
" weathersit temp atemp hum windspeed casual registered \\\n",
"0 2 0.344167 0.363625 0.805833 0.160446 331 654 \n",
"1 2 0.363478 0.353739 0.696087 0.248539 131 670 \n",
"2 1 0.196364 0.189405 0.437273 0.248309 120 1229 \n",
"3 1 0.200000 0.212122 0.590435 0.160296 108 1454 \n",
"4 1 0.226957 0.229270 0.436957 0.186900 82 1518 \n",
"\n",
" cnt \n",
"0 985 \n",
"1 801 \n",
"2 1349 \n",
"3 1562 \n",
"4 1600 "
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df = pd.read_csv(\"raw_data/day.csv\", header=0, sep=\",\", parse_dates=[\"dteday\"])\n",
"df.head()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4a9DrmjyhhEP"
},
"source": [
"# Define column mapping"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"executionInfo": {
"elapsed": 5,
"status": "ok",
"timestamp": 1697468132141,
"user": {
"displayName": "Francesco Motoko",
"userId": "00974636158007469548"
},
"user_tz": -120
},
"id": "_bkEZuM8gELe"
},
"outputs": [],
"source": [
"from evidently.pipeline.column_mapping import ColumnMapping"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"executionInfo": {
"elapsed": 5,
"status": "ok",
"timestamp": 1697468132141,
"user": {
"displayName": "Francesco Motoko",
"userId": "00974636158007469548"
},
"user_tz": -120
},
"id": "dLIZqkHAgEuo"
},
"outputs": [],
"source": [
"data_columns = ColumnMapping()\n",
"data_columns.numerical_features = [\"weathersit\", \"temp\", \"atemp\", \"hum\", \"windspeed\"]\n",
"data_columns.categorical_features = [\"holiday\", \"workingday\"]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "yNBKbk51hpyz"
},
"source": [
"# Define what to log"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"executionInfo": {
"elapsed": 4428,
"status": "ok",
"timestamp": 1697468136565,
"user": {
"displayName": "Francesco Motoko",
"userId": "00974636158007469548"
},
"user_tz": -120
},
"id": "owblpS3Ahw0o"
},
"outputs": [],
"source": [
"from evidently.metric_preset import DataDriftPreset\n",
"from evidently.report import Report"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"executionInfo": {
"elapsed": 3,
"status": "ok",
"timestamp": 1697468136565,
"user": {
"displayName": "Francesco Motoko",
"userId": "00974636158007469548"
},
"user_tz": -120
},
"id": "vRF8PjiYho6z"
},
"outputs": [],
"source": [
"def eval_drift(reference, production, column_mapping):\n",
" data_drift_report = Report(metrics=[DataDriftPreset()])\n",
" data_drift_report.run(\n",
" reference_data=reference, current_data=production, column_mapping=column_mapping\n",
" )\n",
" report = data_drift_report.as_dict()\n",
"\n",
" drifts = []\n",
"\n",
" for feature in (\n",
" column_mapping.numerical_features + column_mapping.categorical_features\n",
" ):\n",
" drifts.append(\n",
" (\n",
" feature,\n",
" report[\"metrics\"][1][\"result\"][\"drift_by_columns\"][feature][\n",
" \"drift_score\"\n",
" ],\n",
" )\n",
" )\n",
"\n",
" return drifts"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4Yhet51mh6Xz"
},
"source": [
"# Define the comparison windows"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"executionInfo": {
"elapsed": 3,
"status": "ok",
"timestamp": 1697468136565,
"user": {
"displayName": "Francesco Motoko",
"userId": "00974636158007469548"
},
"user_tz": -120
},
"id": "nTq8xUbGh3Ux"
},
"outputs": [],
"source": [
"# set reference dates\n",
"reference_dates = (\"2011-01-01 00:00:00\", \"2011-01-28 23:00:00\")\n",
"\n",
"# set experiment batches dates\n",
"experiment_batches = [\n",
" (\"2011-01-01 00:00:00\", \"2011-01-29 23:00:00\"),\n",
" (\"2011-01-29 00:00:00\", \"2011-02-07 23:00:00\"),\n",
" (\"2011-02-07 00:00:00\", \"2011-02-14 23:00:00\"),\n",
" (\"2011-02-15 00:00:00\", \"2011-02-21 23:00:00\"),\n",
"]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "8lNq9OdniDss"
},
"source": [
"# Run and log experiments with DVCLive"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"executionInfo": {
"elapsed": 3,
"status": "ok",
"timestamp": 1697468136565,
"user": {
"displayName": "Francesco Motoko",
"userId": "00974636158007469548"
},
"user_tz": -120
},
"id": "zUt5jrVSRIqD"
},
"outputs": [],
"source": [
"!git config --global user.email \"you@example.com\"\n",
"!git config --global user.name \"Your Name\""
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"executionInfo": {
"elapsed": 1231,
"status": "ok",
"timestamp": 1697468137794,
"user": {
"displayName": "Francesco Motoko",
"userId": "00974636158007469548"
},
"user_tz": -120
},
"id": "5Hx1jI9PnT3C"
},
"outputs": [],
"source": [
"from dvclive import Live"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "jTsrtISaSF7D"
},
"source": [
"There are two ways to use DVC, put all the drift evaluation steps in one single experiment (corresponding to a git commit), or to save each step as a separate experiment (git commit)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "RGrEbbla30jr"
},
"source": [
"## In one experiment"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"executionInfo": {
"elapsed": 2844,
"status": "ok",
"timestamp": 1697468140631,
"user": {
"displayName": "Francesco Motoko",
"userId": "00974636158007469548"
},
"user_tz": -120
},
"id": "ijUf_HhRobl0",
"outputId": "796d7eec-17dc-40b2-a4c9-5bdcf9184c58"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"/content\n",
"/content/experiments\n",
"hint: Using 'master' as the name for the initial branch. This default branch name\n",
"hint: is subject to change. To configure the initial branch name to use in all\n",
"hint: of your new repositories, which will suppress this warning, call:\n",
"hint: \n",
"hint: \tgit config --global init.defaultBranch <name>\n",
"hint: \n",
"hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and\n",
"hint: 'development'. The just-created branch can be renamed via this command:\n",
"hint: \n",
"hint: \tgit branch -m <name>\n",
"Initialized empty Git repository in /content/experiments/.git/\n",
"fatal: pathspec '.gitignore' did not match any files\n",
"On branch master\n",
"\n",
"Initial commit\n",
"\n",
"nothing to commit (create/copy files and use \"git add\" to track)\n",
"Initialized DVC repository.\n",
"\n",
"You can now commit the changes to git.\n",
"\n",
"+---------------------------------------------------------------------+\n",
"| |\n",
"| DVC has enabled anonymous aggregate usage analytics. |\n",
"| Read the analytics documentation (and how to opt-out) here: |\n",
"| <https://dvc.org/doc/user-guide/analytics> |\n",
"| |\n",
"+---------------------------------------------------------------------+\n",
"\n",
"What's next?\n",
"------------\n",
"- Check out the documentation: <https://dvc.org/doc>\n",
"- Get help and share ideas: <https://dvc.org/chat>\n",
"- Star us on GitHub: <https://github.com/iterative/dvc>\n",
"[master (root-commit) 9220260] Init DVC\n",
" 3 files changed, 6 insertions(+)\n",
" create mode 100644 .dvc/.gitignore\n",
" create mode 100644 .dvc/config\n",
" create mode 100644 .dvcignore\n"
]
}
],
"source": [
"# Setup a git repo with dvc\n",
"\n",
"%cd /content\n",
"!rm -rf experiments && mkdir experiments\n",
"%cd experiments\n",
"\n",
"!git init\n",
"!git add .gitignore\n",
"!git commit -m \"Init repo\"\n",
"!dvc init\n",
"!git commit -m \"Init DVC\""
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"executionInfo": {
"elapsed": 16055,
"status": "ok",
"timestamp": 1697468156663,
"user": {
"displayName": "Francesco Motoko",
"userId": "00974636158007469548"
},
"user_tz": -120
},
"id": "_h-jGJqPiA30",
"outputId": "0b949e24-8c53-4765-a8ee-64d002b3801e"
},
"outputs": [
{
"data": {
"text/markdown": "# DVC Report\n\nparams.yaml\n\n| begin | end |\n|---------------------|---------------------|\n| 2011-02-15 00:00:00 | 2011-02-21 23:00:00 |\n\nmetrics.json\n\n| weathersit | temp | atemp | hum | windspeed | holiday | workingday | step |\n|--------------|--------|---------|-------|-------------|-----------|--------------|--------|\n| 0.231 | 0 | 0 | 0.062 | 0.012 | 0.275 | 0.593 | 3 |\n\n\n\n\n\n\n\n\n\n
FILE: noxfile.py
function tests (line 17) | def tests(session: nox.Session) -> None:
function core_tests (line 29) | def core_tests(session: nox.Session) -> None:
function lint (line 42) | def lint(session: nox.Session) -> None:
function safety (line 52) | def safety(session: nox.Session) -> None:
function build (line 60) | def build(session: nox.Session) -> None:
function dev (line 68) | def dev(session: nox.Session) -> None:
FILE: src/dvclive/dvc.py
function _dvc_dir (line 20) | def _dvc_dir(dirname: StrPath) -> str:
function _find_dvc_root (line 24) | def _find_dvc_root(root: Optional[StrPath] = None) -> Optional[str]:
function get_dvc_repo (line 43) | def get_dvc_repo() -> Optional["Repo"]:
function make_dvcyaml (line 58) | def make_dvcyaml(live) -> None: # noqa: C901
function update_dvcyaml (line 93) | def update_dvcyaml(live, updates):
function get_exp_name (line 133) | def get_exp_name(name, scm, baseline_rev) -> str:
function find_overlapping_stage (line 161) | def find_overlapping_stage(dvc_repo: "Repo", path: StrPath) -> Optional[...
function ensure_dir_is_tracked (line 170) | def ensure_dir_is_tracked(directory: str, dvc_repo: "Repo") -> None:
FILE: src/dvclive/error.py
class DvcLiveError (line 4) | class DvcLiveError(Exception):
class InvalidDataTypeError (line 8) | class InvalidDataTypeError(DvcLiveError):
method __init__ (line 9) | def __init__(self, name, val):
class InvalidDvcyamlError (line 15) | class InvalidDvcyamlError(DvcLiveError):
method __init__ (line 16) | def __init__(self):
class InvalidImageNameError (line 20) | class InvalidImageNameError(DvcLiveError):
method __init__ (line 21) | def __init__(self, name):
class InvalidPlotTypeError (line 26) | class InvalidPlotTypeError(DvcLiveError):
method __init__ (line 27) | def __init__(self, name):
class InvalidParameterTypeError (line 37) | class InvalidParameterTypeError(DvcLiveError):
method __init__ (line 38) | def __init__(self, msg: Any):
class InvalidReportModeError (line 42) | class InvalidReportModeError(DvcLiveError):
method __init__ (line 43) | def __init__(self, val):
FILE: src/dvclive/fabric.py
class DVCLiveLogger (line 35) | class DVCLiveLogger(Logger):
method __init__ (line 38) | def __init__(
method name (line 52) | def name(self) -> str:
method version (line 56) | def version(self) -> Union[int, str]:
method experiment (line 63) | def experiment(self) -> "Live":
method log_metrics (line 78) | def log_metrics(
method log_hyperparams (line 112) | def log_hyperparams(self, params: Union[dict[str, Any], Namespace]) ->...
method finalize (line 124) | def finalize(self, status: str) -> None: # noqa: ARG002
method _sanitize_params (line 129) | def _sanitize_params(params: Union[dict[str, Any], Namespace]) -> dict...
method __getstate__ (line 145) | def __getstate__(self) -> dict[str, Any]:
FILE: src/dvclive/fastai.py
function _inside_fine_tune (line 10) | def _inside_fine_tune():
class DVCLiveCallback (line 27) | class DVCLiveCallback(Callback):
method __init__ (line 28) | def __init__(
method before_fit (line 39) | def before_fit(self):
method after_epoch (line 52) | def after_epoch(self):
method after_fit (line 69) | def after_fit(self):
FILE: src/dvclive/huggingface.py
class DVCLiveCallback (line 20) | class DVCLiveCallback(TrainerCallback):
method __init__ (line 21) | def __init__(
method on_train_begin (line 36) | def on_train_begin(
method on_log (line 45) | def on_log(
method on_save (line 57) | def on_save(
method on_train_end (line 68) | def on_train_end(
FILE: src/dvclive/keras.py
class DVCLiveCallback (line 10) | class DVCLiveCallback(tf.keras.callbacks.Callback):
method __init__ (line 11) | def __init__(
method on_epoch_end (line 21) | def on_epoch_end(self, epoch: int, logs: Optional[dict] = None):
method on_train_end (line 27) | def on_train_end(self, logs: Optional[dict] = None):
FILE: src/dvclive/lgbm.py
class DVCLiveCallback (line 6) | class DVCLiveCallback:
method __init__ (line 7) | def __init__(self, live: Optional[Live] = None, **kwargs):
method __call__ (line 11) | def __call__(self, env):
FILE: src/dvclive/lightning.py
function _should_sync (line 32) | def _should_sync():
class DVCLiveLogger (line 50) | class DVCLiveLogger(Logger, FabricDVCLiveLogger):
method __init__ (line 51) | def __init__(
method log_metrics (line 71) | def log_metrics(
method after_save_checkpoint (line 81) | def after_save_checkpoint(self, checkpoint_callback: ModelCheckpoint) ...
method finalize (line 92) | def finalize(self, status: str) -> None:
method _scan_checkpoints (line 103) | def _scan_checkpoints(self, checkpoint_callback: ModelCheckpoint) -> N...
method _save_checkpoints (line 112) | def _save_checkpoints(self, checkpoint_callback: ModelCheckpoint) -> N...
FILE: src/dvclive/live.py
class Live (line 78) | class Live:
method __init__ (line 79) | def __init__(
method _init_resume (line 183) | def _init_resume(self):
method _init_cleanup (line 192) | def _init_cleanup(self):
method _init_dvc (line 211) | def _init_dvc(self): # noqa: C901
method _init_dvc_file (line 267) | def _init_dvc_file(self) -> str:
method _init_dvc_pipeline (line 282) | def _init_dvc_pipeline(self):
method _init_studio (line 296) | def _init_studio(self):
method _init_report (line 310) | def _init_report(self):
method _init_test (line 335) | def _init_test(self):
method dir (line 355) | def dir(self) -> str:
method params_file (line 360) | def params_file(self) -> str:
method metrics_file (line 364) | def metrics_file(self) -> str:
method dvc_file (line 368) | def dvc_file(self) -> str:
method plots_dir (line 373) | def plots_dir(self) -> str:
method artifacts_dir (line 377) | def artifacts_dir(self) -> str:
method report_file (line 381) | def report_file(self) -> Optional[str]:
method step (line 388) | def step(self) -> int:
method step (line 392) | def step(self, value: int) -> None:
method monitor_system (line 396) | def monitor_system(
method sync (line 433) | def sync(self):
method next_step (line 443) | def next_step(self):
method log_metric (line 459) | def log_metric(
method log_image (line 507) | def log_image(
method log_plot (line 569) | def log_plot(
method log_sklearn_plot (line 629) | def log_sklearn_plot(
method _read_params (line 694) | def _read_params(self):
method _dump_params (line 699) | def _dump_params(self):
method log_params (line 705) | def log_params(self, params: dict[str, ParamLike]):
method log_param (line 724) | def log_param(self, name: str, val: ParamLike):
method log_artifact (line 741) | def log_artifact(
method cache (line 828) | def cache(self, path):
method make_summary (line 859) | def make_summary(self):
method make_report (line 876) | def make_report(self):
method make_dvcyaml (line 893) | def make_dvcyaml(self):
method _get_live_data (line 904) | def _get_live_data(self) -> Optional[dict[str, Any]]:
method post_data_to_studio (line 926) | def post_data_to_studio(self):
method _wait_for_studio_updates_posted (line 947) | def _wait_for_studio_updates_posted(self):
method end (line 952) | def end(self):
method read_step (line 1000) | def read_step(self):
method read_latest (line 1004) | def read_latest(self):
method __enter__ (line 1010) | def __enter__(self):
method __exit__ (line 1014) | def __exit__(self, exc_type, exc_val, exc_tb):
method save_dvc_exp (line 1019) | def save_dvc_exp(self):
FILE: src/dvclive/monitor_system.py
class _SystemMonitor (line 49) | class _SystemMonitor:
method __init__ (line 58) | def __init__(
method _check_interval (line 82) | def _check_interval(self, interval: float, max_interval: float) -> float:
method _check_num_samples (line 91) | def _check_num_samples(
method _check_directories_to_monitor (line 104) | def _check_directories_to_monitor(
method _monitoring_loop (line 117) | def _monitoring_loop(self):
method _get_metrics (line 148) | def _get_metrics(self) -> dict[str, Union[float, int]]:
method _get_ram_info (line 156) | def _get_ram_info(self) -> dict[str, Union[float, int]]:
method _get_cpu_info (line 164) | def _get_cpu_info(self) -> dict[str, Union[float, int]]:
method _get_disk_info (line 181) | def _get_disk_info(self) -> dict[str, Union[float, int]]:
method _get_gpu_info (line 204) | def _get_gpu_info(self) -> dict[str, Union[float, int]]:
method end (line 240) | def end(self):
FILE: src/dvclive/optuna.py
class DVCLiveCallback (line 5) | class DVCLiveCallback:
method __init__ (line 6) | def __init__(self, metric_name="metric", **kwargs) -> None:
method __call__ (line 12) | def __call__(self, study, trial) -> None:
method _log_metrics (line 17) | def _log_metrics(self, values, live):
FILE: src/dvclive/plots/base.py
class Data (line 5) | class Data(abc.ABC):
method __init__ (line 6) | def __init__(self, name: str, output_folder: str) -> None:
method step (line 12) | def step(self) -> int:
method step (line 16) | def step(self, val: int) -> None:
method output_path (line 21) | def output_path(self) -> Path:
method subfolder (line 26) | def subfolder(self):
method could_log (line 31) | def could_log(val) -> bool:
method dump (line 35) | def dump(self, val, **kwargs):
FILE: src/dvclive/plots/custom.py
class CustomPlot (line 9) | class CustomPlot(Data):
method __init__ (line 13) | def __init__(
method output_path (line 40) | def output_path(self) -> Path:
method could_log (line 46) | def could_log(val: object) -> bool:
method plot_config (line 50) | def plot_config(self):
method dump (line 53) | def dump(self, val, **kwargs) -> None: # noqa: ARG002
FILE: src/dvclive/plots/image.py
class Image (line 8) | class Image(Data):
method output_path (line 13) | def output_path(self) -> Path:
method could_log (line 19) | def could_log(val: object) -> bool:
method dump (line 30) | def dump(self, val, **kwargs) -> None: # noqa: ARG002
FILE: src/dvclive/plots/metric.py
class Metric (line 10) | class Metric(Data):
method could_log (line 15) | def could_log(val: object) -> bool:
method output_path (line 24) | def output_path(self) -> Path:
method dump (line 29) | def dump(self, val, **kwargs) -> None:
method summary_keys (line 46) | def summary_keys(self) -> list[str]:
FILE: src/dvclive/plots/sklearn.py
class SKLearnPlot (line 6) | class SKLearnPlot(CustomPlot):
method could_log (line 10) | def could_log(val: object) -> bool:
class Roc (line 14) | class Roc(SKLearnPlot):
method __init__ (line 15) | def __init__(self, name: str, output_folder: str, **plot_config) -> None:
method dump (line 26) | def dump(self, val, **kwargs) -> None:
class PrecisionRecall (line 41) | class PrecisionRecall(SKLearnPlot):
method __init__ (line 42) | def __init__(self, name: str, output_folder: str, **plot_config) -> None:
method dump (line 51) | def dump(self, val, **kwargs) -> None:
class Det (line 67) | class Det(SKLearnPlot):
method __init__ (line 68) | def __init__(self, name: str, output_folder: str, **plot_config) -> None:
method dump (line 79) | def dump(self, val, **kwargs) -> None:
class ConfusionMatrix (line 95) | class ConfusionMatrix(SKLearnPlot):
method __init__ (line 96) | def __init__(self, name: str, output_folder: str, **plot_config) -> None:
method dump (line 109) | def dump(self, val, **kwargs) -> None: # noqa: ARG002
class Calibration (line 117) | class Calibration(SKLearnPlot):
method __init__ (line 118) | def __init__(self, name: str, output_folder: str, **plot_config) -> None:
method dump (line 129) | def dump(self, val, **kwargs) -> None:
FILE: src/dvclive/plots/utils.py
class NumpyEncoder (line 19) | class NumpyEncoder(json.JSONEncoder):
method default (line 20) | def default(self, o):
FILE: src/dvclive/report.py
function get_scalar_renderers (line 30) | def get_scalar_renderers(metrics_path):
function get_image_renderers (line 48) | def get_image_renderers(images_folder):
function get_custom_plot_renderers (line 66) | def get_custom_plot_renderers(plots_folder, live):
function get_sklearn_plot_renderers (line 84) | def get_sklearn_plot_renderers(plots_folder, live):
function get_metrics_renderers (line 110) | def get_metrics_renderers(dvclive_summary):
function get_params_renderers (line 122) | def get_params_renderers(dvclive_params):
function make_report (line 134) | def make_report(live: "Live"):
FILE: src/dvclive/serialize.py
class YAMLError (line 8) | class YAMLError(DvcLiveError):
class YAMLFileCorruptedError (line 12) | class YAMLFileCorruptedError(YAMLError):
method __init__ (line 13) | def __init__(self, path):
function load_yaml (line 17) | def load_yaml(path, typ="safe"):
function get_yaml (line 29) | def get_yaml():
function dump_yaml (line 41) | def dump_yaml(content, output_file):
function dump_json (line 48) | def dump_json(content, output_file, indent=4, **kwargs):
function make_dir (line 55) | def make_dir(output_file):
FILE: src/dvclive/studio.py
function _cast_to_numbers (line 25) | def _cast_to_numbers(datapoints: Mapping):
function _adapt_path (line 41) | def _adapt_path(live: "Live", name: StrPath):
function _adapt_image (line 49) | def _adapt_image(image_path: StrPath):
function _adapt_images (line 54) | def _adapt_images(live: "Live", images: "list[Image]"):
function _get_studio_updates (line 62) | def _get_studio_updates(live: "Live", data: dict[str, Any]):
function get_dvc_studio_config (line 90) | def get_dvc_studio_config(live: "Live"):
function increment_num_points_sent_to_studio (line 97) | def increment_num_points_sent_to_studio(live, plots_sent, data):
function post_to_studio (line 108) | def post_to_studio( # noqa: C901
FILE: src/dvclive/utils.py
function run_once (line 31) | def run_once(f):
function open_file_in_browser (line 43) | def open_file_in_browser(file) -> bool:
function env2bool (line 50) | def env2bool(var, undefined=False):
function standardize_metric_name (line 60) | def standardize_metric_name(metric_name: str, framework: str) -> str:
function parse_tsv (line 96) | def parse_tsv(path):
function parse_json (line 102) | def parse_json(path):
function parse_metrics (line 107) | def parse_metrics(live):
function matplotlib_installed (line 119) | def matplotlib_installed() -> bool:
function inside_colab (line 127) | def inside_colab() -> bool:
function inside_notebook (line 135) | def inside_notebook() -> bool:
function clean_and_copy_into (line 151) | def clean_and_copy_into(src: StrPath, dst: StrPath) -> str:
function isinstance_without_import (line 170) | def isinstance_without_import(val, module, name):
function catch_and_warn (line 177) | def catch_and_warn(exception, logger, on_finally=None):
function rel_path (line 193) | def rel_path(path, dvc_root_path):
function read_history (line 198) | def read_history(live, metric):
function read_latest (line 211) | def read_latest(live, metric_name):
function convert_datapoints_to_list_of_dicts (line 216) | def convert_datapoints_to_list_of_dicts(
FILE: src/dvclive/vscode.py
function _dvc_exps_run_dir (line 11) | def _dvc_exps_run_dir(dirname: StrPath) -> str:
function _dvclive_only_signal_file (line 15) | def _dvclive_only_signal_file(root_dir: StrPath) -> str:
function _dvclive_step_completed_signal_file (line 20) | def _dvclive_step_completed_signal_file(root_dir: StrPath) -> str:
function _find_non_queue_root (line 25) | def _find_non_queue_root() -> Optional[str]:
function _write_file (line 29) | def _write_file(file: str, contents: dict[str, Union[str, int]]):
function mark_dvclive_step_completed (line 40) | def mark_dvclive_step_completed(step: int) -> None:
function cleanup_dvclive_step_completed (line 59) | def cleanup_dvclive_step_completed() -> None:
function mark_dvclive_only_started (line 73) | def mark_dvclive_only_started(exp_name: str) -> None:
function mark_dvclive_only_ended (line 90) | def mark_dvclive_only_ended() -> None:
FILE: src/dvclive/xgb.py
class DVCLiveCallback (line 10) | class DVCLiveCallback(TrainingCallback):
method __init__ (line 11) | def __init__(
method after_iteration (line 27) | def after_iteration(self, model, epoch, evals_log):
method after_training (line 35) | def after_training(self, model):
FILE: tests/conftest.py
function tmp_dir (line 10) | def tmp_dir(tmp_path, monkeypatch):
function mocked_dvc_repo (line 16) | def mocked_dvc_repo(tmp_dir, mocker):
function mocked_dvc_subrepo (line 31) | def mocked_dvc_subrepo(tmp_dir, mocker, mocked_dvc_repo):
function dvc_repo (line 37) | def dvc_repo(tmp_dir):
function _capture_wrap (line 48) | def _capture_wrap():
function _mocked_webbrowser_open (line 55) | def _mocked_webbrowser_open(mocker):
function _mocked_ci (line 60) | def _mocked_ci(monkeypatch):
function mocked_studio_post (line 65) | def mocked_studio_post(mocker, monkeypatch):
FILE: tests/frameworks/test_fabric.py
class BoringModel (line 15) | class BoringModel(torch.nn.Module):
method __init__ (line 16) | def __init__(self):
method forward (line 20) | def forward(self, x):
function test_dvclive_log_metrics (line 26) | def test_dvclive_log_metrics(tmp_path, mocked_dvc_repo, step_idx):
function test_dvclive_log_hyperparams (line 37) | def test_dvclive_log_hyperparams(tmp_path, mocked_dvc_repo):
function test_dvclive_finalize (line 54) | def test_dvclive_finalize(monkeypatch, tmp_path, mocked_dvc_repo):
FILE: tests/frameworks/test_fastai.py
function data_loader (line 25) | def data_loader():
function test_fastai_callback (line 44) | def test_fastai_callback(tmp_dir, data_loader, mocker):
function test_fastai_pass_logger (line 71) | def test_fastai_pass_logger():
function test_fast_ai_resume (line 78) | def test_fast_ai_resume(tmp_dir, data_loader, mocker):
function test_fast_ai_avoid_unnecessary_end_calls (line 97) | def test_fast_ai_avoid_unnecessary_end_calls(tmp_dir, data_loader, mocker):
function test_fastai_save_model_callback (line 114) | def test_fastai_save_model_callback(tmp_dir, data_loader, mocker):
FILE: tests/frameworks/test_huggingface.py
function compute_metrics (line 27) | def compute_metrics(eval_preds):
class RegressionDataset (line 38) | class RegressionDataset:
method __init__ (line 39) | def __init__(self, a=2, b=3, length=64, seed=42, label_names=None):
method __len__ (line 50) | def __len__(self):
method __getitem__ (line 53) | def __getitem__(self, i):
class RegressionModelConfig (line 59) | class RegressionModelConfig(PretrainedConfig):
method __init__ (line 60) | def __init__(self, a=0, b=0, double_output=False, random_torch=True, *...
class RegressionPreTrainedModel (line 69) | class RegressionPreTrainedModel(PreTrainedModel):
method __init__ (line 73) | def __init__(self, config):
method forward (line 79) | def forward(self, input_x, labels=None, **kwargs):
function data (line 88) | def data():
function model (line 93) | def model():
function args (line 99) | def args():
function test_huggingface_integration (line 111) | def test_huggingface_integration(tmp_dir, model, args, data, mocker, cal...
function test_huggingface_log_model (line 144) | def test_huggingface_log_model(
function test_huggingface_pass_logger (line 196) | def test_huggingface_pass_logger(callback):
function test_huggingface_report_to (line 204) | def test_huggingface_report_to(model, report_to):
FILE: tests/frameworks/test_keras.py
function xor_model (line 16) | def xor_model():
function test_keras_callback (line 37) | def test_keras_callback(tmp_dir, xor_model, mocker):
function test_keras_callback_pass_logger (line 61) | def test_keras_callback_pass_logger():
FILE: tests/frameworks/test_lgbm.py
function model_params (line 21) | def model_params():
function iris_data (line 26) | def iris_data():
function test_lgbm_integration (line 37) | def test_lgbm_integration(tmp_dir, model_params, iris_data):
function test_lgbm_integration_multi_eval (line 59) | def test_lgbm_integration_multi_eval(tmp_dir, model_params, iris_data):
function test_lgbm_pass_logger (line 84) | def test_lgbm_pass_logger():
FILE: tests/frameworks/test_lightning.py
class XORDataset (line 31) | class XORDataset(Dataset):
method __init__ (line 32) | def __init__(self, *args, **kwargs):
method __getitem__ (line 36) | def __getitem__(self, index):
method __len__ (line 41) | def __len__(self):
class LitXOR (line 45) | class LitXOR(LightningModule):
method __init__ (line 46) | def __init__(
method forward (line 60) | def forward(self, *args, **kwargs):
method train_loader (line 69) | def train_loader(self):
method train_dataloader (line 73) | def train_dataloader(self):
method training_step (line 76) | def training_step(self, *args, **kwargs):
method configure_optimizers (line 91) | def configure_optimizers(self):
method predict_dataloader (line 94) | def predict_dataloader(self):
method test_dataloader (line 97) | def test_dataloader(self):
method val_dataloader (line 100) | def val_dataloader(self):
function test_lightning_integration (line 104) | def test_lightning_integration(tmp_dir, mocker):
function test_lightning_default_dir (line 143) | def test_lightning_default_dir(tmp_dir):
function test_lightning_kwargs (line 158) | def test_lightning_kwargs(tmp_dir):
function test_lightning_log_model (line 180) | def test_lightning_log_model(tmp_dir, mocker, log_model, save_top_k):
function test_lightning_steps (line 213) | def test_lightning_steps(tmp_dir, mocker):
class ValLitXOR (line 245) | class ValLitXOR(LitXOR):
method val_loader (line 246) | def val_loader(self):
method val_dataloader (line 250) | def val_dataloader(self):
method training_step (line 253) | def training_step(self, *args, **kwargs):
method validation_step (line 261) | def validation_step(self, *args, **kwargs):
function test_lightning_force_init (line 270) | def test_lightning_force_init(tmp_dir, mocker):
class TestModel (line 281) | class TestModel(BoringModel):
method __init__ (line 282) | def __init__(self, foo, bar=5):
function _test_logger_init_args (line 288) | def _test_logger_init_args(logger_name, init, unresolved={}): # noqa: B006
function test_dvclive_logger_init_args (line 313) | def test_dvclive_logger_init_args():
FILE: tests/frameworks/test_optuna.py
function objective (line 14) | def objective(trial):
function test_optuna_ (line 19) | def test_optuna_(tmp_dir, mocked_dvc_repo):
FILE: tests/frameworks/test_xgboost.py
function train_params (line 22) | def train_params():
function iris_data (line 27) | def iris_data():
function iris_train_eval_data (line 35) | def iris_train_eval_data():
function test_xgb_integration (line 54) | def test_xgb_integration(
function test_xgb_pass_logger (line 82) | def test_xgb_pass_logger():
FILE: tests/plots/test_custom.py
function test_log_custom_plot (line 7) | def test_log_custom_plot(tmp_dir):
function test_log_custom_plot_multi_y (line 34) | def test_log_custom_plot_multi_y(tmp_dir):
function test_log_custom_plot_with_template_as_empty_string (line 61) | def test_log_custom_plot_with_template_as_empty_string(tmp_dir):
FILE: tests/plots/test_image.py
class ExtendedImage (line 12) | class ExtendedImage(Image.Image):
method __init__ (line 13) | def __init__(self, img):
method __getattr__ (line 16) | def __getattr__(self, key):
function test_pil (line 20) | def test_pil(tmp_dir):
function test_pil_omitting_extension_doesnt_save_without_valid_format (line 28) | def test_pil_omitting_extension_doesnt_save_without_valid_format(tmp_dir):
function test_pil_omitting_extension_sets_the_format_if_path_given (line 37) | def test_pil_omitting_extension_sets_the_format_if_path_given(tmp_dir):
function test_invalid_extension (line 53) | def test_invalid_extension(tmp_dir):
function test_numpy (line 63) | def test_numpy(tmp_dir, shape):
function test_path (line 77) | def test_path(tmp_dir):
function test_override_on_step (line 98) | def test_override_on_step(tmp_dir):
function test_cleanup (line 113) | def test_cleanup(tmp_dir):
function test_custom_class (line 125) | def test_custom_class(tmp_dir):
function test_matplotlib (line 134) | def test_matplotlib(tmp_dir):
function test_cache_images (line 149) | def test_cache_images(tmp_dir, dvc_repo, cache):
FILE: tests/plots/test_metric.py
function test_numpy (line 13) | def test_numpy(tmp_dir, dtype):
function test_name_with_dot (line 27) | def test_name_with_dot(tmp_dir):
FILE: tests/plots/test_sklearn.py
function y_true_y_pred_y_score (line 12) | def y_true_y_pred_y_score():
function test_log_calibration_curve (line 28) | def test_log_calibration_curve(tmp_dir, y_true_y_pred_y_score, mocker):
function test_log_det_curve (line 43) | def test_log_det_curve(tmp_dir, y_true_y_pred_y_score, mocker):
function test_log_roc_curve (line 57) | def test_log_roc_curve(tmp_dir, y_true_y_pred_y_score, mocker):
function test_log_prc_curve (line 71) | def test_log_prc_curve(tmp_dir, y_true_y_pred_y_score, mocker):
function test_log_confusion_matrix (line 85) | def test_log_confusion_matrix(tmp_dir, y_true_y_pred_y_score, mocker):
function test_dump_kwargs (line 101) | def test_dump_kwargs(tmp_dir, y_true_y_pred_y_score, mocker):
function test_override_on_step (line 113) | def test_override_on_step(tmp_dir):
function test_cleanup (line 129) | def test_cleanup(tmp_dir, y_true_y_pred_y_score):
function test_custom_name (line 144) | def test_custom_name(tmp_dir, y_true_y_pred_y_score):
function test_custom_title (line 160) | def test_custom_title(tmp_dir, y_true_y_pred_y_score):
function test_custom_labels (line 193) | def test_custom_labels(tmp_dir, y_true_y_pred_y_score):
FILE: tests/test_cleanup.py
function test_cleanup (line 15) | def test_cleanup(tmp_dir, html, dvcyaml):
FILE: tests/test_context_manager.py
function test_context_manager (line 7) | def test_context_manager(tmp_dir):
function test_context_manager_skips_end_calls (line 21) | def test_context_manager_skips_end_calls(tmp_dir):
FILE: tests/test_dvc.py
function test_get_dvc_repo (line 14) | def test_get_dvc_repo(tmp_dir):
function test_get_dvc_repo_subdir (line 20) | def test_get_dvc_repo_subdir(tmp_dir):
function test_exp_save_on_end (line 29) | def test_exp_save_on_end(tmp_dir, save, mocked_dvc_repo):
function test_exp_save_skip_on_env_vars (line 45) | def test_exp_save_skip_on_env_vars(tmp_dir, monkeypatch):
function test_exp_save_with_dvc_files (line 60) | def test_exp_save_with_dvc_files(tmp_dir, mocker):
function test_exp_save_dvcexception_is_ignored (line 83) | def test_exp_save_dvcexception_is_ignored(tmp_dir, mocker):
function test_untracked_dvclive_files_inside_dvc_exp_run_are_added (line 98) | def test_untracked_dvclive_files_inside_dvc_exp_run_are_added(
function test_dvc_outs_are_not_added (line 116) | def test_dvc_outs_are_not_added(tmp_dir, mocked_dvc_repo, monkeypatch):
function test_errors_on_git_add_are_catched (line 135) | def test_errors_on_git_add_are_catched(tmp_dir, mocked_dvc_repo, monkeyp...
function test_exp_save_message (line 145) | def test_exp_save_message(tmp_dir, mocked_dvc_repo):
function test_exp_save_name (line 156) | def test_exp_save_name(tmp_dir, mocked_dvc_repo):
function test_no_scm_repo (line 167) | def test_no_scm_repo(tmp_dir, mocker):
function test_dvc_repro (line 179) | def test_dvc_repro(tmp_dir, monkeypatch, mocked_dvc_repo, mocked_studio_...
function test_get_exp_name_valid (line 188) | def test_get_exp_name_valid(tmp_dir, mocked_dvc_repo):
function test_get_exp_name_random (line 193) | def test_get_exp_name_random(tmp_dir, mocked_dvc_repo, mocker):
function test_get_exp_name_invalid (line 201) | def test_get_exp_name_invalid(tmp_dir, mocked_dvc_repo, mocker, caplog):
function test_get_exp_name_duplicate (line 211) | def test_get_exp_name_duplicate(tmp_dir, mocked_dvc_repo, mocker, caplog):
function test_test_mode (line 223) | def test_test_mode(tmp_dir, monkeypatch, mocked_dvc_repo):
FILE: tests/test_log_artifact.py
function test_log_artifact (line 21) | def test_log_artifact(tmp_dir, dvc_repo, cache):
function test_log_artifact_on_existing_dvc_file (line 30) | def test_log_artifact_on_existing_dvc_file(tmp_dir, dvc_repo):
function test_log_artifact_twice (line 45) | def test_log_artifact_twice(tmp_dir, dvc_repo):
function test_log_artifact_with_save_dvc_exp (line 54) | def test_log_artifact_with_save_dvc_exp(tmp_dir, mocker, mocked_dvc_repo):
function test_log_artifact_type_model (line 68) | def test_log_artifact_type_model(tmp_dir, mocked_dvc_repo):
function test_log_artifact_dvc_symlink (line 79) | def test_log_artifact_dvc_symlink(tmp_dir, dvc_repo):
function test_log_artifact_copy (line 91) | def test_log_artifact_copy(tmp_dir, dvc_repo):
function test_log_artifact_copy_overwrite (line 106) | def test_log_artifact_copy_overwrite(tmp_dir, dvc_repo):
function test_log_artifact_copy_directory_overwrite (line 126) | def test_log_artifact_copy_directory_overwrite(tmp_dir, dvc_repo):
function test_log_artifact_type_model_provided_name (line 156) | def test_log_artifact_type_model_provided_name(tmp_dir, mocked_dvc_repo):
function test_log_artifact_type_model_on_step_and_final (line 167) | def test_log_artifact_type_model_on_step_and_final(tmp_dir, mocked_dvc_r...
function test_log_artifact_type_model_on_step (line 183) | def test_log_artifact_type_model_on_step(tmp_dir, mocked_dvc_repo):
function test_log_artifact_attrs (line 199) | def test_log_artifact_attrs(tmp_dir, mocked_dvc_repo):
function test_log_artifact_type_model_when_dvc_add_fails (line 219) | def test_log_artifact_type_model_when_dvc_add_fails(tmp_dir, mocker, moc...
function test_log_artifact_inside_pipeline (line 231) | def test_log_artifact_inside_pipeline(tmp_dir, mocker, dvc_repo, tracked):
function test_log_artifact_inside_pipeline_subdir (line 270) | def test_log_artifact_inside_pipeline_subdir(tmp_dir, mocker, dvc_repo):
function test_log_artifact_no_repo (line 291) | def test_log_artifact_no_repo(tmp_dir, mocker):
function test_log_artifact_invalid_path_type (line 302) | def test_log_artifact_invalid_path_type(invalid_path, tmp_dir):
FILE: tests/test_log_metric.py
function test_logging_no_step (line 14) | def test_logging_no_step(tmp_dir):
function test_logging_step (line 29) | def test_logging_step(tmp_dir, path):
function test_nested_logging (line 42) | def test_nested_logging(tmp_dir):
function test_log_metric_timestamp (line 70) | def test_log_metric_timestamp(tmp_dir, timestamp):
function test_invalid_metric_type (line 81) | def test_invalid_metric_type(tmp_dir, invalid_type):
function test_log_metric_inf_nan (line 95) | def test_log_metric_inf_nan(tmp_dir, val):
function test_log_metic_str (line 101) | def test_log_metic_str(tmp_dir):
FILE: tests/test_log_param.py
function test_cleanup_params (line 10) | def test_cleanup_params(tmp_dir):
function test_log_param (line 45) | def test_log_param(tmp_dir, param_name, param_value):
function test_log_params (line 54) | def test_log_params(tmp_dir):
function test_log_params_resume (line 71) | def test_log_params_resume(tmp_dir, resume):
function test_log_param_custom_obj (line 79) | def test_log_param_custom_obj(tmp_dir):
FILE: tests/test_logging.py
function test_logger (line 6) | def test_logger(tmp_dir, mocker):
function test_suppress_dvc_logs (line 21) | def test_suppress_dvc_logs(tmp_dir, mocked_dvc_repo):
FILE: tests/test_make_dvcyaml.py
function test_make_dvcyaml_empty (line 13) | def test_make_dvcyaml_empty(tmp_dir):
function test_make_dvcyaml_param (line 20) | def test_make_dvcyaml_param(tmp_dir):
function test_make_dvcyaml_metrics (line 30) | def test_make_dvcyaml_metrics(tmp_dir):
function test_make_dvcyaml_metrics_no_plots (line 41) | def test_make_dvcyaml_metrics_no_plots(tmp_dir):
function test_make_dvcyaml_summary (line 51) | def test_make_dvcyaml_summary(tmp_dir):
function test_make_dvcyaml_all_plots (line 61) | def test_make_dvcyaml_all_plots(tmp_dir):
function test_make_dvcyaml_relpath (line 117) | def test_make_dvcyaml_relpath(tmp_dir, mocked_dvc_repo):
function test_make_dvcyaml_update (line 200) | def test_make_dvcyaml_update(tmp_dir, orig_yaml, updated_yaml):
function test_make_dvcyaml_update_artifact (line 259) | def test_make_dvcyaml_update_artifact(
function test_make_dvcyaml_update_all (line 272) | def test_make_dvcyaml_update_all(tmp_dir, mocked_dvc_repo):
function test_make_dvcyaml_update_multiple (line 347) | def test_make_dvcyaml_update_multiple(tmp_dir, mocked_dvc_repo):
function test_dvcyaml_on_next_step (line 372) | def test_dvcyaml_on_next_step(tmp_dir, dvcyaml, mocked_dvc_repo):
function test_dvcyaml_on_end (line 382) | def test_dvcyaml_on_end(tmp_dir, dvcyaml, mocked_dvc_repo):
function test_make_dvcyaml_idempotent (line 391) | def test_make_dvcyaml_idempotent(tmp_dir, mocked_dvc_repo):
function test_warn_on_dvcyaml_output_overlap (line 407) | def test_warn_on_dvcyaml_output_overlap(tmp_dir, mocker, mocked_dvc_repo...
function test_make_dvcyaml (line 429) | def test_make_dvcyaml(tmp_dir, mocked_dvc_repo, dvcyaml):
function test_make_dvcyaml_no_repo (line 443) | def test_make_dvcyaml_no_repo(tmp_dir, mocker):
function test_make_dvcyaml_invalid (line 450) | def test_make_dvcyaml_invalid(tmp_dir, mocker):
function test_make_dvcyaml_on_end (line 455) | def test_make_dvcyaml_on_end(tmp_dir, mocker):
function test_make_dvcyaml_false (line 462) | def test_make_dvcyaml_false(tmp_dir, mocker):
function test_make_dvcyaml_none (line 469) | def test_make_dvcyaml_none(tmp_dir, mocker):
FILE: tests/test_make_report.py
function test_get_image_renderers (line 22) | def test_get_image_renderers(tmp_dir, mode):
function test_get_renderers (line 36) | def test_get_renderers(tmp_dir, mocker):
function test_report_init (line 71) | def test_report_init(monkeypatch, mocker):
function test_make_report (line 92) | def test_make_report(tmp_dir, mode):
function test_make_report_open (line 107) | def test_make_report_open(tmp_dir, mocker, monkeypatch):
function test_get_plot_renderers_sklearn (line 131) | def test_get_plot_renderers_sklearn(tmp_dir):
function test_get_plot_renderers_custom (line 173) | def test_get_plot_renderers_custom(tmp_dir):
function test_report_notebook (line 204) | def test_report_notebook(tmp_dir, mocker):
FILE: tests/test_make_summary.py
function test_make_summary_without_calling_log (line 7) | def test_make_summary_without_calling_log(tmp_dir):
function test_make_summary_is_called_on_end (line 21) | def test_make_summary_is_called_on_end(tmp_dir):
function test_make_summary_on_end_dont_increment_step (line 35) | def test_make_summary_on_end_dont_increment_step(tmp_dir):
FILE: tests/test_monitor_system.py
function mock_psutil_cpu (line 30) | def mock_psutil_cpu(mocker):
function mock_psutil_ram (line 38) | def mock_psutil_ram(mocker):
function mock_psutil_disk (line 48) | def mock_psutil_disk(mocker):
function mock_psutil_disk_with_oserror (line 56) | def mock_psutil_disk_with_oserror(mocker):
function mock_pynvml (line 72) | def mock_pynvml(mocker, num_gpus=2):
function cpu_metrics (line 93) | def cpu_metrics():
function _timeserie_schema (line 111) | def _timeserie_schema(name, value):
function cpu_timeseries (line 116) | def cpu_timeseries():
function gpu_timeseries (line 136) | def gpu_timeseries():
function test_monitor_system_is_false (line 147) | def test_monitor_system_is_false(tmp_dir, mocker):
function test_monitor_system_is_true (line 159) | def test_monitor_system_is_true(tmp_dir, mocker):
function test_all_threads_close (line 172) | def test_all_threads_close(tmp_dir, mocker):
function test_ignore_non_existent_directories (line 195) | def test_ignore_non_existent_directories(tmp_dir, mocker):
function test_monitor_system_metrics (line 223) | def test_monitor_system_metrics(tmp_dir, cpu_metrics, mocker):
function test_monitor_system_timeseries (line 247) | def test_monitor_system_timeseries(tmp_dir, cpu_timeseries, mocker):
function test_monitor_system_metrics_with_gpu (line 274) | def test_monitor_system_metrics_with_gpu(tmp_dir, cpu_metrics, mocker):
function test_monitor_system_timeseries_with_gpu (line 307) | def test_monitor_system_timeseries_with_gpu(
FILE: tests/test_post_to_studio.py
function get_studio_call (line 19) | def get_studio_call(event_type, exp_name, **kwargs):
function test_post_to_studio (line 38) | def test_post_to_studio(tmp_dir, mocked_dvc_repo, mocked_studio_post):
function test_post_to_studio_subrepo (line 95) | def test_post_to_studio_subrepo(tmp_dir, mocked_dvc_subrepo, mocked_stud...
function test_post_to_studio_repo_url (line 107) | def test_post_to_studio_repo_url(tmp_dir, dvc_repo, mocked_studio_post, ...
function test_post_to_studio_failed_data_request (line 118) | def test_post_to_studio_failed_data_request(
function test_post_to_studio_failed_start_request (line 157) | def test_post_to_studio_failed_start_request(
function test_post_to_studio_done_only_once (line 176) | def test_post_to_studio_done_only_once(tmp_dir, mocked_dvc_repo, mocked_...
function test_post_to_studio_snapshots_data_to_send (line 196) | def test_post_to_studio_snapshots_data_to_send(
function test_studio_updates_posted_on_end (line 251) | def test_studio_updates_posted_on_end(tmp_path, mocked_dvc_repo, mocked_...
function test_studio_update_raises_exception (line 275) | def test_studio_update_raises_exception(tmp_path, mocked_dvc_repo, mocke...
function test_post_to_studio_skip_start_and_done_on_env_var (line 306) | def test_post_to_studio_skip_start_and_done_on_env_var(
function test_post_to_studio_dvc_studio_config (line 325) | def test_post_to_studio_dvc_studio_config(
function test_post_to_studio_skip_if_no_token (line 348) | def test_post_to_studio_skip_if_no_token(
function test_post_to_studio_shorten_names (line 371) | def test_post_to_studio_shorten_names(tmp_dir, mocked_dvc_repo, mocked_s...
function test_post_to_studio_inside_dvc_exp (line 395) | def test_post_to_studio_inside_dvc_exp(
function test_post_to_studio_inside_subdir (line 417) | def test_post_to_studio_inside_subdir(
function test_post_to_studio_inside_subdir_dvc_exp (line 446) | def test_post_to_studio_inside_subdir_dvc_exp(
function test_post_to_studio_without_exp (line 477) | def test_post_to_studio_without_exp(tmp_dir, mocked_dvc_repo, mocked_stu...
function test_get_dvc_studio_config_none (line 481) | def test_get_dvc_studio_config_none(mocker):
function test_get_dvc_studio_config_env_var (line 487) | def test_get_dvc_studio_config_env_var(monkeypatch, mocker):
function test_get_dvc_studio_config_dvc_repo (line 499) | def test_get_dvc_studio_config_dvc_repo(mocked_dvc_repo):
function test_post_to_studio_images (line 509) | def test_post_to_studio_images(tmp_dir, mocked_dvc_repo, mocked_studio_p...
function test_post_to_studio_message (line 533) | def test_post_to_studio_message(tmp_dir, mocked_dvc_repo, mocked_studio_...
function test_post_to_studio_name (line 544) | def test_post_to_studio_name(tmp_dir, mocked_dvc_repo, mocked_studio_post):
function test_post_to_studio_if_done_skipped (line 555) | def test_post_to_studio_if_done_skipped(tmp_dir, mocked_dvc_repo, mocked...
function test_post_to_studio_no_repo (line 571) | def test_post_to_studio_no_repo(tmp_dir, monkeypatch, mocked_studio_post):
function test_post_to_studio_skip_if_no_repo_url (line 629) | def test_post_to_studio_skip_if_no_repo_url(
function test_post_to_studio_repeat_step (line 648) | def test_post_to_studio_repeat_step(tmp_dir, mocked_dvc_repo, mocked_stu...
FILE: tests/test_resume.py
function test_resume (line 12) | def test_resume(tmp_dir, resume, steps, metrics):
function test_resume_on_first_init (line 39) | def test_resume_on_first_init(tmp_dir):
function test_resume_env_var (line 45) | def test_resume_env_var(tmp_dir, monkeypatch):
FILE: tests/test_step.py
function test_allow_step_override (line 10) | def test_allow_step_override(tmp_dir, metric):
function test_custom_steps (line 17) | def test_custom_steps(tmp_dir):
function test_log_reset_with_set_step (line 32) | def test_log_reset_with_set_step(tmp_dir):
function test_get_step_resume (line 51) | def test_get_step_resume(tmp_dir):
function test_get_step_custom_steps (line 67) | def test_get_step_custom_steps(tmp_dir):
function test_get_step_control_flow (line 79) | def test_get_step_control_flow(tmp_dir):
function test_set_step_only (line 91) | def test_set_step_only(tmp_dir):
function test_step_on_end (line 100) | def test_step_on_end(tmp_dir):
FILE: tests/test_utils.py
function test_standardize_metric_name (line 19) | def test_standardize_metric_name(framework, logged, standardized):
function test_convert_datapoints_to_list_of_dicts (line 39) | def test_convert_datapoints_to_list_of_dicts(input_data, expected_output):
function test_unsupported_format (line 43) | def test_unsupported_format():
FILE: tests/test_vscode.py
function test_vscode_dvclive_step_completed_signal_file (line 11) | def test_vscode_dvclive_step_completed_signal_file(
function test_vscode_dvclive_only_signal_file (line 63) | def test_vscode_dvclive_only_signal_file(tmp_dir, dvc_root, mocker):
Condensed preview — 80 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (654K chars).
[
{
"path": ".cruft.json",
"chars": 700,
"preview": "{\n \"template\": \"https://github.com/iterative/py-template\",\n \"commit\": \"e4ec95f4cfd03d4af0a8604d462ee11d07d63b42\",\n \"c"
},
{
"path": ".gitattributes",
"chars": 19,
"preview": "* text=auto eol=lf\n"
},
{
"path": ".github/PULL_REQUEST_TEMPLATE.md",
"chars": 402,
"preview": "- [ ] ❗ I have followed the [Contributing to DVCLive](https://github.com/iterative/dvclive/blob/main/CONTRIBUTING.rst) g"
},
{
"path": ".github/codecov.yml",
"chars": 354,
"preview": "coverage:\n status:\n project:\n default:\n # auto compares coverage to the previous base commit\n tar"
},
{
"path": ".github/dependabot.yml",
"chars": 275,
"preview": "version: 2\n\nupdates:\n - directory: \"/\"\n package-ecosystem: \"pip\"\n schedule:\n interval: \"weekly\"\n labels:\n"
},
{
"path": ".github/workflows/release.yml",
"chars": 763,
"preview": "name: Release\n\non:\n release:\n types: [published]\n workflow_dispatch:\n\nenv:\n FORCE_COLOR: \"1\"\n\njobs:\n release:\n "
},
{
"path": ".github/workflows/tests.yml",
"chars": 2193,
"preview": "name: Tests\n\non:\n push:\n branches: [main]\n pull_request:\n workflow_dispatch:\n\nenv:\n FORCE_COLOR: \"1\"\n\nconcurrency"
},
{
"path": ".github/workflows/update-template.yaml",
"chars": 306,
"preview": "name: Update template\n\non:\n schedule:\n - cron: '5 1 * * *' # every day at 01:05\n workflow_dispatch:\n\njobs:\n updat"
},
{
"path": ".gitignore",
"chars": 2110,
"preview": "# Byte-compiled / optimized / DLL files\n__pycache__/\n*.py[cod]\n*$py.class\n\n# C extensions\n*.so\n\n# Distribution / packagi"
},
{
"path": ".pre-commit-config.yaml",
"chars": 1278,
"preview": "default_language_version:\n python: python3\nrepos:\n - repo: https://github.com/pre-commit/pre-commit-hooks\n rev: v6."
},
{
"path": "CODE_OF_CONDUCT.rst",
"chars": 5389,
"preview": "Contributor Covenant Code of Conduct\n====================================\n\nOur Pledge\n----------\n\nWe as members, contrib"
},
{
"path": "CONTRIBUTING.rst",
"chars": 2731,
"preview": "Contributor Guide\n=================\n\nThank you for your interest in improving this project.\nThis project is open-source "
},
{
"path": "LICENSE",
"chars": 11340,
"preview": " Apache License\n Version 2.0, January 2004\n "
},
{
"path": "README.md",
"chars": 7074,
"preview": "# DVCLive\n\n[](https://pypi.org/project/dvclive/)\n[![Status](https://im"
},
{
"path": "examples/DVCLive-Evidently.ipynb",
"chars": 296320,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"id\": \"WpfOFaqHcnAt\"\n },\n \"source\": [\n \"# Inst"
},
{
"path": "examples/DVCLive-Fabric.ipynb",
"chars": 10711,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"id\": \"QKSE19fW_Dnj\"\n },\n \"source\": [\n \"# DVCL"
},
{
"path": "examples/DVCLive-HuggingFace.ipynb",
"chars": 8529,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"id\": \"3SJ8SY6ldmsS\"\n },\n \"source\": [\n \"### Ho"
},
{
"path": "examples/DVCLive-PyTorch-Lightning.ipynb",
"chars": 6660,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"id\": \"A812CVYi_B2b\"\n },\n \"source\": [\n \"<a hre"
},
{
"path": "examples/DVCLive-Quickstart.ipynb",
"chars": 9933,
"preview": "{\n \"cells\": [\n {\n \"attachments\": {},\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"<a href=\\\"htt"
},
{
"path": "examples/DVCLive-YOLO.ipynb",
"chars": 3199,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"<a href=\\\"https://colab.research.go"
},
{
"path": "examples/DVCLive-scikit-learn.ipynb",
"chars": 4243,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"<a href=\\\"https://colab.research.go"
},
{
"path": "noxfile.py",
"chars": 2137,
"preview": "\"\"\"Automation using nox.\"\"\"\n\nimport glob\nimport os\n\nimport nox\n\nnox.options.default_venv_backend = \"uv|virtualenv\"\nnox.o"
},
{
"path": "pyproject.toml",
"chars": 4072,
"preview": "[build-system]\nrequires = [\"setuptools>=77\", \"setuptools_scm[toml]>=8\"]\nbuild-backend = \"setuptools.build_meta\"\n\n[projec"
},
{
"path": "src/dvclive/__init__.py",
"chars": 37,
"preview": "from .live import Live # noqa: F401\n"
},
{
"path": "src/dvclive/dvc.py",
"chars": 5815,
"preview": "# ruff: noqa: SLF001\nimport copy\nimport logging\nimport os\nfrom pathlib import Path\nfrom typing import TYPE_CHECKING, Any"
},
{
"path": "src/dvclive/env.py",
"chars": 230,
"preview": "DVCLIVE_LOGLEVEL = \"DVCLIVE_LOGLEVEL\"\nDVCLIVE_OPEN = \"DVCLIVE_OPEN\"\nDVCLIVE_RESUME = \"DVCLIVE_RESUME\"\nDVCLIVE_TEST = \"DV"
},
{
"path": "src/dvclive/error.py",
"chars": 1197,
"preview": "from typing import Any\n\n\nclass DvcLiveError(Exception):\n pass\n\n\nclass InvalidDataTypeError(DvcLiveError):\n def __i"
},
{
"path": "src/dvclive/fabric.py",
"chars": 4615,
"preview": "# mypy: disable-error-code=\"no-redef\"\nfrom argparse import Namespace\nfrom collections.abc import Mapping\nfrom typing imp"
},
{
"path": "src/dvclive/fastai.py",
"chars": 2586,
"preview": "import inspect\nfrom typing import Optional\n\nfrom fastai.callback.core import Callback\n\nfrom dvclive import Live\nfrom dvc"
},
{
"path": "src/dvclive/huggingface.py",
"chars": 2589,
"preview": "# ruff: noqa: ARG002\nimport logging\nimport os\nfrom typing import Literal, Optional, Union\n\nfrom transformers import (\n "
},
{
"path": "src/dvclive/keras.py",
"chars": 816,
"preview": "# ruff: noqa: ARG002\nfrom typing import Optional\n\nimport tensorflow as tf\n\nfrom dvclive import Live\nfrom dvclive.utils i"
},
{
"path": "src/dvclive/lgbm.py",
"chars": 593,
"preview": "from typing import Optional\n\nfrom dvclive import Live\n\n\nclass DVCLiveCallback:\n def __init__(self, live: Optional[Liv"
},
{
"path": "src/dvclive/lightning.py",
"chars": 4301,
"preview": "# mypy: disable-error-code=\"no-redef\"\nimport inspect\nfrom collections.abc import Mapping\nfrom pathlib import Path\nfrom t"
},
{
"path": "src/dvclive/live.py",
"chars": 39467,
"preview": "import builtins\nimport glob\nimport json\nimport logging\nimport math\nimport os\nimport queue\nimport shutil\nimport tempfile\n"
},
{
"path": "src/dvclive/monitor_system.py",
"chars": 8510,
"preview": "import logging\nimport os\nfrom statistics import mean\nfrom threading import Event, Thread\nfrom typing import Union\n\nimpor"
},
{
"path": "src/dvclive/optuna.py",
"chars": 1445,
"preview": "# ruff: noqa: ARG002\nfrom dvclive import Live\n\n\nclass DVCLiveCallback:\n def __init__(self, metric_name=\"metric\", **kw"
},
{
"path": "src/dvclive/plots/__init__.py",
"chars": 439,
"preview": "from .custom import CustomPlot\nfrom .image import Image\nfrom .metric import Metric\nfrom .sklearn import Calibration, Con"
},
{
"path": "src/dvclive/plots/base.py",
"chars": 722,
"preview": "import abc\nfrom pathlib import Path\n\n\nclass Data(abc.ABC):\n def __init__(self, name: str, output_folder: str) -> None"
},
{
"path": "src/dvclive/plots/custom.py",
"chars": 1424,
"preview": "from pathlib import Path\nfrom typing import Optional, Union\n\nfrom dvclive.serialize import dump_json\n\nfrom .base import "
},
{
"path": "src/dvclive/plots/image.py",
"chars": 1305,
"preview": "from pathlib import Path, PurePath\n\nfrom dvclive.utils import isinstance_without_import\n\nfrom .base import Data\n\n\nclass "
},
{
"path": "src/dvclive/plots/metric.py",
"chars": 1351,
"preview": "import csv\nimport os\nimport time\nfrom pathlib import Path\n\nfrom .base import Data\nfrom .utils import NUMPY_SCALARS\n\n\ncla"
},
{
"path": "src/dvclive/plots/sklearn.py",
"chars": 5243,
"preview": "from dvclive.serialize import dump_json\n\nfrom .custom import CustomPlot\n\n\nclass SKLearnPlot(CustomPlot):\n subfolder ="
},
{
"path": "src/dvclive/plots/utils.py",
"chars": 563,
"preview": "import json\n\nNUMPY_INTS = [\n \"intc\",\n \"intp\",\n \"int8\",\n \"int16\",\n \"int32\",\n \"int64\",\n \"uint8\",\n "
},
{
"path": "src/dvclive/py.typed",
"chars": 0,
"preview": ""
},
{
"path": "src/dvclive/report.py",
"chars": 5102,
"preview": "# ruff: noqa: SLF001\nimport base64\nimport json\nfrom pathlib import Path\nfrom typing import TYPE_CHECKING\n\nfrom dvc_rende"
},
{
"path": "src/dvclive/serialize.py",
"chars": 1443,
"preview": "import json\nimport os\nfrom collections import OrderedDict\n\nfrom dvclive.error import DvcLiveError\n\n\nclass YAMLError(DvcL"
},
{
"path": "src/dvclive/studio.py",
"chars": 4898,
"preview": "# ruff: noqa: SLF001\nimport base64\nimport logging\nimport math\nimport os\nfrom collections.abc import Mapping\nfrom pathlib"
},
{
"path": "src/dvclive/utils.py",
"chars": 6558,
"preview": "import csv\nimport json\nimport os\nimport re\nimport shutil\nimport webbrowser\nfrom pathlib import Path, PurePath\nfrom platf"
},
{
"path": "src/dvclive/vscode.py",
"chars": 2716,
"preview": "import json\nimport os\nfrom typing import Optional, Union\n\nfrom dvclive.dvc import _find_dvc_root\nfrom dvclive.utils impo"
},
{
"path": "src/dvclive/xgb.py",
"chars": 1108,
"preview": "# ruff: noqa: ARG002\nfrom typing import Optional\nfrom warnings import warn\n\nfrom xgboost.callback import TrainingCallbac"
},
{
"path": "tests/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "tests/conftest.py",
"chars": 1985,
"preview": "import sys\n\nimport pytest\nfrom dvc_studio_client.env import DVC_STUDIO_TOKEN, DVC_STUDIO_URL, STUDIO_REPO_URL\n\nfrom dvcl"
},
{
"path": "tests/frameworks/test_fabric.py",
"chars": 1999,
"preview": "from argparse import Namespace\nfrom unittest.mock import Mock\n\nimport numpy as np\nimport pytest\n\ntry:\n import torch\n\n"
},
{
"path": "tests/frameworks/test_fastai.py",
"chars": 3668,
"preview": "import os\n\nimport pytest\n\nfrom dvclive import Live\nfrom dvclive.plots.metric import Metric\n\ntry:\n from fastai.callbac"
},
{
"path": "tests/frameworks/test_huggingface.py",
"chars": 6156,
"preview": "import os\n\nimport pytest\n\nfrom dvclive import Live\nfrom dvclive.plots.metric import Metric\nfrom dvclive.serialize import"
},
{
"path": "tests/frameworks/test_keras.py",
"chars": 1665,
"preview": "import os\n\nimport pytest\n\nfrom dvclive import Live\nfrom dvclive.plots.metric import Metric\nfrom dvclive.utils import par"
},
{
"path": "tests/frameworks/test_lgbm.py",
"chars": 2475,
"preview": "import os\nfrom sys import platform\n\nimport pytest\n\nfrom dvclive import Live\nfrom dvclive.utils import parse_metrics\n\ntry"
},
{
"path": "tests/frameworks/test_lightning.py",
"chars": 9476,
"preview": "import os\nfrom contextlib import redirect_stdout\nfrom io import StringIO\nfrom unittest import mock\n\nimport pytest\nimport"
},
{
"path": "tests/frameworks/test_optuna.py",
"chars": 898,
"preview": "import pytest\n\nfrom dvclive.serialize import load_yaml\nfrom dvclive.utils import parse_json\n\ntry:\n import optuna\n\n "
},
{
"path": "tests/frameworks/test_xgboost.py",
"chars": 2317,
"preview": "import os\nfrom contextlib import nullcontext\n\nimport pytest\n\nfrom dvclive import Live\nfrom dvclive.plots.metric import M"
},
{
"path": "tests/plots/test_custom.py",
"chars": 2250,
"preview": "import json\n\nfrom dvclive import Live\nfrom dvclive.plots.custom import CustomPlot\n\n\ndef test_log_custom_plot(tmp_dir):\n "
},
{
"path": "tests/plots/test_image.py",
"chars": 4384,
"preview": "import matplotlib.pyplot as plt\nimport numpy as np\nimport pytest\nfrom PIL import Image\n\nfrom dvclive import Live\nfrom dv"
},
{
"path": "tests/plots/test_metric.py",
"chars": 1065,
"preview": "import json\n\nimport numpy as np\nimport pytest\n\nfrom dvclive import Live\nfrom dvclive.plots.metric import Metric\nfrom dvc"
},
{
"path": "tests/plots/test_sklearn.py",
"chars": 6198,
"preview": "# ruff: noqa: N806\nimport json\n\nimport pytest\nfrom sklearn import calibration, metrics\n\nfrom dvclive import Live\nfrom dv"
},
{
"path": "tests/test_cleanup.py",
"chars": 1215,
"preview": "import pytest\n\nfrom dvclive import Live\nfrom dvclive.plots import Metric\n\n\n@pytest.mark.parametrize(\n \"html\",\n [Tr"
},
{
"path": "tests/test_context_manager.py",
"chars": 716,
"preview": "import json\n\nfrom dvclive import Live\nfrom dvclive.plots import Metric\n\n\ndef test_context_manager(tmp_dir):\n with Liv"
},
{
"path": "tests/test_dvc.py",
"chars": 7085,
"preview": "import os\n\nimport pytest\nfrom dvc.exceptions import DvcException\nfrom dvc.repo import Repo\nfrom dvc.scm import NoSCM\nfro"
},
{
"path": "tests/test_log_artifact.py",
"chars": 10224,
"preview": "import shutil\nfrom pathlib import Path\n\nimport pytest\nfrom dvc.exceptions import DvcException\n\nfrom dvclive import Live\n"
},
{
"path": "tests/test_log_metric.py",
"chars": 2956,
"preview": "import math\nimport os\n\nimport numpy as np\nimport pytest\n\nfrom dvclive import Live\nfrom dvclive.error import InvalidDataT"
},
{
"path": "tests/test_log_param.py",
"chars": 2120,
"preview": "import os\n\nimport pytest\n\nfrom dvclive import Live\nfrom dvclive.error import InvalidParameterTypeError\nfrom dvclive.seri"
},
{
"path": "tests/test_logging.py",
"chars": 551,
"preview": "import logging\n\nfrom dvclive import Live\n\n\ndef test_logger(tmp_dir, mocker):\n logger = mocker.patch(\"dvclive.live.log"
},
{
"path": "tests/test_make_dvcyaml.py",
"chars": 14035,
"preview": "import os\nfrom pathlib import Path\n\nimport pytest\nfrom PIL import Image\n\nfrom dvclive import Live\nfrom dvclive.dvc impor"
},
{
"path": "tests/test_make_report.py",
"chars": 6973,
"preview": "import numpy as np\nimport pytest\nfrom PIL import Image\n\nfrom dvclive import Live\nfrom dvclive.env import DVCLIVE_OPEN\nfr"
},
{
"path": "tests/test_make_summary.py",
"chars": 1064,
"preview": "import json\n\nfrom dvclive import Live\nfrom dvclive.plots import Metric\n\n\ndef test_make_summary_without_calling_log(tmp_d"
},
{
"path": "tests/test_monitor_system.py",
"chars": 10366,
"preview": "import time\nfrom pathlib import Path\n\nimport dpath\nimport pytest\nfrom pytest_voluptuous import S\n\nfrom dvclive import Li"
},
{
"path": "tests/test_post_to_studio.py",
"chars": 20944,
"preview": "import time\nimport unittest\nfrom collections import defaultdict\nfrom copy import deepcopy\nfrom pathlib import Path\n\nimpo"
},
{
"path": "tests/test_resume.py",
"chars": 1331,
"preview": "import pytest\n\nfrom dvclive import Live\nfrom dvclive.env import DVCLIVE_RESUME\nfrom dvclive.utils import read_history, r"
},
{
"path": "tests/test_step.py",
"chars": 2637,
"preview": "import os\n\nimport pytest\n\nfrom dvclive import Live\nfrom dvclive.utils import read_history, read_latest\n\n\n@pytest.mark.pa"
},
{
"path": "tests/test_utils.py",
"chars": 1649,
"preview": "import numpy as np\nimport pandas as pd\nimport pytest\n\nfrom dvclive.error import InvalidDataTypeError\nfrom dvclive.utils "
},
{
"path": "tests/test_vscode.py",
"chars": 2684,
"preview": "import json\nimport os\n\nimport pytest\n\nfrom dvclive import Live, env\n\n\n@pytest.mark.vscode\n@pytest.mark.parametrize(\"dvc_"
}
]
About this extraction
This page contains the full source code of the iterative/dvclive GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 80 files (602.4 KB), approximately 258.6k tokens, and a symbol index with 477 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.