Showing preview only (316K chars total). Download the full file or copy to clipboard to get everything.
Repository: peterdemin/pip-compile-multi
Branch: master
Commit: ffb2003af628
Files: 146
Total size: 284.1 KB
Directory structure:
gitextract_snhzpegr/
├── .coveragerc
├── .editorconfig
├── .github/
│ └── workflows/
│ ├── pipcompilemulti.yml
│ ├── python310-windows.yml
│ ├── python310.yml
│ ├── python311.yml
│ ├── python312.yml
│ ├── python313.yml
│ └── python314.yml
├── .gitignore
├── .pre-commit-config.yaml
├── .pre-commit-hooks.yaml
├── .pylintrc
├── .readthedocs.yml
├── .travis.yml
├── AUTHORS.rst
├── CONTRIBUTING.rst
├── Dockerfile
├── HISTORY.rst
├── LICENSE.txt
├── MANIFEST.in
├── Makefile
├── README.rst
├── docs/
│ ├── Makefile
│ ├── _static/
│ │ └── custom.css
│ ├── afterword.rst
│ ├── boilerplate.rst
│ ├── conf.py
│ ├── features.rst
│ ├── history.rst
│ ├── index.rst
│ ├── installation.rst
│ ├── migration.rst
│ ├── precommit.rst
│ └── why.rst
├── how-to.md
├── nested/
│ ├── base.in
│ ├── base.txt
│ ├── diamond.in
│ ├── diamond.txt
│ ├── subproject/
│ │ ├── base.in
│ │ ├── base.txt
│ │ ├── sub.in
│ │ └── sub.txt
│ ├── up.in
│ └── up.txt
├── pipcompilemulti/
│ ├── __init__.py
│ ├── actions.py
│ ├── cli_v1.py
│ ├── cli_v2.py
│ ├── config.py
│ ├── deduplicate.py
│ ├── dependency.py
│ ├── discover.py
│ ├── environment.py
│ ├── features/
│ │ ├── __init__.py
│ │ ├── add_hashes.py
│ │ ├── annotate_index.py
│ │ ├── autoresolve.py
│ │ ├── backtracking.py
│ │ ├── base.py
│ │ ├── base_dir.py
│ │ ├── build_isolation.py
│ │ ├── compatible.py
│ │ ├── controller.py
│ │ ├── emit_find_links.py
│ │ ├── emit_trusted_host.py
│ │ ├── extra_index_url.py
│ │ ├── file_extensions.py
│ │ ├── forbid_post.py
│ │ ├── forward.py
│ │ ├── header.py
│ │ ├── limit_in_paths.py
│ │ ├── live_output.py
│ │ ├── skip_constraint_comments.py
│ │ ├── strip_extras.py
│ │ ├── unsafe.py
│ │ ├── upgrade.py
│ │ ├── use_cache.py
│ │ └── use_uv.py
│ ├── options.py
│ ├── utils.py
│ └── verify.py
├── requirements/
│ ├── base.hash
│ ├── base.in
│ ├── base.txt
│ ├── local.hash
│ ├── local.in
│ ├── local.txt
│ ├── test.hash
│ ├── test.in
│ ├── test.txt
│ ├── testwin.hash
│ ├── testwin.in
│ └── testwin.txt
├── setup.cfg
├── setup.py
├── tests/
│ ├── __init__.py
│ ├── configs/
│ │ ├── pyproject.toml
│ │ ├── requirements.ini
│ │ └── setup.cfg
│ ├── conflicting-in-merge/
│ │ ├── base1.in
│ │ ├── base1.txt
│ │ ├── base2.in
│ │ ├── base2.txt
│ │ ├── together.in
│ │ └── together.txt
│ ├── conflicting-in-ref/
│ │ ├── base1.in
│ │ ├── base1.txt
│ │ ├── base2.in
│ │ └── base2.txt
│ ├── conftest.py
│ ├── sys_platform/
│ │ ├── base.in
│ │ ├── base.txt
│ │ ├── test.in
│ │ └── test.txt
│ ├── test_add_hashes.py
│ ├── test_cli_v1.py
│ ├── test_cli_v2.py
│ ├── test_config.py
│ ├── test_conflicts.py
│ ├── test_deduplicate.py
│ ├── test_dependency.py
│ ├── test_discover.py
│ ├── test_pipcompilemulti.py
│ ├── test_skip_constraint_comments.py
│ ├── test_upgrade_feature.py
│ ├── test_utils.py
│ ├── upgrade/
│ │ ├── base.in
│ │ └── base.txt
│ ├── upgrade-autoresolve-with-range/
│ │ ├── base.in
│ │ ├── base.txt
│ │ ├── prod.in
│ │ └── prod.txt
│ ├── upgrade-autoresolve-with-range-expected/
│ │ ├── base.in
│ │ ├── base.txt
│ │ ├── prod.in
│ │ └── prod.txt
│ ├── upgrade-expected/
│ │ ├── base.in
│ │ └── base.txt
│ ├── upgrade-with-range/
│ │ ├── base.in
│ │ └── base.txt
│ ├── upgrade-with-range-expected/
│ │ ├── base.in
│ │ └── base.txt
│ └── utils.py
└── tox.ini
================================================
FILE CONTENTS
================================================
================================================
FILE: .coveragerc
================================================
# .coveragerc to control coverage.py
[run]
branch = True
[report]
# Regexes for lines to exclude from consideration
exclude_lines =
# Have to re-enable the standard pragma
pragma: no cover
# Don't complain about missing debug-only code:
def __repr__
if self\.debug
# Don't complain if tests don't hit defensive assertion code:
raise AssertionError
raise NotImplementedError
# Don't complain if non-runnable code isn't run:
if 0:
if __name__ == .__main__.:
ignore_errors = True
[html]
directory = htmlcov
================================================
FILE: .editorconfig
================================================
# http://editorconfig.org
root = true
[*]
indent_style = space
indent_size = 4
trim_trailing_whitespace = true
insert_final_newline = true
charset = utf-8
end_of_line = lf
[*.bat]
indent_style = tab
end_of_line = crlf
[LICENSE]
insert_final_newline = false
[Makefile]
indent_style = tab
================================================
FILE: .github/workflows/pipcompilemulti.yml
================================================
name: Update Dependencies
on:
schedule:
- cron: '15 15 * * 3'
workflow_dispatch: {}
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.10
uses: actions/setup-python@v4
with:
python-version: '3.10'
- name: Install tox
run: |
python -m pip install --upgrade pip
pip install tox
- name: Update dependencies
run: |
tox -e upgrade
- name: Create commits
run: |
git config user.name 'pip-compile-multi'
git config user.email 'pip-compile-multi@users.noreply.github.com'
git commit -am "Update dependencies"
- name: Create Pull Request
uses: peter-evans/create-pull-request@v3
with:
author: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
token: ${{ secrets.PAT }}
title: Update dependencies
body: |
Auto-generated by [pip-compile-multi](https://github.com/peterdemin/pip-compile-multi)
branch: update-dependencies
branch-suffix: timestamp
================================================
FILE: .github/workflows/python310-windows.yml
================================================
name: Python 3.10 Windows
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
build:
runs-on: windows-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.10
uses: actions/setup-python@v4
with:
python-version: '3.10'
- name: Install tox
run: |
python -m pip install --upgrade pip
python -m pip install tox
- name: Run tox
run: |
tox -e py310-windows
================================================
FILE: .github/workflows/python310.yml
================================================
name: Python 3.10
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.10
uses: actions/setup-python@v4
with:
python-version: '3.10'
- name: Install tox
run: |
python -m pip install --upgrade pip
python -m pip install tox
- name: Run tox
run: |
tox
================================================
FILE: .github/workflows/python311.yml
================================================
name: Python 3.11
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.11
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Install tox
run: |
python -m pip install --upgrade pip
python -m pip install tox
- name: Run tox
run: |
tox -e py311-linux || true
================================================
FILE: .github/workflows/python312.yml
================================================
name: Python 3.12
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.12
uses: actions/setup-python@v4
with:
python-version: '3.12'
- name: Install tox
run: |
python -m pip install --upgrade pip
python -m pip install tox
- name: Run tox
run: |
tox -e py312-linux || true
================================================
FILE: .github/workflows/python313.yml
================================================
name: Python 3.13
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.13
uses: actions/setup-python@v4
with:
python-version: '3.13'
- name: Install tox
run: |
python -m pip install --upgrade pip
python -m pip install tox
- name: Run tox
run: |
tox -e py313-linux || true
================================================
FILE: .github/workflows/python314.yml
================================================
name: Python 3.14
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.14
uses: actions/setup-python@v4
with:
python-version: '3.14'
- name: Install tox
run: |
python -m pip install --upgrade pip
python -m pip install tox
- name: Run tox
run: |
tox -e py314-linux || true
================================================
FILE: .gitignore
================================================
*.py[cod]
/.env/
/.venv*/
# C extensions
*.so
# Packages
*.egg
*.egg-info
dist
build
eggs
parts
bin
var
sdist
develop-eggs
.installed.cfg
lib
lib64
# Installer logs
pip-log.txt
# Unit test / coverage reports
.coverage
.tox
nosetests.xml
/.pytest_cache/
# Conflict tests side-effects
/conflicting-in-*/*.txt
# Translations
*.mo
# Mr Developer
.mr.developer.cfg
.project
.pydevproject
# Complexity
output/*.html
output/*/index.html
# Sphinx
docs/_build
# Cookiecutter
output/
boilerplate/
/.cache/
/.eggs/
# Vim
Session.vim
/htmlcov/
/.idea/
*.swp
================================================
FILE: .pre-commit-config.yaml
================================================
repos:
- repo: https://github.com/peterdemin/pip-compile-multi
rev: v2.6.2
hooks:
- id: pip-compile-multi-verify
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v2.1.0
hooks:
- id: flake8
- id: trailing-whitespace
================================================
FILE: .pre-commit-hooks.yaml
================================================
- id: pip-compile-multi-verify
name: pip-compile-multi verify
language: python
entry: pip-compile-multi verify
files: ^requirements/
pass_filenames: false
require_serial: true
types: [file, non-executable, text]
================================================
FILE: .pylintrc
================================================
[MASTER]
# A comma-separated list of package or module names from where C extensions may
# be loaded. Extensions are loading into the active Python interpreter and may
# run arbitrary code
extension-pkg-whitelist=
# Add files or directories to the blacklist. They should be base names, not
# paths.
ignore=CVS
# Add files or directories matching the regex patterns to the blacklist. The
# regex matches against base names, not paths.
ignore-patterns=
# Python code to execute, usually for sys.path manipulation such as
# pygtk.require().
#init-hook=
# Use multiple processes to speed up Pylint.
jobs=1
# List of plugins (as comma separated values of python modules names) to load,
# usually to register additional checkers.
load-plugins=
# Pickle collected data for later comparisons.
persistent=yes
# Specify a configuration file.
#rcfile=
# Allow loading of arbitrary C extensions. Extensions are imported into the
# active Python interpreter and may run arbitrary code.
unsafe-load-any-extension=no
[MESSAGES CONTROL]
# Only show warnings with the listed confidence levels. Leave empty to show
# all. Valid levels: HIGH, INFERENCE, INFERENCE_FAILURE, UNDEFINED
confidence=
# Disable the message, report, category or checker with the given id(s). You
# can either give multiple identifiers separated by comma (,) or put this
# option multiple times (only on the command line, not in the configuration
# file where it should appear only once).You can also use "--disable=all" to
# disable everything first and then reenable specific checks. For example, if
# you want to run only the similarities checker, you can use "--disable=all
# --enable=similarities". If you want to run only the classes checker, but have
# no Warning level messages displayed, use"--disable=all --enable=classes
# --disable=W"
disable=bad-inline-option,
consider-using-f-string,
deprecated-pragma,
file-ignored,
locally-disabled,
no-else-return,
raw-checker-failed,
suppressed-message,
useless-object-inheritance,
useless-suppression
# Enable the message, report, category or checker with the given id(s). You can
# either give multiple identifier separated by comma (,) or put this option
# multiple time (only on the command line, not in the configuration file where
# it should appear only once). See also the "--disable" option for examples.
enable=c-extension-no-member
[REPORTS]
# Python expression which should return a note less than 10 (10 is the highest
# note). You have access to the variables errors warning, statement which
# respectively contain the number of errors / warnings messages and the total
# number of statements analyzed. This is used by the global evaluation report
# (RP0004).
evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10)
# Template used to display messages. This is a python new-style format string
# used to format the message information. See doc for all details
#msg-template=
# Set the output format. Available formats are text, parseable, colorized, json
# and msvs (visual studio).You can also give a reporter class, eg
# mypackage.mymodule.MyReporterClass.
output-format=text
# Tells whether to display a full report or only the messages
reports=no
# Activate the evaluation score.
score=yes
[REFACTORING]
# Maximum number of nested blocks for function / method body
max-nested-blocks=5
# Complete name of functions that never returns. When checking for
# inconsistent-return-statements if a never returning function is called then
# it will be considered as an explicit return statement and no message will be
# printed.
never-returning-functions=optparse.Values,sys.exit
[SIMILARITIES]
# Ignore comments when computing similarities.
ignore-comments=yes
# Ignore docstrings when computing similarities.
ignore-docstrings=yes
# Ignore imports when computing similarities.
ignore-imports=no
# Minimum lines number of a similarity.
min-similarity-lines=4
[FORMAT]
# Expected format of line ending, e.g. empty (any line ending), LF or CRLF.
expected-line-ending-format=
# Regexp for a line that is allowed to be longer than the limit.
ignore-long-lines=^\s*(# )?<?https?://\S+>?$
# Number of spaces of indent required inside a hanging or continued line.
indent-after-paren=4
# String used as indentation unit. This is usually " " (4 spaces) or "\t" (1
# tab).
indent-string=' '
# Maximum number of characters on a single line.
max-line-length=100
# Maximum number of lines in a module
max-module-lines=1000
# Allow the body of a class to be on the same line as the declaration if body
# contains single statement.
single-line-class-stmt=no
# Allow the body of an if to be on the same line as the test if there is no
# else.
single-line-if-stmt=no
[MISCELLANEOUS]
# List of note tags to take in consideration, separated by a comma.
notes=FIXME,
XXX,
TODO
[SPELLING]
# Limits count of emitted suggestions for spelling mistakes
max-spelling-suggestions=4
# Spelling dictionary name. Available dictionaries: none. To make it working
# install python-enchant package.
spelling-dict=
# List of comma separated words that should not be checked.
spelling-ignore-words=
# A path to a file that contains private dictionary; one word per line.
spelling-private-dict-file=
# Tells whether to store unknown words to indicated private dictionary in
# --spelling-private-dict-file option instead of raising a message.
spelling-store-unknown-words=no
[LOGGING]
# Logging modules to check that the string format arguments are in logging
# function parameter format
logging-modules=logging
[BASIC]
# Naming style matching correct argument names
argument-naming-style=snake_case
# Regular expression matching correct argument names. Overrides argument-
# naming-style
#argument-rgx=
# Naming style matching correct attribute names
attr-naming-style=snake_case
# Regular expression matching correct attribute names. Overrides attr-naming-
# style
#attr-rgx=
# Bad variable names which should always be refused, separated by a comma
bad-names=foo,
bar,
baz,
toto,
tutu,
tata
# Naming style matching correct class attribute names
class-attribute-naming-style=any
# Regular expression matching correct class attribute names. Overrides class-
# attribute-naming-style
#class-attribute-rgx=
# Naming style matching correct class names
class-naming-style=PascalCase
# Regular expression matching correct class names. Overrides class-naming-style
#class-rgx=
# Naming style matching correct constant names
const-naming-style=UPPER_CASE
# Regular expression matching correct constant names. Overrides const-naming-
# style
#const-rgx=
# Minimum line length for functions/classes that require docstrings, shorter
# ones are exempt.
docstring-min-length=-1
# Naming style matching correct function names
function-naming-style=snake_case
# Regular expression matching correct function names. Overrides function-
# naming-style
# Allow long test names
function-rgx=[a-z_][a-z0-9_]{2,70}$
# Good variable names which should always be accepted, separated by a comma
good-names=i,
j,
k,
ex,
Run,
_,
fp,
logger,
options
# Include a hint for the correct naming format with invalid-name
include-naming-hint=no
# Naming style matching correct inline iteration names
inlinevar-naming-style=any
# Regular expression matching correct inline iteration names. Overrides
# inlinevar-naming-style
#inlinevar-rgx=
# Naming style matching correct method names
method-naming-style=snake_case
# Regular expression matching correct method names. Overrides method-naming-
# style
#method-rgx=
# Naming style matching correct module names
module-naming-style=snake_case
# Regular expression matching correct module names. Overrides module-naming-
# style
#module-rgx=
# Colon-delimited sets of names that determine each other's naming style when
# the name regexes allow several styles.
name-group=
# Regular expression which should only match function or class names that do
# not require a docstring.
no-docstring-rgx=^_
# List of decorators that produce properties, such as abc.abstractproperty. Add
# to this list to register other decorators that produce valid properties.
property-classes=abc.abstractproperty
# Naming style matching correct variable names
variable-naming-style=snake_case
# Regular expression matching correct variable names. Overrides variable-
# naming-style
#variable-rgx=
[TYPECHECK]
# List of decorators that produce context managers, such as
# contextlib.contextmanager. Add to this list to register other decorators that
# produce valid context managers.
contextmanager-decorators=contextlib.contextmanager
# List of members which are set dynamically and missed by pylint inference
# system, and so shouldn't trigger E1101 when accessed. Python regular
# expressions are accepted.
generated-members=
# Tells whether missing members accessed in mixin class should be ignored. A
# mixin class is detected if its name ends with "mixin" (case insensitive).
ignore-mixin-members=yes
# This flag controls whether pylint should warn about no-member and similar
# checks whenever an opaque object is returned when inferring. The inference
# can return multiple potential results while evaluating a Python object, but
# some branches might not be evaluated, which results in partial inference. In
# that case, it might be useful to still emit no-member and other checks for
# the rest of the inferred objects.
ignore-on-opaque-inference=yes
# List of class names for which member attributes should not be checked (useful
# for classes with dynamically set attributes). This supports the use of
# qualified names.
ignored-classes=optparse.Values,thread._local,_thread._local
# List of module names for which member attributes should not be checked
# (useful for modules/projects where namespaces are manipulated during runtime
# and thus existing member attributes cannot be deduced by static analysis. It
# supports qualified module names, as well as Unix pattern matching.
ignored-modules=
# Show a hint with possible names when a member name was not found. The aspect
# of finding the hint is based on edit distance.
missing-member-hint=yes
# The minimum edit distance a name should have in order to be considered a
# similar match for a missing member name.
missing-member-hint-distance=1
# The total number of similar names that should be taken in consideration when
# showing a hint for a missing member.
missing-member-max-choices=1
[VARIABLES]
# List of additional names supposed to be defined in builtins. Remember that
# you should avoid to define new builtins when possible.
additional-builtins=
# Tells whether unused global variables should be treated as a violation.
allow-global-unused-variables=yes
# List of strings which can identify a callback function by name. A callback
# name must start or end with one of those strings.
callbacks=cb_,
_cb
# A regular expression matching the name of dummy variables (i.e. expectedly
# not used).
dummy-variables-rgx=_+$|(_[a-zA-Z0-9_]*[a-zA-Z0-9]+?$)|dummy|^ignored_|^unused_
# Argument names that match this expression will be ignored. Default to name
# with leading underscore
ignored-argument-names=_.*|^ignored_|^unused_
# Tells whether we should check for unused import in __init__ files.
init-import=no
# List of qualified module names which can have objects that can redefine
# builtins.
redefining-builtins-modules=six.moves,past.builtins,future.builtins
[CLASSES]
# List of method names used to declare (i.e. assign) instance attributes.
defining-attr-methods=__init__,
__new__,
setUp
# List of member names, which should be excluded from the protected access
# warning.
exclude-protected=_asdict,
_fields,
_replace,
_source,
_make
# List of valid names for the first argument in a class method.
valid-classmethod-first-arg=cls
# List of valid names for the first argument in a metaclass class method.
valid-metaclass-classmethod-first-arg=mcs
[IMPORTS]
# Allow wildcard imports from modules that define __all__.
allow-wildcard-with-all=no
# Analyse import fallback blocks. This can be used to support both Python 2 and
# 3 compatible code, which means that the block might have code that exists
# only in one or another interpreter, leading to false positives when analysed.
analyse-fallback-blocks=no
# Deprecated modules which should not be used, separated by a comma
deprecated-modules=optparse,tkinter.tix
# Create a graph of external dependencies in the given file (report RP0402 must
# not be disabled)
ext-import-graph=
# Create a graph of every (i.e. internal and external) dependencies in the
# given file (report RP0402 must not be disabled)
import-graph=
# Create a graph of internal dependencies in the given file (report RP0402 must
# not be disabled)
int-import-graph=
# Force import order to recognize a module as part of the standard
# compatibility libraries.
known-standard-library=
# Force import order to recognize a module as part of a third party library.
known-third-party=enchant
[DESIGN]
# Maximum number of arguments for function / method
max-args=500
# Maximum number of attributes for a class (see R0902).
max-attributes=7
# Maximum number of boolean expressions in a if statement
max-bool-expr=5
# Maximum number of branch for function / method body
max-branches=12
# Maximum number of locals for function / method body
max-locals=15
# Maximum number of parents for a class (see R0901).
max-parents=7
# Maximum number of public methods for a class (see R0904).
max-public-methods=20
# Maximum number of return / yield for function / method body
max-returns=6
# Maximum number of statements in function / method body
max-statements=50
# Minimum number of public methods for a class (see R0903).
min-public-methods=2
[EXCEPTIONS]
# Exceptions that will emit a warning when being caught. Defaults to
# "Exception"
overgeneral-exceptions=builtins.Exception
================================================
FILE: .readthedocs.yml
================================================
# .readthedocs.yml
# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
# Required
version: 2
# Build documentation in the docs/ directory with Sphinx
sphinx:
configuration: docs/conf.py
# Optionally build your docs in additional formats such as PDF and ePub
formats: all
# Optionally set the version of Python and requirements required to build your docs
build:
os: ubuntu-22.04
tools:
python: "3.9"
commands:
- pip install -r requirements/local.hash
- pip install -e .
- make -C docs html
- mkdir -p $READTHEDOCS_OUTPUT/
- mv docs/_build/html $READTHEDOCS_OUTPUT/
================================================
FILE: .travis.yml
================================================
sudo: false
language: python
python:
- "3.6"
- "3.7"
- "3.8"
matrix:
include:
- os: linux
python: 3.6
env: TOXENV=lint
before_install:
- pip install codecov coveralls
install: pip install tox-travis
script: tox
after_success:
- codecov
- coveralls
deploy:
provider: pypi
user: peterdemin
password:
secure: "ZZTWaBiHQicobKw/xiD6IGQoAdrGmZ/DAXXitbnTE4jGpDKq1X+0JDNbYo1EVXQ/r+wmL3nBTz1DPrEymo//hli/4GzffargF7lnWg7GfrUSAxWUSWj36+AAyRLWGqwhSPQCxDymAi6TJF5VSKw1qDrs7vicjSOwtXnUxBANMaBxeFxhJJbhQpViBCTs6w5ZsPRFaWIVkxdOSFQY09ZgDBU17VYaVyzB5okZ5Ogk+3Xj5cqZKPDrptmuYSjdHm/LEj5q6gesuOOYTXfloqLirSTxPedo+gLUHoUIDgvPhNm1VJiiBx+d5DjIKhsOeV385rGYWUq92EznfrdPmorAmh62KkP4pTL1+Jd9gcS/FIXiMk3ga9tQLH51dqkRRgwgM5HMVYZIyl+D2I2Nw6RdMcViJZkv8VV5wF/58JiHdIdiutgo0Y14dtbUjryDoz5Ivsgp2NodyA/AJ97NCf8CVrYxOJivjEJ5tkP/ANGzg3/gDTy3t1qVRBaNyDRQK77TwbvpR9LWODZXKuSh7WXeZpflSYT1PL8mLtBgho0onzXPjmgyYgGMEDEAx+kaXc5xLj2uH1Z+RC7bXbYkeAxjnpkxwWKNcCnnpELE56732JYqdDh8Pjc6eJtPCS4mAFxNPAafR4wl6NNWOgT4vjnh6ZJAGeFBIKw21AWsYI99mc4="
on:
tags: true
distributions: "sdist bdist_wheel"
skip_existing: true
================================================
FILE: AUTHORS.rst
================================================
=======
Credits
=======
Development Lead
----------------
* Peter Demin <peterdemin@gmail.com>
Contributors
------------
None yet. Why not be the first?
================================================
FILE: CONTRIBUTING.rst
================================================
============
Contributing
============
Contributions are welcome, and they are greatly appreciated! Every
little bit helps, and credit will always be given.
You can contribute in many ways:
Types of Contributions
----------------------
Report Bugs
~~~~~~~~~~~
Report bugs at https://github.com/peterdemin/pip-compile-multi/issues.
If you are reporting a bug, please include:
* Your operating system name and version.
* Any details about your local setup that might be helpful in troubleshooting.
* Detailed steps to reproduce the bug.
Fix Bugs
~~~~~~~~
Look through the GitHub issues for bugs. Anything tagged with "bug"
is open to whoever wants to implement it.
Implement Features
~~~~~~~~~~~~~~~~~~
Look through the GitHub issues for features. Anything tagged with "feature"
is open to whoever wants to implement it.
Write Documentation
~~~~~~~~~~~~~~~~~~~
pip-compile-multi could always use more documentation, whether as part of the
official pip-compile-multi docs, in docstrings, or even on the web in blog posts,
articles, and such.
Submit Feedback
~~~~~~~~~~~~~~~
The best way to send feedback is to file an issue at https://github.com/peterdemin/pip-compile-multi/issues.
If you are proposing a feature:
* Explain in detail how it would work.
* Keep the scope as narrow as possible, to make it easier to implement.
* Remember that this is a volunteer-driven project, and that contributions
are welcome :)
Get Started!
------------
Ready to contribute? Here's how to set up `pip-compile-multi` for local development.
1. Fork the `pip-compile-multi` repo on GitHub.
2. Clone your fork locally::
$ git clone git@github.com:your_name_here/pip-compile-multi.git
3. Install your local copy into a virtualenv. Assuming you have virtualenvwrapper installed, this is how you set up your fork for local development::
$ mkvirtualenv pip-compile-multi
$ cd pip-compile-multi/
$ python setup.py develop
4. Create a branch for local development::
$ git checkout -b name-of-your-bugfix-or-feature
Now you can make your changes locally.
5. When you're done making changes, check that your changes pass flake8 and the tests, including testing other Python versions with tox::
$ flake8 pip-compile-multi.py test_pip-compile-multi.py
$ py.test
$ tox
To get flake8 and tox, just pip install them into your virtualenv.
6. Commit your changes and push your branch to GitHub::
$ git add .
$ git commit -m "Your detailed description of your changes."
$ git push origin name-of-your-bugfix-or-feature
7. Submit a pull request through the GitHub website.
Pull Request Guidelines
-----------------------
Before you submit a pull request, check that it meets these guidelines:
1. The pull request should include tests.
2. If the pull request adds functionality, the docs should be updated. Put
your new functionality into a function with a docstring, and add the
feature to the list in README.rst.
3. The pull request should work for Python 2.6, 2.7, 3.3, and 3.4, and for PyPy. Check
https://travis-ci.org/peterdemin/pip-compile-multi/pull_requests
and make sure that the tests pass for all supported Python versions.
Tips
----
To run a subset of tests::
TODO
================================================
FILE: Dockerfile
================================================
FROM python:3.10
RUN apt-get update && apt-get install -y build-essential
WORKDIR /pcm
================================================
FILE: HISTORY.rst
================================================
History
=======
3.3.1 (2025-04-19)
------------------
* Add support for `--emit-find-links/--no-emit-find-links`.
(Issue `#554`_, PR `#556`_, thanks to `Aidas Bendoraitis`_.)
.. _#554: https://github.com/peterdemin/pip-compile-multi/issues/554
.. _#556: https://github.com/peterdemin/pip-compile-multi/pull/556
.. _Aidas Bendoraitis: https://github.com/archatas
3.3.0 (2025-03-25)
------------------
* Add Python 3.14 to supported versions.
* Remove Python 3.9 from supported versions.
* Using Python 3.10 as the base version going forward.
3.2.2 (2025-06-19)
------------------
* Fix ``TypeError`` is not triggered when sections is None.
(Issue `#516`_, PR `#517`_, thanks to `Nikola Trandafilovic`_.)
.. _#516: https://github.com/peterdemin/pip-compile-multi/issues/516
.. _#517: https://github.com/peterdemin/pip-compile-multi/pull/517
.. _Nikola Trandafilovic: https://github.com/elrik
3.2.1 (2025-06-19)
------------------
* Fix a bug when loading configuration from ``pyproject.toml`` that doesn't have ``[tool.requirements]`` section.
3.2.0 (2025-06-17)
------------------
* Enable annotations for uv resolver. (Requires uv>=0.7)
(Issue `#505`_)
.. _#505: https://github.com/peterdemin/pip-compile-multi/issues/505
3.1.0 (2025-05-23)
------------------
* Add support for ``pyproject.toml`` for ``requirements`` configuration.
(Issue `#496`_, PR `#498`_, thanks to `Abir A`_.)
.. _#496: https://github.com/peterdemin/pip-compile-multi/issues/496
.. _#498: https://github.com/peterdemin/pip-compile-multi/pull/498
.. _Abir A: https://github.com/Some7hing0riginal
3.0.0 (2025-04-23)
------------------
* Add second-generation CLI entrypoint ``requirements``.
* Make ``--use-cache`` enabled by default.
* Make ``--skip-constraints`` enabled by default.
* Removed deprecated ``-n, --only-name`` option. Use ``-t, --only-path`` instead.
2.8.0 (2024-11-27)
------------------
* Add ``--uv`` flag to use UV instead of pip-tools.
(PR `#483`_, thanks to `Mustafa Tevfik Kadan`_).
.. _#483: https://github.com/peterdemin/pip-compile-multi/pull/483
.. _Mustafa Tevfik Kadan: https://github.com/ktevfik
2.7.0 (2024-11-27)
------------------
* Remove Python 3.8 from tested versions.
Using Python 3.9 as the base version going forward.
2.6.4 (2023-06-06)
------------------
* Add ``--strip-extras`` pass-through flag.
(PR `#455`_, thanks to `Tim Vergenz`_).
.. _#455: https://github.com/peterdemin/pip-compile-multi/pull/455
.. _Tim Vergenz: https://github.com/vergenzt
2.6.3 (2023-05-05)
------------------
* Allow version constraints in ``--upgrade-package`` parameters.
(Issue `#392`_, PR `#394`_, thanks to `Peter Law`_).
.. _#392: https://github.com/peterdemin/pip-compile-multi/issues/392
.. _#394: https://github.com/peterdemin/pip-compile-multi/pull/394
.. _Peter Law: https://github.com/PeterJCLaw
2.6.2 (2023-02-23)
------------------
* Fix package name normalization for names with delimiters (``urltemplate`` != ``url-template``).
2.6.1 (2022-11-23)
------------------
* Add ``--backtracking/--no-backtracking`` flag
(`backtracking docs <https://pip-compile-multi.readthedocs.io/en/latest/features.html#backtracking-resolver>`_).
(Issue `#345`_, PR `#360`_)
* Bumped ``pip-tools`` minimum version constraint to ``6.8.0`` to support ``--resolver`` option.
.. _#345: https://github.com/peterdemin/pip-compile-multi/issues/345
.. _#360: https://github.com/peterdemin/pip-compile-multi/pull/360
2.6.0 (2022-11-23)
------------------
* Fix cross-env package names matching when they use delimiters (-_.) and upper/lower case.
2.5.0 (2022-11-03)
------------------
* Add ``--emit_trusted_host/--no-emit_trusted_host`` flag
(`trusted host annotation docs <https://pip-compile-multi.readthedocs.io/en/latest/features.html#add-trusted-host-annotation>`_).
Thanks to `Phil Blackwood`_
(Issue `#351`_, PR `#353`_).
* Remove Python 3.6 and 3.7 from tested versions
(package works, but maintaining CI becomes problematic).
Using Python 3.8 as the base version going forward.
.. _Phil Blackwood: https://github.com/philblckwd
.. _#351: https://github.com/peterdemin/pip-compile-multi/issues/351
.. _#353: https://github.com/peterdemin/pip-compile-multi/pull/353
2.4.6 (2022-07-22)
------------------
* Add support for `PEP-426: environment markers <https://peps.python.org/pep-0426/>`_.
2.4.5 (2022-04-01)
------------------
* Fix ``--build-isolation`` flag passing.
Thanks to `Jake Schmidt`_
(Issue `#312`_, PR `#313`_).
.. _#312: https://github.com/peterdemin/pip-compile-multi/issues/312
.. _#313: https://github.com/peterdemin/pip-compile-multi/pull/313
2.4.4 (2022-03-30)
------------------
* Add ``--build-isolation`` flag.
Thanks to `Jake Schmidt`_
(PR `#311`_).
* Fix ``--skip-constraints`` feature for URL dependencies.
.. _Jake Schmidt: https://github.com/schmidt-jake
.. _#311: https://github.com/peterdemin/pip-compile-multi/pull/311
2.4.3 (2022-01-19)
------------------
* Prioritize URL depedencies when parsing to enable URLs that have version pins.
* Make ``--autoresolve`` work correctly with ``-t`` and ``-n`` options.
* Add support for "package @ url" notation.
2.4.2 (2021-09-01)
------------------
* Fix ``--autoresolve`` when ``--directory="."``.
2.4.1 (2021-04-02)
------------------
* Add ``--live/--no-live`` option to control when to print debug output.
Thanks to `John Sandall`_ and `Thomas Grainger`_
(Issue `#153`_, PRs `#247`_ and `#251`_).
* Add ``--extra-index-url <url>`` option.
Thanks to `Erik Jan de Vries`_
(PRs `#250`_ and `#252`_).
.. _#153: https://github.com/peterdemin/pip-compile-multi/issues/153
.. _#247: https://github.com/peterdemin/pip-compile-multi/pull/247
.. _#250: https://github.com/peterdemin/pip-compile-multi/pull/250
.. _#251: https://github.com/peterdemin/pip-compile-multi/pull/251
.. _#252: https://github.com/peterdemin/pip-compile-multi/pull/252
.. _John Sandall: https://github.com/john-sandall
.. _Thomas Grainger: https://github.com/graingert
.. _Erik Jan de Vries: https://github.com/erikjandevries
2.4.0 (2021-03-17)
------------------
* Update --index/--no-index to --emit-index-url/--no-emit-index-url
for compatibility with pip-tools 6.0
(Issue `#243`_).
.. _#243: https://github.com/peterdemin/pip-compile-multi/issues/243
2.3.2 (2021-02-18)
------------------
* Fix cross-feature logic for --autoresolve and --upgrade-package
(PR `#236`_).
.. _#236: https://github.com/peterdemin/pip-compile-multi/pull/236
2.3.1 (2021-02-16)
------------------
* Fix for a bug introduced in 2.2.2 when running pip-compile-multi
installed for Python 3, and having ``python`` symlinked to Python 2
(Issue `#233`_, PR `#234`_).
.. _#233: https://github.com/peterdemin/pip-compile-multi/issues/233
.. _#234: https://github.com/peterdemin/pip-compile-multi/pull/234
2.3.0 (2021-02-04)
------------------
* Make SHA1 hashes of input files in a more robust way (Issue `#215`_).
Now it ignores changes to comments, whitespace and order of packages.
.. _#215: https://github.com/peterdemin/pip-compile-multi/issues/215
2.2.2 (2021-01-29)
------------------
* Add support for calling using `python -m pipcompilemulti.cli_v1` notation.
2.2.1 (2021-01-29)
------------------
* Add ``--skip-constraints`` option.
* Fix bootstrapping for autoresolve case with missing output files.
2.2.0 (2020-01-22)
------------------
* Add ``--autoresolve`` option for conflict-free compilations (PR #224).
* Auto-discover requirements in other directories by following references (PR #221).
* Add support for new-style multiline "via" comments from pip-tools (PR #222).
2.1.0 (2020-08-19)
------------------
* Update dependencies.
* Revert relative path normalization, introduced in #167 (thanks to @john-bodley #200).
2.0.0 (2020-05-18)
------------------
* Drop Python 2.7 support. pip-tools 4 no longer works with the latest pip,
there's no way to continue Python 2.7 support.
1.5.9 (2020-03-23)
------------------
* Remove directory path from "via" annotations (thanks to @HALtheWise #166 #167).
1.5.8 (2019-09-27)
------------------
* Add option ``--annotate-index`` (thanks to @john-bodley #160).
1.5.7 (2019-09-27)
------------------
* Enable accidentially disabled ``--upgrade`` option.
.. _1.5.6:
1.5.6 (2019-09-18)
------------------
* Minor fixes to packaging and documentation.
Warning: this version is broken and won't pass ``--upgrade`` option to ``pip-compile``.
If you have this version installed, you need to manually upgrade it.
For example, using command::
pip-compile-multi --upgrade-package pip-compile-multi
Like in this `PR <https://github.com/mozilla-releng/shipit/pull/1>`_.
1.5.4 (2019-09-16)
------------------
* Fixed MANIFEST to include features directory
Warning: this version is broken and won't pass ``--upgrade`` option to ``pip-compile``.
See notes for 1.5.6_ for details.
1.5.3 (2019-09-14)
------------------
* Refactored features to separate modules.
* Allow passing verify options after verify command.
* Trim irrelevant entries from the traceback.
Warning: this version is broken and won't install ``features`` directory.
See notes for 1.5.6_ for details.
1.5.2 (2019-09-12)
------------------
* Added option ``--allow-unsafe``. (thanks to @mozbhearsum #157).
1.5.1 (2019-08-08)
------------------
* Added option ``--use-cache``. (thanks to @kolotev #149).
1.5.0 (2019-08-06)
------------------
* Changed short option for ``--forbid-post`` from ``-P`` to ``-p``
(as it conflicted with ``-P`` for ``--upgrade-package`` #147).
1.3.1 (2019-02-19)
------------------
* Re-removed workaround for future[s] packages in Python3
1.3.0 (2018-12-27)
------------------
* Introduced CLI v2 (disabled by default)
1.2.2 (2018-11-20)
------------------
* Removed workaround for future[s] packages in Python3 (no longer needed)
1.2.1 (2018-04-16)
-------------------
* Fixed Restructured text formatting (thanks to @yigor)
* Updated test dependencies (and hashes)
1.2.0 (2018-04-03)
-------------------
* Added --forbid-post option
1.1.12 (2018-02-23)
-------------------
* Added checks for conflicting package versions
* Added support for VCS dependencies
* Added --no-upgrade option
1.1.11 (2018-02-09)
-------------------
* Propagate --only-name option to references
* Fixed extension override options
1.1.10 (2018-02-09)
-------------------
* Added ``--generate-hashes`` option
1.1.9 (2018-02-08)
------------------
* Fixed directory override option
* Added --only-name option
1.1.8 (2018-01-25)
------------------
* Fixed comment justification
1.1.6 (2018-01-19)
------------------
* Added ``pip-compile-multi verify`` command
1.1.5 (2018-01-16)
------------------
* Omit future[s] packages for Python3
1.1.0 (2018-01-12)
------------------
* Added files discovery.
1.0.0 (2018-01-11)
------------------
* First release on PyPI.
================================================
FILE: LICENSE.txt
================================================
MIT License
Copyright (c) [year] [fullname]
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: MANIFEST.in
================================================
include AUTHORS.rst
include CONTRIBUTING.rst
include HISTORY.rst
include LICENSE
include requirements/base.in
================================================
FILE: Makefile
================================================
.PHONY: virtual_env_set
virtual_env_set:
ifndef VIRTUAL_ENV
$(error VIRTUAL_ENV not set)
endif
### DEPENDENCIES ###
.PHONY: install
install: requirements/local.hash virtual_env_set
pip install -r requirements/local.hash
pip install -e . --no-deps
.PHONY: sync
sync: requirements/local.hash virtual_env_set
pip-sync requirements/local.hash
pip install -e . --no-deps
.PHONY: %-docker
%-docker: ## Could be lock-docker or upgrade-docker
docker run --rm -it -v $(PWD):/pcm $$(docker build -q .) /usr/bin/make $*-ubuntu
.PHONY: %-ubuntu
%-ubuntu: .venv3
.venv3/bin/python3 -m pip install tox
.venv3/bin/python3 -m tox -e $*
.venv3:
python3.10 -m venv .venv3
.PHONY: lock
lock: virtual_env_set
tox -e lock
.PHONY: upgrade
upgrade: virtual_env_set
tox -e upgrade
### CI ###
.PHONY: test
test:
tox
.PHONY: clean
clean:
rm -rf build dist pip-compile-multi.egg-info docs/_build
rm -rf .venv3
find . -name "*.pyc" -delete
find * -type d -name '__pycache__' | xargs rm -rf
.PHONY: build
build: clean
python setup.py sdist bdist_wheel
.PHONY: release
release: build
twine upload dist/*
### MISC ###
.PHONY: docs
docs: virtual_env_set
make -C docs html
================================================
FILE: README.rst
================================================
=================
pip-compile-multi
=================
.. image:: https://badge.fury.io/py/pip-compile-multi.png
:target: https://badge.fury.io/py/pip-compile-multi
.. image:: https://github.com/peterdemin/pip-compile-multi/actions/workflows/python38.yml/badge.svg
:target: https://github.com/peterdemin/pip-compile-multi/actions/workflows/python38.yml
.. image:: https://img.shields.io/pypi/pyversions/pip-compile-multi.svg
:target: https://pypi.python.org/pypi/pip-compile-multi
`Docs <https://pip-compile-multi.readthedocs.io/en/latest/>`_
Compile multiple requirements files to lock dependency versions.
Install
-------
.. code-block:: shell
pip install pip-compile-multi
Run
----
.. code-block:: shell
pip-compile-multi
Trusted by
----------
|uber| |mozilla| |twitter|
|nih| |skydio| |pallets|
|moveworks|
Help needed
-----------
The following issues have the highest impact for the project. Contributions are welcome!
1. `Pull requirements from pyproject.toml <https://github.com/peterdemin/pip-compile-multi/issues/283>`_
2. `Productionize GitHub Action to update dependencies on schedule <https://github.com/peterdemin/pip-compile-multi/issues/188>`_
Your mission, should you choose to accept it, is to comment on the issue you want to work on, and open a PR.
I'll review/merge the PR in a timely fashion, and release a new version with your name in the `changelog <https://github.com/peterdemin/pip-compile-multi/blob/master/HISTORY.rst>`_.
Read the Docs
-------------
* `Why use pip-compile-multi <https://pip-compile-multi.readthedocs.io/en/latest/why.html>`_
* `How to start using pip-compile-multi <https://pip-compile-multi.readthedocs.io/en/latest/migration.html>`_
* `List of features <https://pip-compile-multi.readthedocs.io/en/latest/features.html>`_
.. |nih| image:: docs/NIH_logo.svg
:width: 200 px
:height: 200 px
:target: https://www.nih.gov/
.. |uber| image:: docs/Uber_Logo_Black_RGB.svg
:width: 200 px
:height: 200 px
:target: https://www.uber.com/
.. |mozilla| image:: docs/moz-logo-bw-rgb.svg
:width: 200 px
:height: 200 px
:target: https://www.mozilla.org/
.. |skydio| image:: docs/skydio-logo-black.svg
:width: 200 px
:height: 200 px
:target: https://www.skydio.com/
.. |pallets| image:: docs/pallets.png
:width: 200 px
:height: 200 px
:target: https://palletsprojects.com/
.. |twitter| image:: docs/twitter_logo.svg
:width: 200 px
:height: 200 px
:target: https://twitter.com/
.. |moveworks| image:: docs/Moveworks.svg
:width: 400 px
:height: 200 px
:target: https://moveworks.com/
================================================
FILE: docs/Makefile
================================================
# Minimal makefile for Sphinx documentation
#
# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
SOURCEDIR = .
BUILDDIR = _build
# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
.PHONY: help Makefile
# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
================================================
FILE: docs/_static/custom.css
================================================
body {
font-family: 'minion pro', 'bell mt', Georgia, 'Hiragino Mincho Pro', serif;
}
div.document {
width: 71em;
}
================================================
FILE: docs/afterword.rst
================================================
Have fun!
---------
Now that occasional backward incompatible dependency release can't ruin your day,
you can **spread the word** about ``pip-compile-multi``, ask for a new feature in a `GitHub issue`_,
or even open a PR ;-).
.. _GitHub issue: https://github.com/peterdemin/pip-compile-multi/issues
================================================
FILE: docs/boilerplate.rst
================================================
Bonus: boilerplate to put in project's README
---------------------------------------------
Nice way of introducing dependency management process to new team members for copy-pasting to `README.md`:
.. code-block:: text
## Dependency management
This project uses [pip-compile-multi](https://pypi.org/project/pip-compile-multi/) for hard-pinning dependencies versions.
Please see its documentation for usage instructions.
In short, `requirements/base.in` contains the list of direct requirements with occasional version constraints (like `Django~=4.0`)
and `requirements/base.txt` is automatically generated from it by adding recursive tree of dependencies with fixed versions.
The same goes for `test` and `dev`.
To add a new dependency, add it to the appropriate environment's `in` file,
(e.g. `requirements/base.in`) and run `pip-compile-multi --no-upgrade`.
To upgrade dependency versions, run `pip-compile-multi`.
For installation always use `.txt` files. For example, command `pip install -e . -r requirements/dev.txt` will install
this project in development mode, testing requirements and development tools.
Another useful command is `pip-sync requirements/dev.txt`, it uninstalls packages from your virtualenv that aren't listed in the file.
If project is using the second-generation ``requirements`` CLI:
.. code-block:: text
## Dependency management
This project uses [pip-compile-multi](https://pypi.org/project/pip-compile-multi/) for hard-pinning dependencies versions.
Please see its documentation for usage instructions.
In short, `requirements/base.in` contains the list of direct requirements with occasional version constraints (like `Django~=4.0`)
and `requirements/base.txt` is automatically generated from it by adding recursive tree of dependencies with fixed versions.
The same goes for `test` and `dev`.
To add a new dependency, add it to the appropriate environment's `in` file,
(e.g. `requirements/base.in`) and run `requirements lock`.
To upgrade dependency versions, run `requirements upgrade`.
For installation always use `.txt` files. For example, command `pip install -Ue . -r requirements/dev.txt` will install
this project in development mode, testing requirements and development tools.
Another useful command is `pip-sync requirements/dev.txt`, it uninstalls packages from your virtualenv that aren't listed in the file.
================================================
FILE: docs/conf.py
================================================
"""
Configuration file for the Sphinx documentation builder.
This file does only contain a selection of the most common options. For a
full list see the documentation:
http://www.sphinx-doc.org/en/master/config
"""
# pylint: disable=invalid-name,redefined-builtin
# -- Path setup --------------------------------------------------------------
import os
import sys
sys.path.insert(0, os.path.abspath('..'))
# -- Project information -----------------------------------------------------
project = 'pip-compile-multi'
copyright = '2019, Peter Demin'
author = 'Peter Demin'
# The short X.Y version
version = ''
# The full version, including alpha/beta/rc tags
release = '1.5.1'
# -- General configuration ---------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
#
# needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
'sphinx.ext.autodoc',
'sphinx.ext.doctest',
'sphinx.ext.intersphinx',
'sphinx.ext.viewcode',
]
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
#
# source_suffix = ['.rst', '.md']
source_suffix = '.rst'
# The master toctree document.
master_doc = 'index'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = 'en'
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = None
# -- Options for HTML output -------------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
html_theme = 'alabaster'
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
html_theme_options = {
'github_user': 'peterdemin',
'github_repo': 'pip-compile-multi',
'github_type': 'star',
'github_banner': True,
'github_button': True,
'extra_nav_links': {
'Code': 'https://github.com/peterdemin/pip-compile-multi',
'Releases': 'https://pypi.org/project/pip-compile-multi/',
'Issue tracker': 'https://github.com/peterdemin/pip-compile-multi/issues',
},
}
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# Custom sidebar templates, must be a dictionary that maps document names
# to template names.
#
# The default sidebars (for documents that don't match any pattern) are
# defined by theme itself. Builtin themes are using these templates by
# default: ``['localtoc.html', 'relations.html', 'sourcelink.html',
# 'searchbox.html']``.
html_sidebars = {
'**': [
'about.html',
'navigation.html',
'searchbox.html',
'sourcelink.html',
]
}
# -- Options for HTMLHelp output ---------------------------------------------
# Output file base name for HTML help builder.
htmlhelp_basename = 'pip-compile-multidoc'
# -- Options for LaTeX output ------------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#
# 'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#
# 'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#
# 'preamble': '',
# Latex figure (float) alignment
#
# 'figure_align': 'htbp',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
(master_doc, 'pip-compile-multi.tex', 'pip-compile-multi Documentation',
'Peter Demin', 'manual'),
]
# -- Options for manual page output ------------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
(master_doc, 'pip-compile-multi', 'pip-compile-multi Documentation',
[author], 1)
]
# -- Options for Texinfo output ----------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
(master_doc, 'pip-compile-multi', 'pip-compile-multi Documentation',
author, 'pip-compile-multi', 'One line description of project.',
'Miscellaneous'),
]
# -- Options for Epub output -------------------------------------------------
# Bibliographic Dublin Core info.
epub_title = project
# The unique identifier of the text. This can be a ISBN number
# or the project homepage.
#
# epub_identifier = ''
# A unique identification for the text.
#
# epub_uid = ''
# A list of files that should not be packed into the epub file.
epub_exclude_files = ['search.html']
# -- Extension configuration -------------------------------------------------
# -- Options for intersphinx extension ---------------------------------------
# Example configuration for intersphinx: refer to the Python standard library.
intersphinx_mapping = {
'python': ('https://docs.python.org/', None),
}
================================================
FILE: docs/features.rst
================================================
Features
--------
``pip-compile-multi`` supports many options to customize compilation.
Each option can be specified in requirements configuration file, by replacing dashes with underscores.
For example, `--use-cache` becomes `use_cache`.
Supported configuration files: ``pyproject.toml``, ``requirements.ini``, ``setup.cfg``, ``tox.ini``.
In INI files, use section name starting with ``requirements:``, for example: ``[requirements:Python 3]``
In ``pyproject.toml`` prefix section name with ``tool.requirements``, for example: ``[tool.requirements.python3]`` or ``[tool.requirements]``.
.. automodule:: pipcompilemulti.features.base_dir
.. automodule:: pipcompilemulti.features.file_extensions
.. automodule:: pipcompilemulti.features.upgrade
.. automodule:: pipcompilemulti.features.use_cache
.. automodule:: pipcompilemulti.features.compatible
.. automodule:: pipcompilemulti.features.add_hashes
.. automodule:: pipcompilemulti.features.unsafe
.. automodule:: pipcompilemulti.features.header
.. automodule:: pipcompilemulti.features.limit_in_paths
.. automodule:: pipcompilemulti.features.annotate_index
.. automodule:: pipcompilemulti.features.extra_index_url
.. automodule:: pipcompilemulti.features.emit_trusted_host
.. automodule:: pipcompilemulti.features.emit_find_links
.. automodule:: pipcompilemulti.features.autoresolve
.. automodule:: pipcompilemulti.features.backtracking
.. automodule:: pipcompilemulti.features.skip_constraint_comments
.. automodule:: pipcompilemulti.features.strip_extras
.. automodule:: pipcompilemulti.features.live_output
.. automodule:: pipcompilemulti.features.use_uv
.. automodule:: pipcompilemulti.verify
================================================
FILE: docs/history.rst
================================================
.. include:: ../HISTORY.rst
================================================
FILE: docs/index.rst
================================================
.. pip-compile-multi documentation master file, created by
sphinx-quickstart on Thu Aug 8 15:36:37 2019.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
pip-compile-multi
~~~~~~~~~~~~~~~~~
Pip-compile-multi is a command line utility, that compiles multiple
requirements files to lock dependency versions.
Underneath it uses `pip-tools`_ or uv_ for actual compilation.
Pip-compile-multi targets complex projects and provides high
level of automation and flexibility.
.. _pip-tools: https://github.com/jazzband/pip-tools
.. _uv: https://docs.astral.sh/uv/
To install:
.. code-block:: shell
pip install pip-compile-multi
To run:
.. code-block:: shell
pip-compile-multi
Introduced in 3.0.0, the new CLI reads configuration from ``pyproject.toml`` (or one of INI files: ``requirements.ini``, ``setup.cfg``, ``tox.ini``):
.. code-block:: shell
requirements [lock|upgrade|verify]
Why use pip-compile-multi?
==========================
.. toctree::
:maxdepth: 2
why
How to use pip-compile-multi?
=============================
.. toctree::
:maxdepth: 2
installation
migration
features
precommit
boilerplate
Release notes
=============
.. toctree::
:maxdepth: 2
history
.. include:: afterword.rst
================================================
FILE: docs/installation.rst
================================================
Installation
------------
Python Version
==============
We recommend using the latest version of Python 3.
Pip-compile-multi supports Python 3.5 and newer, Python 2.7, and PyPy.
Dependencies
============
These distributions will be installed automatically when installing pip-compile-multi.
* `Click`_ is a framework for writing command line applications.
* `pip-tools`_ is a set of command line tools to help you keep your pip-based
packages fresh, even when you've pinned them.
* `toposort`_ implements topological sort algorithm. Pip-compile-multi uses it
to compose compilation order of requirements files.
.. _Click: https://palletsprojects.com/p/click/
.. _pip-tools: https://github.com/jazzband/pip-tools
.. _toposort: https://pypi.org/project/toposort/
Virtual environments
====================
Use a virtual environment to manage the dependencies for your project, both in
development and in production.
What problem does a virtual environment solve? The more Python projects you
have, the more likely it is that you need to work with different versions of
Python libraries, or even Python itself. Newer versions of libraries for one
project can break compatibility in another project.
Virtual environments are independent groups of Python libraries, one for each
project. Packages installed for one project will not affect other projects or
the operating system's packages.
Python 3 comes bundled with the :mod:`venv` module to create virtual
environments. If you're using a modern version of Python, you can continue on
to the next section.
If you're using Python 2, see :ref:`install-install-virtualenv` first.
.. _install-create-env:
Create an environment
~~~~~~~~~~~~~~~~~~~~~
Create a project folder and a :file:`venv` folder within:
.. code-block:: sh
$ mkdir myproject
$ cd myproject
$ python3 -m venv venv
On Windows:
.. code-block:: bat
$ py -3 -m venv venv
If you needed to install virtualenv because you are using Python 2, use
the following command instead:
.. code-block:: sh
$ python2 -m virtualenv venv
On Windows:
.. code-block:: bat
> \Python27\Scripts\virtualenv.exe venv
.. _install-activate-env:
Activate the environment
~~~~~~~~~~~~~~~~~~~~~~~~
Before you work on your project, activate the corresponding environment:
.. code-block:: sh
$ . venv/bin/activate
On Windows:
.. code-block:: bat
> venv\Scripts\activate
Your shell prompt will change to show the name of the activated environment.
Install pip-compile-multi
=========================
Within the activated environment, use the following command to install pip-compile-multi:
.. code-block:: shell
pip install pip-compile-multi
pip-compile-multi is now installed. Check out the :doc:`/features` or go to the
:doc:`Documentation Overview </index>`.
.. _install-install-virtualenv:
Install virtualenv
==================
If you are using Python 2, the venv module is not available. Instead,
install `virtualenv`_.
On Linux, virtualenv is provided by your package manager:
.. code-block:: sh
# Debian, Ubuntu
$ sudo apt-get install python-virtualenv
# CentOS, Fedora
$ sudo yum install python-virtualenv
# Arch
$ sudo pacman -S python-virtualenv
If you are on Mac OS X or Windows, download `get-pip.py`_, then:
.. code-block:: sh
$ sudo python2 Downloads/get-pip.py
$ sudo python2 -m pip install virtualenv
On Windows, as an administrator:
.. code-block:: bat
> \Python27\python.exe Downloads\get-pip.py
> \Python27\python.exe -m pip install virtualenv
Now you can return above and :ref:`install-create-env`.
.. _virtualenv: https://virtualenv.pypa.io/
.. _get-pip.py: https://bootstrap.pypa.io/get-pip.py
================================================
FILE: docs/migration.rst
================================================
How to start using pip-compile-multi on existing project
--------------------------------------------------------
Initial situation
=================
There are various ways to declare dependencies in Python project.
The most straightforward is to just put them right into ``setup.py``,
like `Flask`_ does.
Another common option is to have one or more ``requirements.txt`` files in a project,
like `Deluge`_ have.
.. _Flask: https://github.com/pallets/flask/blob/master/setup.py#L52-L75
.. _Deluge: https://github.com/deluge-torrent/deluge/blob/develop/requirements.txt
Migration steps
===============
1. Create ``requirements`` directory.
2. Copy-paste the list of project runtime dependencies
to ``requirements/base.in``.
3. Create ``requirements/test.in`` with test time dependencies.
Make sure it has a reference to runtime dependencies - ``-r base.in``.
4. Run ``pip-compile-multi``. It will produce two more files:
* ``requirements/base.txt``
* ``requirements/test.txt``
5. :ref:`Unpin <unpin>` packages in ``.in`` files.
6. Run ``pip-compile-multi`` again to upgrade the compiled files.
.. _unpin:
How to unpin packages
=====================
No constraints
~~~~~~~~~~~~~~
Some projects don't constraint it's dependencies. In this case, there's nothing to unpin, no more work needed.
Hard-pinned versions (==)
~~~~~~~~~~~~~~~~~~~~~~~~~
Some projects hard-pin all dependencies, just to be safe.
Most likely, the code will run fine with the next patch release, but it's hard to tell for sure.
For such cases, comprehensive test suite is vital.
Your milage may vary, but generally the fastest workflow is as follows:
1. Remove all the constraints, by deleting everything after package names (e.g. ``==1.2.3``).
2. Recompile ``.txt`` requirements.
3. Install new versions.
4. Run tests.
5. If tests passed, job's done.
6. If some of the tests fails, it's likely that some of the original constraints
was indeed required. Try to find what's the incompatible package version, maybe read package's CHANGELOG.
Maybe the simpliest way is to return whatever constraint was originally set to move the needle.
7. After updating one of the ``.in`` files, go to step 2.
Two-way constraints (>1,<2)
~~~~~~~~~~~~~~~~~~~~~~~~~~~
The popular requirements policy is to add packages constrained to minor releases, like this::
uwsgi>2.0.0,<2.1.0
The idea is that dependency is following SemVer and patch release won't break any functionality.
Of course, the reality is that some times patch release introduces a bug,
and some times next major release is backwards compatible.
But looking at this line you can't tell if ``uwsgi==2.1.0`` is going to break your app.
The only way to know it is to actually try.
So for such projects the general approach is to remove all ``<`` constraints and see
which of them were really needed.
================================================
FILE: docs/precommit.rst
================================================
Verify as pre-commit hook
=========================
To verify that ``pip-compile-multi`` has been run after changing ``.in`` files as a `PreCommit`_ hook, just add the following to your local repo's ``.pre-commit-config.yaml`` file:
.. code-block:: yaml
- repo: https://github.com/peterdemin/pip-compile-multi
rev: v2.6.2
hooks:
- id: pip-compile-multi-verify
.. _PreCommit: https://pre-commit.com/
================================================
FILE: docs/why.rst
================================================
Motivation
----------
I will start from the very basics of dependency management and will go very slow,
so if you feel bored, just scroll to the next section.
Suppose you have a python project with following direct dependencies:
.. code-block:: text
click
pip-tools
(Yes I took pip-compile-multi as an example).
Let's save them as-is in ``requirements/base.in``.
Those are unpinned libraries. It means that whenever developer runs
.. code-block:: shell
pip install -r requirements/base.in
they will get *some* version of these libraries.
And the chances are that if several developers do the same over some period,
some will have different dependency versions than others.
Also, if the project is online service, one day it may stop working after
redeployment because some of the dependencies had backward incompatible release.
These backward incompatible changes are relatively common.
To avoid this problem, Python developers are hard-pinning (aka locking) their dependencies.
So instead of a list of libraries, they have something like:
.. code-block:: text
click==6.7
pip-tools==1.11.0
(To keep things neat let's put this into ``requirements/base.txt``)
That's good for a starter. But there are two significant drawbacks:
1. Developers have to do non-trivial operations if they want to keep up with
newer versions (that have bug fixes and performance improvements).
2. Indirect dependencies (that is dependencies of dependencies) may still have
backward-incompatible releases, that break everything.
Let's put aside point 1 and fight point 2. Let's do
.. code-block:: shell
pip freeze > requirements/base.txt
Now we have full hierarchy of dependencies hard-pinned:
.. code-block:: text
click==6.7
first==2.0.1
pip-tools==1.11.0
six==1.11.0
That's great, and solves the main problem - service will be deployed exactly [1]_
the same every single time and all developers will have same environments.
.. [1] That's not true.
Someone could re-upload broken package under existing version on PyPI.
For 100% reproducible builds use hashes.
This case is so common that there already are some tools to solve it.
Two worth mentioning are:
1. `Pip Tools`_ - a mature package that is enhanced by ``pip-compile-multi``.
2. `PipEnv`_ - a fresh approach that is going to become the "official" Python way of locking dependencies some day.
.. _Pip Tools: https://github.com/jazzband/pip-tools
.. _PipEnv: https://github.com/pypa/pipenv
But what if the project uses some packages that are not required by the service itself?
For example ``pytest``, that is needed to run unit tests, but should never
be deployed to a production site. Or ``flake8`` - syntax checking tool.
If they are installed in the current virtual environment, they will get into
``pip freeze`` output.
That's no good.
And removing them manually from ``requirements/base.txt`` is not an option.
But still, these packages must be pinned to ensure, that tests are running
the same way on all development machines (and build server).
So let's get hands dirty and put all the testing stuff into ``requirements/test.in``:
.. code-block:: text
-r base.in
prospector
pylint
flake8
mock
six
Note, how I put ``-r base.in`` in the beginning, so that *test* dependencies are installed
along with the *base*.
Now installation command is
.. code-block:: shell
pip install -r requirements/test.in
For one single time (exceptionally to show how unacceptable is this task)
let's manually compose ``requirements/test.txt``.
After installation, run freeze to bring the whole list of all locked packages:
.. code-block:: shell
$ pip freeze
astroid==1.6.0
click==6.7
dodgy==0.1.9
first==2.0.1
flake8==3.5.0
flake8-polyfill==1.0.2
isort==4.2.15
lazy-object-proxy==1.3.1
mccabe==0.6.1
mock==2.0.0
pbr==3.1.1
pep8-naming==0.5.0
pip-tools==1.11.0
prospector==0.12.7
pycodestyle==2.0.0
pydocstyle==2.1.1
pyflakes==1.6.0
pylint==1.8.1
pylint-celery==0.3
pylint-common==0.2.5
pylint-django==0.7.2
pylint-flask==0.5
pylint-plugin-utils==0.2.6
PyYAML==3.12
requirements-detector==0.5.2
setoptconf==0.2.0
six==1.11.0
snowballstemmer==1.2.1
wrapt==1.10.11
Wow! That's quite a list! But we remember what goes into base.txt:
1. click
2. first
3. pip-tools
4. six
Good, everything else can be put into ``requirements/test.txt``.
But wait, ``six`` is included in ``test.in`` and is missing in ``test.txt``.
That feels wrong. Ah, it's because we've moved ``six`` to the ``base.txt``.
It's good that we didn't forget, that it should be in *base*.
We might forget next time though.
Why don't we automate it? That's what ``pip-compile-multi`` is for.
Managing dependency versions in multiple environments
-----------------------------------------------------
Let's rehearse. Example service has two groups of dependencies
(or, as I call them, environments):
.. code-block:: shell
$ cat requirements/base.in
click
pip-tools
$ cat requirements/test.in
-r base.in
prospector
pylint
flake8
mock
six
To make automation even more appealing, let's add one more environment.
I'll call it *local* - things that are needed during development, but are not
required by tests, or service itself.
.. code-block:: shell
$ cat requirements/local.in
-r test.in
tox
Now we want to put all *base* dependencies along with all their recursive dependencies
in ``base.txt``,
all recursive *test* dependencies except for *base* into ``test.txt``,
and all recursive *local* dependencies except for *base* and *test* into ``local.txt``.
.. code-block:: shell
$ pip-compile-multi
Locking requirements/base.in to requirements/base.txt. References: []
Locking requirements/test.in to requirements/test.txt. References: ['base']
Locking requirements/local.in to requirements/local.txt. References: ['base', 'test']
Yes, that's right. All the tedious dependency versions management job done with
a single command that doesn't even have options.
Now you can run ``git diff`` to review the changes and ``git commit`` to save them.
To install the new set of versions run:
.. code-block:: shell
pip install -Ur requirements/local.txt
It's a perfect time to run all the tests and make sure, that updates were
backward compatible enough for your needs.
More often than I'd like in big projects, it's not so.
Let's say the new version of ``pylint`` dropped support of old Python version,
that you still need to support.
Than you open ``test.in`` and soft-pin it with descriptive comment:
.. code-block:: shell
$ cat requirements/test.in
-r base.in
prospector
pylint<1.8 # Newer versions dropped support for Python 2.4
flake8
mock
six
I know, this example is made up. But you get the idea.
That re-run ``pip-compile-multi`` to compile new ``test.txt`` and check new set.
Benefits
--------
I want to summarise, why ``pip-compile-multi`` might be a good addition to your project.
Some of the benefits are achievable with other methods, but I want to be general:
1. Production will not suddenly break after redeployment because of
backward incompatible dependency release.
2. Every development machine will have the same package versions.
3. Service still uses most recent versions of packages.
And fresh means best here.
4. Dependencies are upgraded when the time is suitable for the service,
not whenever they are released.
5. Different environments are separated into different files.
6. ``*.in`` files are small and manageable because they store only direct dependencies.
7. ``*.txt`` files are exhaustive and precise (but you don't need to edit them).
================================================
FILE: how-to.md
================================================
# Managing dependencies in multi-platform Python project
I'm going to describe the setup I made for my Python
[project](https://github.com/peterdemin/pip-compile-multi).
The project supports Python version 2.7 and 3.4+, PyPy, Linux, and Windows.
It runs tests on every commit in Travis CI and AppVeyor.
The project relies on a few runtime packages (5),
a bunch for testing (12) and a lot for development (36).
Some dependencies are shared, some specific to Python 2.7 and some to Windows.
The project uses hard-pinned packages with hashes to verify the integrity of installed versions.
## Problems
Let's summarize what we are solving:
1. The project has 3 set of dependencies: base, test and local which are needed respectfully during running, testing and developing.
2. Some packages are needed only under Python 2.7.
3. Other packages are needed only under Windows.
4. All packages must be hard-pinned and hashed.
Alternatively, if turned into tasks:
1. Organize dependencies for each environment.
2. Orchestrate installation in different environments.
## Solutions
The first tool we are using is [pip-compile-multi](https://github.com/peterdemin/pip-compile-multi),
which is ~~ironically~~ the project we are using as an example.
It has verbose documentation, so I'll briefly outline how it is applied here.
There are 6 `.in` files in the `requirements` directory: base.in, test.in, local.in, py27.in, local27.in, testwin.in.
`base`, `test` and `local` are meant to be used under Python 3.
`py27` and `local27` are holding Python 2.7 backports of Python 3 packages and version constraints for projects, which dropped Python 2 support in newer versions.
`testwin` has a single entry: `colorama`, which is `pytest` dependency that is installed only under windows.
First, we are pinning packages to the current versions with the following command:
```
$ pip-compile-multi -n local -n testwin
```
It produces files `base.txt`, `test.txt`, `local.txt` and `testwin.txt` with recursively retrieved hard-pinned package versions.
The second command takes these `.txt` files and produce `.hash` files with attached package hashes:
```
$ pip-compile-multi -n local -n testwin -g local -g testwin -i txt -o hash
```
The same operation must be repeated separately for Python 2.7 packages:
```
$ pip-compile-multi -n py27 -n local27
$ pip-compile-multi -n py27 -n local27 -g py27 -g local27 -i txt -o hash
```
Separation is required because [pip-tools](https://github.com/jazzband/pip-tools)
can't traverse packages that are not required in the current runtime.
To automate this tasks, I'm using the second tool - [tox](https://tox.readthedocs.io/en/latest/).
Here is my configuration:
```
[testenv:upgrade2]
basepython = python2.7
deps = pip-compile-multi
commands =
pip-compile-multi -n py27 -n local27
pip-compile-multi -n py27 -n local27 -g py27 -g local27 -i txt -o hash
[testenv:upgrade3]
basepython = python3.6
deps = pip-compile-multi
commands =
pip-compile-multi -n local -n testwin
pip-compile-multi -n local -n testwin -g local -g testwin -i txt -o hash
```
To run it, I execute:
```
$ tox -e upgrade3 -e upgrade2
```
To run unit and doc tests locally I have somewhat complex testenv setup:
```
[tox]
envlist = py{27,34,35,36,37}-{linux,windows}
[testenv]
platform = linux: linux
windows: win32
deps =
linux: -r{toxinidir}/requirements/test.hash
windows: -r{toxinidir}/requirements/testwin.hash
py27: -r{toxinidir}/requirements/py27.hash
commands = python -m pytest --doctest-modules pipcompilemulti.py test_pipcompilemulti.py
```
The setup says to use `test.hash` file under Linux,
`testwin.hash` under Windows and
add `py27.hash` if it is also running under Python 2.7.
AppVeyor runs these tests under Windows; its configuration file defines which Python version to use and what parameters to pass to `tox.ini`:
```
environment:
matrix:
- PYTHON: "C:\\Python27"
PYTHON_VERSION: "2.7.8"
PYTHON_ARCH: "32"
TOX_ENV: "py27"
- PYTHON: "C:\\Python34"
PYTHON_VERSION: "3.4.1"
PYTHON_ARCH: "32"
TOX_ENV: "py34"
- PYTHON: "C:\\Python35"
PYTHON_VERSION: "3.5.4"
PYTHON_ARCH: "32"
TOX_ENV: "py35"
- PYTHON: "C:\\Python36"
PYTHON_VERSION: "3.6.4"
PYTHON_ARCH: "32"
TOX_ENV: "py36"
init:
- "ECHO %PYTHON% %PYTHON_VERSION% %PYTHON_ARCH%"
install:
- "appveyor/setup_build_env.cmd"
- "powershell appveyor/install.ps1"
build: false
test_script:
- "%PYTHON%/Scripts/tox -e %TOX_ENV%-windows"
```
`environment.matrix` defines `TOX_ENV` variable, which is passed to `tox` in `test_script` step.
Travis CI has the following configuration to run tests under Linux:
```
# Config file for automatic testing at travis-ci.org
language: python
python:
- "3.7-dev"
- "3.6"
- "3.5"
- "3.4"
- "2.7"
- "pypy"
# command to install dependencies
install:
- ./install_test_deps.sh
- pip install -e .
# command to run tests using coverage, e.g., python setup.py test
script: python -m pytest --doctest-modules pipcompilemulti.py test_pipcompilemulti.py
```
I stashed dependency installation step into bash script `install_test_deps.sh`:
```
#!/bin/sh
python --version 2>&1 | grep -q 'Python 3'
if [ $? -eq 0 ]
then
# Python 3
exec pip install -r requirements/test.hash
else
# Python 2 or PyPy
exec pip install -r requirements/test.hash -r requirements/py27.hash
fi
```
I could have reused tox here too with [tox-travis](https://github.com/tox-dev/tox-travis).
Someday, maybe, I will.
## Conclusion
Python is multi-platform language, but it lacks built-in tools for secure management of dependencies versions.
Tools like `pip-compile-multi` and `tox` accompanied by CI services like `Travis-ci` and `AppVeyor` significantly reduce the effort. However, correct configuration takes time and requires skills and persistence.
Described solution can be used as a boilerplate for project setup, or as guidance for building another tool that puts a framework for complex dependency management.
================================================
FILE: nested/base.in
================================================
-r subproject/sub.in
================================================
FILE: nested/base.txt
================================================
# SHA1:6a7fb5b91341e12583307fb55069246d43096b44
#
# This file is autogenerated by pip-compile-multi
# To update, run:
#
# pip-compile-multi
#
-r subproject/sub.txt
================================================
FILE: nested/diamond.in
================================================
-r base.in
-r subproject/base.in
================================================
FILE: nested/diamond.txt
================================================
# SHA1:1301840447a3dce149692692b41974b7668ba22e
#
# This file is autogenerated by pip-compile-multi
# To update, run:
#
# pip-compile-multi
#
-r base.txt
-r subproject/base.txt
================================================
FILE: nested/subproject/base.in
================================================
-r ../up.in
================================================
FILE: nested/subproject/base.txt
================================================
# SHA1:e975ad2caba20cf1afaea2cd3626c6c566edfacf
#
# This file is autogenerated by pip-compile-multi
# To update, run:
#
# pip-compile-multi
#
-r ../up.txt
================================================
FILE: nested/subproject/sub.in
================================================
-r base.in
================================================
FILE: nested/subproject/sub.txt
================================================
# SHA1:a87fd594461015e819a1f468943967d12880b85d
#
# This file is autogenerated by pip-compile-multi
# To update, run:
#
# pip-compile-multi
#
-r base.txt
================================================
FILE: nested/up.in
================================================
six
================================================
FILE: nested/up.txt
================================================
# SHA1:bec9703f7a456cd2b4ab5fb3220ae016e3e394e3
#
# This file is autogenerated by pip-compile-multi
# To update, run:
#
# pip-compile-multi
#
six==1.15.0
# via -r nested/up.in
================================================
FILE: pipcompilemulti/__init__.py
================================================
"""Pip compile multi aka requirements"""
__author__ = 'Peter Demin'
__email__ = 'peterdemin@gmail.com'
__version__ = '3.3.1'
================================================
FILE: pipcompilemulti/actions.py
================================================
#!/usr/bin/env python
"""High level actions to be called from CLI"""
import logging
from .discover import discover
from .environment import Environment
from .verify import generate_robust_hash_comment
from .features import FEATURES
from .deduplicate import PackageDeduplicator
logger = logging.getLogger("pip-compile-multi")
def recompile():
"""Compile requirements files for all environments."""
env_confs = FEATURES.on_discover(
discover(FEATURES.compose_input_file_path('*'))
)
deduplicator = PackageDeduplicator()
deduplicator.on_discover(env_confs)
sink_in_path = FEATURES.sink_in_path()
if sink_in_path:
sink_env = Environment(in_path=sink_in_path)
logger.info(
"Creating a temporary file with all dependencies at %s",
sink_env.outfile,
)
sink_env.create_lockfile()
compile_topologically(env_confs, deduplicator)
def compile_topologically(env_confs, deduplicator):
"""Compile environments in topological order of reference."""
for conf in env_confs:
env = Environment(in_path=conf['in_path'], deduplicator=deduplicator)
if env.maybe_create_lockfile():
# Only munge lockfile if it was written.
header_text = generate_robust_hash_comment(env.infile) + FEATURES.get_header_text()
env.replace_header(header_text)
env.add_references(conf['refs'])
================================================
FILE: pipcompilemulti/cli_v1.py
================================================
"""The current stable version of command line interface."""
import os
import sys
import logging
from traceback import print_exception
import click
from .actions import recompile
from .verify import verify_environments
from .features import FEATURES
THIS_FILE = os.path.abspath(__file__)
@click.group(invoke_without_command=True)
@click.pass_context
@FEATURES.bind
def cli(ctx):
"""Recompile"""
logging.basicConfig(level=logging.DEBUG, format="%(message)s")
sys.excepthook = exception_hook
if ctx.invoked_subcommand is None:
recompile()
@cli.command()
@click.pass_context
@FEATURES.bind
def verify(ctx):
"""
For each environment verify hash comments and report failures.
If any failure occured, exit with code 1.
"""
sys.excepthook = exception_hook
ctx.exit(0
if verify_environments()
else 1)
def exception_hook(exctype, value, traceback):
"""Strip exception printout above this module."""
print_exception(exctype, value, trim_traceback(traceback))
def trim_traceback(traceback):
"""Trim traceback top so it starts with this module.
Return original traceback if this module is not found.
"""
level = 0
new_traceback = traceback
while new_traceback is not None:
file_path = new_traceback.tb_frame.f_code.co_filename
if THIS_FILE.startswith(file_path):
return new_traceback
level += 1
new_traceback = new_traceback.tb_next
return traceback
if __name__ == '__main__':
cli() # pylint: disable=no-value-for-parameter
================================================
FILE: pipcompilemulti/cli_v2.py
================================================
"""Human-friendly interface to pip-compile-multi"""
import functools
import logging
import click
from .actions import recompile
from .config import read_config, read_sections
from .options import OPTIONS
from .verify import verify_environments
logger = logging.getLogger("pip-compile-multi")
DEFAULT_OPTIONS = {
'directory': 'requirements',
'in_ext': 'in',
'out_ext': 'txt',
'autoresolve': True,
}
@click.group()
def cli():
"""Human-friendly interface to pip-compile-multi"""
logging.basicConfig(level=logging.INFO, format="%(message)s")
@cli.command()
def lock():
"""Lock new dependencies without upgrading."""
run_configurations(recompile, read_config, upgrade=False)
@cli.command()
@click.argument('packages', nargs=-1)
def upgrade(packages):
"""Upgrade locked dependency versions."""
run_configurations(recompile, read_config, upgrade=True, upgrade_packages=packages)
@cli.command()
@click.pass_context
def verify(ctx):
"""Verify environments."""
oks = run_configurations(
skipper(verify_environments),
read_sections,
)
ctx.exit(0
if False not in oks
else 1)
def skipper(func):
"""Decorator that memorizes base_dir, in_ext and out_ext from OPTIONS
and skips execution for duplicates.
"""
@functools.wraps(func)
def wrapped():
"""Dummy docstring to make pylint happy."""
key = (OPTIONS['directory'], OPTIONS['in_ext'], OPTIONS['out_ext'])
if key not in seen:
seen[key] = func()
return seen[key]
seen = {}
return wrapped
def run_configurations(callback, sections_reader, **overrides):
"""Parse configurations and execute callback for matching.
Args:
callback: Function to execute for each matching section
sections_reader: Function that returns configuration sections
"""
sections = sections_reader()
if sections is None:
logger.info("Configuration not found in pyproject.toml "
"or one of .ini files. Running with default settings")
recompile()
return []
elif sections == []:
logger.info("Configuration does not match current runtime. "
"Exiting")
results = []
for section, options in sections:
OPTIONS.clear()
OPTIONS.update(DEFAULT_OPTIONS)
OPTIONS.update(options)
OPTIONS.update(overrides)
logger.debug("Running configuration from section \"%s\". OPTIONS: %r",
section, OPTIONS)
results.append(callback())
return results
if __name__ == '__main__':
cli() # pylint: disable=no-value-for-parameter
================================================
FILE: pipcompilemulti/config.py
================================================
"""Get tasks options from INI file"""
import sys
import os
import configparser
from typing import Dict, List, Union
from functools import lru_cache
from .features.base import BaseFeature, ClickOption
def read_config():
"""Read requirements.ini and return list of pairs (name, options)
If no requirements sections found, return None.
If some sections found, but none matches current runtime, return empty list.
"""
return filter_sections(read_sections())
def filter_sections(sections):
"""Filter through pairs (name, options)
leaving only those that match runtime.
If no requirements sections found, return None.
If some sections found, but none matches current runtime, return empty list.
"""
if not sections:
return None
jobs = []
matchers = python_version_matchers()
for name, options in sections:
target_version = options.pop('python', None)
if target_version in matchers:
jobs.append((name, options))
return jobs
def read_sections():
"""Read ini/toml files and return list of pairs (name, options)"""
sections = []
sections.extend(_read_toml_sections())
sections.extend(_read_cfg_sections())
return sections
def _read_cfg_sections():
parser = configparser.ConfigParser()
parser.read(('requirements.ini', 'setup.cfg', 'tox.ini'))
return [
(
name,
{
key: parse_value(key, parser[name][key])
for key in parser[name]
}
)
for name in parser.sections()
if 'requirements' in name
]
def _read_toml_sections():
# pylint: disable=import-outside-toplevel
try:
import tomllib
except ImportError:
try:
import tomli as tomllib
except ImportError:
return []
if not os.path.isfile("pyproject.toml"):
return []
with open("pyproject.toml", mode="rb") as fp:
toml_config = tomllib.load(fp)
config = toml_config.get("tool", {}).get("requirements")
if not config:
return []
if not all(isinstance(v, dict) for v in config.values()):
# Allow [tool.requirements] section
config = {'config': config}
return [
(
name,
{
key: parse_value(key, _make_toml_scalar(value))
for key, value in section.items()
}
)
for name, section in config.items()
]
def _make_toml_scalar(v: object) -> str:
"""Coalesce TOML value to string.
TOML supports richer data types than ini files (strings, arrays,
floats, ints, etc), however we need to convert all scalar values
to str for compatibility with the rest of the configuration system,
which expects strings only.
"""
return ",".join(v) if isinstance(v, list) else str(v)
@lru_cache(maxsize=None)
def _collect_feature_options() -> Dict[str, ClickOption]:
subclasses = BaseFeature.__subclasses__()
for feature_class in BaseFeature.__subclasses__():
subclasses.extend(feature_class.__subclasses__())
return {
feature_class.OPTION_NAME: feature_class.CLICK_OPTION
for feature_class in subclasses
if feature_class.OPTION_NAME and feature_class.CLICK_OPTION
}
def parse_value(key: str, value: str) -> Union[str, List[str], bool]:
"""Parse value according to the option definition (bool or list)"""
options = _collect_feature_options()
click_option = options.get(key)
if click_option:
if click_option.multiple:
return [item.strip() for item in value.split(',')]
if click_option.is_flag:
return value.lower() not in ('false', 'off', 'no')
return value
def python_version_matchers():
"""Return set of string representations of current python version"""
version = sys.version_info
patterns = [
"{0}",
"{0}{1}",
"{0}.{1}",
]
matchers = [
pattern.format(*version)
for pattern in patterns
] + [None]
return set(matchers)
================================================
FILE: pipcompilemulti/deduplicate.py
================================================
"""Remove packages included in referenced environments."""
import logging
from pipcompilemulti.utils import recursive_refs, merged_packages
logger = logging.getLogger("pip-compile-multi")
class PackageDeduplicator:
"""Remove packages included in referenced environments."""
def __init__(self):
self.env_packages = {}
self.env_confs = None
def on_discover(self, env_confs):
"""Save environment references."""
self.env_confs = env_confs
def register_packages_for_env(self, in_path, packages):
"""Save environment packages."""
self.env_packages[in_path] = packages
def ignored_packages(self, in_path):
"""Get package mapping from name to version for referenced environments."""
if self.env_confs is None:
return {}
rrefs = recursive_refs(self.env_confs, in_path)
return IgnoredPackages(merged_packages(self.env_packages, rrefs))
def recursive_refs(self, in_path):
"""Return recursive list of environment names referenced by in_path."""
if self.env_confs is None:
return {}
return recursive_refs(self.env_confs, in_path)
class IgnoredPackages:
"""Mapping from package name to version.
Handles name normalization for packages like:
zope.interface, zope-interface, zope_interface.
"""
_DELIMITERS = ('_', '-', '.')
def __init__(self, package_versions):
self._package_versions = package_versions
self._stems = {
self._make_stem(name): name
for name in self._package_versions
}
def __getitem__(self, key):
canonical_key = self._stems[self._make_stem(key)]
return self._package_versions[canonical_key]
def __contains__(self, key):
return self._make_stem(key) in self._stems
@classmethod
def _make_stem(cls, name):
for delim in cls._DELIMITERS:
name = name.replace(delim, '-')
return name.lower()
================================================
FILE: pipcompilemulti/dependency.py
================================================
"""Dependency class"""
import re
from .features import FEATURES
class Dependency(object): # pylint: disable=too-many-instance-attributes
r"""Single dependency.
Comment may span multiple lines.
>>> print(Dependency(
... "six==1.0\n "
... " --hash=abcdef\n"
... " # via\n"
... " # app\n"
... " # pkg"
... ).serialize())
six==1.0 \
--hash=abcdef
# via
# app
# pkg
>>> print(Dependency(
... "six==1.0\n"
... " # via\n"
... " # app\n"
... " # pkg"
... ).serialize())
six==1.0
# via
# app
# pkg
>>> # Old-style one-line
>>> print(Dependency("six==1.0 # via pkg").serialize())
six==1.0 # via pkg
>>> print(Dependency("-e https://site#egg=pkg==1\n # via lib").serialize())
https://site#egg=pkg==1
# via lib
>>> print(Dependency('dep==1 ; sys_platform == "darwin"').serialize())
dep==1 ; sys_platform == "darwin"
"""
COMMENT_JUSTIFICATION = 26
# Example:
# unidecode==0.4.21 # via myapp
# [package] [version] [comment]
RE_DEPENDENCY = re.compile(
r'(?iu)(?P<package>\S+)'
r'=='
r'(?P<version>\S+)'
r'(?P<markers>\s+;\s+.+?)?'
r'(?P<hashes>(?:\s*--hash=\S+)+)?'
r'(?P<comment>(?:\s*#.*)+)?$'
)
RE_EDITABLE_FLAG = re.compile(
r'^-e '
)
# -e git+https://github.com/ansible/docutils.git@master#egg=docutils
# -e "git+https://github.com/zulip/python-zulip-api.git@
# 0.4.1#egg=zulip==0.4.1_git&subdirectory=zulip"
RE_VCS_DEPENDENCY = re.compile(
r'(?iu)(?P<editable>-e)?'
r'\s*'
r'(?P<prefix>\S+#egg=)'
r'(?P<package>[a-z0-9-_.]+)'
r'(?P<postfix>\S*)'
r'(?P<markers>\s+;\s+.+?)?'
r'(?P<comment>(?:\s*#.*)+)?$'
)
# New format, replacing old VCS dependency:
# docutils @ git+https://github.com/ansible/docutils.git@master
RE_AT_DEPENDENCY = re.compile(
r'(?iu)(?P<editable>-e)?'
r'\s*'
r'(?P<package>[a-z0-9-_.]+)'
r' @ '
r'(?P<url>\S+)'
r'(?P<markers>\s+;\s+.+?)?'
r'(?P<comment>(?:\s*#.*)+)?$'
)
# Example: 2022.02.1 -> 2022.2.1
RE_IGNORED_ZEROS = re.compile(r"(?<=\.)0+(?=\d)")
def __init__(self, line):
self.is_vcs = False
self.is_at = False
self.valid = True
self.line = line
vcs = self.RE_VCS_DEPENDENCY.match(line)
if vcs:
self.is_vcs = True
self.package = vcs.group('package')
self.version = ''
self.markers = vcs.group('markers') or ''
self.hashes = '' # No way!
self.comment = (vcs.group('comment') or '').rstrip()
self.comment_span = self._adjust_span(vcs.span('comment'), vcs)
return
at_url = self.RE_AT_DEPENDENCY.match(line)
if at_url:
self.is_at = True
self.package = at_url.group('package')
self.version = ''
self.markers = at_url.group('markers') or ''
self.hashes = '' # ???
self.comment = (at_url.group('comment') or '').rstrip()
self.comment_span = self._adjust_span(at_url.span('comment'), at_url)
return
regular = self.RE_DEPENDENCY.match(line)
if regular:
self.package = regular.group('package')
self.version = self.RE_IGNORED_ZEROS.sub("", regular.group('version').strip())
self.markers = regular.group('markers') or ''
self.hashes = (regular.group('hashes') or '').strip()
self.comment = (regular.group('comment') or '').rstrip()
self.comment_span = self._adjust_span(regular.span('comment'), regular)
return
self.valid = False
def serialize(self):
"""
Render dependency back in string using:
~= if package is internal
== otherwise
"""
if self.is_vcs or self.is_at:
return "{}{}".format(
self.without_editable(self.line[:self.comment_span[0]]).strip(),
FEATURES.process_dependency_comments(self.comment),
)
equal = FEATURES.constraint(self.package)
package_version = '{package}{equal}{version} '.format(
package=self.without_editable(self.package),
version=self.version,
equal=equal,
)
if self.markers:
package_version = '{}{} '.format(package_version.strip(), self.markers)
if self.hashes:
hashes = self.hashes.split()
lines = [package_version.strip()]
lines.extend(hashes)
result = ' \\\n '.join(lines)
result += FEATURES.process_dependency_comments(self.comment)
return result
else:
if self.comment.startswith('\n'):
return (
package_version.rstrip() +
FEATURES.process_dependency_comments(self.comment).rstrip()
)
return '{0}{1}'.format(
package_version.ljust(self.COMMENT_JUSTIFICATION),
self.comment.lstrip(),
).rstrip() # rstrip for empty comment
@classmethod
def without_editable(cls, line):
"""
Remove the editable flag.
It's there because pip-compile can't yet do without it
(see https://github.com/jazzband/pip-tools/issues/272 upstream),
but in the output of pip-compile it's no longer needed.
"""
if 'git+git@' in line:
# git+git can't be installed without -e:
return line
return cls.RE_EDITABLE_FLAG.sub('', line)
def drop_post(self, in_path):
"""Remove .postXXXX postfix from version if needed."""
self.version = FEATURES.drop_post(in_path, self.package, self.version)
@staticmethod
def _adjust_span(span, matchobj):
if span == (-1, -1):
length = matchobj.span()[1]
return (length, length)
return span
================================================
FILE: pipcompilemulti/discover.py
================================================
"""Environment discovery"""
import os
import glob
from collections import deque
from toposort import toposort_flatten
from .environment import Environment
from .utils import fix_reference_path, extract_env_name
__all__ = ('discover',)
def discover(glob_pattern):
"""
Find all files matching given glob_pattern,
parse them, and return list of environments:
Recursively follow referenced files not matched by glob_pattern.
>>> import os
>>> envs = discover(os.path.join('requirements', '*.in'))
>>> # import pprint; pprint.pprint(envs)
>>> envs == [
... {'in_path': os.path.join('requirements', 'base.in'), 'name': 'base',
... 'refs': set()},
... {'in_path': os.path.join('requirements', 'test.in'), 'name': 'test',
... 'refs': {'base.in'}},
... {'in_path': os.path.join('requirements', 'local.in'), 'name': 'local',
... 'refs': {'test.in'}},
... {'in_path': os.path.join('requirements', 'testwin.in'), 'name': 'testwin',
... 'refs': {'test.in'}}
... ]
True
"""
to_visit = deque(map(os.path.normpath, glob.glob(glob_pattern)))
envs, all_in_paths = {}, set()
while to_visit:
in_path = to_visit.pop()
# name =
if in_path in all_in_paths:
continue
all_in_paths.add(in_path)
envs[in_path] = {
'in_path': in_path,
'name': extract_env_name(in_path),
'refs': Environment.parse_references(in_path),
}
for ref in envs[in_path]['refs']:
to_visit.append(fix_reference_path(
orig_path=in_path,
ref_path=ref
))
return order_by_refs(envs.values())
def order_by_refs(envs):
"""Return topologicaly sorted list of environments.
I.e. all referenced environments are placed before their references.
"""
topology = {
env['in_path']: set(fix_reference_path(env['in_path'], ref)
for ref in env['refs'])
for env in envs
}
by_in_path = {
env['in_path']: env
for env in envs
}
return [
by_in_path[in_path]
for in_path in toposort_flatten(topology)
]
================================================
FILE: pipcompilemulti/environment.py
================================================
"""Environment class"""
import os
import re
import sys
import logging
import subprocess
from .dependency import Dependency
from .features import FEATURES
from .deduplicate import PackageDeduplicator
from .utils import extract_env_name
logger = logging.getLogger("pip-compile-multi")
class Environment(object):
"""requirements file"""
RE_REF = re.compile(r'^(?:-r|--requirement)\s*(?P<path>\S+).*$')
RE_COMMENT = re.compile(r'^\s*#.*$')
def __init__(self, in_path, deduplicator=None):
"""
Args:
in_path: relative path to input file, e.g. requirements/base.in
"""
self.in_path = in_path
self._dedup = deduplicator or PackageDeduplicator()
self.ignore = self._dedup.ignored_packages(in_path)
self.packages = {}
self._outfile_pkg_names = None
def maybe_create_lockfile(self):
"""
Write recursive dependencies list to outfile unless the goal is
to upgrade specific package(s) which don't already appear.
Populate package ignore set in either case and return
boolean indicating whether outfile was written.
"""
logger.info(
"Locking %s to %s. References: %r",
self.infile,
self.outfile,
sorted(self._dedup.recursive_refs(self.in_path)),
)
if not FEATURES.affected(self.in_path):
self.fix_lockfile() # populate ignore set
return False
self.create_lockfile()
return True
def create_lockfile(self):
"""
Write recursive dependencies list to outfile
with hard-pinned versions.
Then fix it.
"""
original_in_file = ""
sink_out_path = FEATURES.sink_out_path()
try:
if sink_out_path and sink_out_path != self.outfile:
original_in_file = self._read_infile()
self._inject_sink()
with subprocess.Popen(self.pin_command, **FEATURES.pipe_arguments()) as process:
stdout, stderr = process.communicate()
finally:
if original_in_file:
self._restore_in_file(original_in_file)
if process.returncode == 0:
self.fix_lockfile()
else:
logger.critical("ERROR executing %s", ' '.join(self.pin_command))
logger.critical("Exit code: %s", process.returncode)
if stdout:
logger.critical(stdout.decode('utf-8'))
if stderr:
logger.critical(stderr.decode('utf-8'))
raise RuntimeError("Failed to pip-compile {0}".format(self.infile))
@classmethod
def parse_references(cls, filename):
"""
Read filename line by line searching for pattern:
-r file.in
or
--requirement file.in
return set of matched file names.
E.g. {'file1.in', 'file2.in'}
"""
references = set()
with open(filename, encoding="utf-8") as fobj:
for line in fobj:
matched = cls.RE_REF.match(line)
if matched:
references.add(matched.group('path'))
return references
@property
def name(self):
"""Generate environment name from in_path."""
return extract_env_name(self.in_path)
@property
def infile(self):
"""Path of the input file"""
return self.in_path
@property
def outfile(self):
"""Path of the output file"""
return FEATURES.compose_output_file_path(self.in_path)
@property
def pin_command(self):
"""Compose dependency resolution command based on selected tool"""
# Use the same interpreter binary
parts = [sys.executable or 'python', '-m']
parts.extend(FEATURES.pin_command())
parts.extend(FEATURES.pin_options(self.in_path))
parts.extend(['--output-file', self.outfile, self.infile])
return parts
def fix_lockfile(self):
"""Run each section of outfile through fix_pin"""
with open(self.outfile, 'rt', encoding="utf-8") as fp:
sections = [
self.fix_pin(section)
for section in self.parse_sections(self.concatenated(fp))
]
with open(self.outfile, 'wt', encoding="utf-8") as fp:
fp.writelines([
section + '\n'
for section in sections
if section is not None
])
self._dedup.register_packages_for_env(self.in_path, self.packages)
@staticmethod
def concatenated(fp):
r"""Read lines from fp concatenating on backslash (\\)
>>> env = Environment('')
>>> list(env.concatenated([
... 'pkg', 'pkg # comment', 'pkg', '# comment', '# one more',
... 'foo', ' # via', '', ' # pkg',
... ]))
['pkg', 'pkg # comment', 'pkg', '# comment', '# one more', 'foo', ' # via', '', ' # pkg']
"""
line_parts = []
for line in fp:
line = line.rstrip()
if line.endswith('\\'):
line_parts.append(line[:-1].rstrip())
else:
line_parts.append(line)
yield ' '.join(line_parts)
line_parts[:] = []
if line_parts:
# Impossible:
raise RuntimeError("Compiled file ends with backslash \\")
def parse_sections(self, lines):
r"""Combine lines with following comments into sections.
>>> env = Environment('')
>>> list(env.parse_sections([
... 'pkg', 'pkg # comment', 'pkg', '# comment', '# one more',
... 'foo', ' # via', '', ' # pkg',
... ]))
['pkg', 'pkg # comment', 'pkg\n# comment\n# one more', 'foo\n # via', '\n # pkg']
"""
section = []
for line in lines:
if self.RE_COMMENT.match(line):
section.append(line)
else:
if section:
yield '\n'.join(section)
section = [line]
if section:
yield '\n'.join(section)
def fix_pin(self, section):
"""
Fix dependency by removing post-releases from versions
and loosing constraints on internal packages.
Drop packages from ignore set
Also populate packages set
"""
dep = Dependency(section)
if dep.valid:
if dep.package in self.ignore:
ignored_version = self.ignore[dep.package]
if ignored_version is not None:
# ignored_version can be None to disable conflict detection
if dep.version and dep.version != ignored_version:
logger.error(
"Package %s was resolved to different "
"versions in different environments: %s and %s",
dep.package, dep.version, ignored_version,
)
raise RuntimeError(
"Please add constraints for the package "
"version listed above"
)
return None
self.packages[dep.package] = dep.version
dep.drop_post(self.in_path)
return dep.serialize()
return section.rstrip()
def add_references(self, other_in_paths):
"""Add references to other_in_paths in outfile"""
if not other_in_paths:
# Skip on empty list
return
with open(self.outfile, 'rt', encoding="utf-8") as fp:
header, body = self.split_header(fp)
with open(self.outfile, 'wt', encoding="utf-8") as fp:
fp.writelines(header)
fp.writelines(
'-r {0}\n'.format(
FEATURES.compose_output_file_path(other_in_path)
)
for other_in_path in sorted(other_in_paths)
)
fp.writelines(body)
@staticmethod
def split_header(fp):
"""
Read file pointer and return pair of lines lists:
first - header, second - the rest.
"""
body_start, header_ended = 0, False
lines = []
for line in fp:
if line.startswith('#') and not header_ended:
# Header text
body_start += 1
else:
header_ended = True
lines.append(line)
return lines[:body_start], lines[body_start:]
def replace_header(self, header_text):
"""Replace pip-compile header with custom text"""
with open(self.outfile, 'rt', encoding="utf-8") as fp:
_, body = self.split_header(fp)
with open(self.outfile, 'wt', encoding="utf-8") as fp:
fp.write(header_text)
fp.writelines(body)
def _read_infile(self):
with open(self.infile, "rt", encoding="utf-8") as fp:
return fp.read()
def _restore_in_file(self, content):
with open(self.infile, "wt", encoding="utf-8") as fp:
return fp.write(content)
def _inject_sink(self):
rel_sink_out_path = os.path.normpath(os.path.relpath(
FEATURES.sink_out_path(),
os.path.dirname(self.infile),
))
with open(self.infile, "at", encoding="utf-8") as fp:
return fp.write("\n\n-c {}\n".format(rel_sink_out_path))
================================================
FILE: pipcompilemulti/features/__init__.py
================================================
"""Features as modules."""
from .controller import FeaturesController
FEATURES = FeaturesController()
================================================
FILE: pipcompilemulti/features/add_hashes.py
================================================
"""
Generate hashes
===============
Put package hash after pinned version for additional security.
Format for this option is
.. code-block:: text
-g, --generate-hashes TEXT Input file name (base.in, requirements/test.in, etc)
that needs packages hashes.
Can be supplied multiple times.
For backwards compatibility can be short
environment name (base, test, etc.)
In configuration file, use ``generate_hashes`` option with comma-separated list of paths::
[requirements]
generate_hashes = requirements/base.txt, requirements/docs.txt
Example invocation:
.. code-block:: shell
$ pip-compile-multi -g requirements/base.txt -g requirements/docs.txt
Example output:
.. code-block:: text
pip-tools==1.11.0 \
--hash=sha256:50288eb066ce66dbef5401a21530712a93c659fe480c7d8d34e2379300555fa1 \
--hash=sha256:ba427b68443466c389e3b0b0ef55f537ab39344190ea980dfebb333d0e6a50a3
first==2.0.1 \
--hash=sha256:3bb3de3582cb27071cfb514f00ed784dc444b7f96dc21e140de65fe00585c95e \
--hash=sha256:41d5b64e70507d0c3ca742d68010a76060eea8a3d863e9b5130ab11a4a91aa0e \
# via pip-tools
``pip`` requires all packages to have hashes if at least one has it.
``pip-compile-multi`` will recursively propagate this option to all
environments that are referencing or referenced by selected environments.
""" # noqa: E501
import os
from pipcompilemulti.utils import reference_cluster
from .base import BaseFeature, ClickOption
class AddHashes(BaseFeature):
"""Write hashes for pinned packages."""
OPTION_NAME = 'generate_hashes'
CLICK_OPTION = ClickOption(
long_option='--generate-hashes',
short_option='-g',
multiple=True,
help_text='Input file name (base.in, requirements/test.in, etc) '
'that needs packages hashes. '
'Can be supplied multiple times.',
)
def __init__(self, controller):
self._controller = controller
self._hashed_by_reference = None
@property
def enabled_in_paths(self):
"""Convert list of .in paths to a set.
For backwards compatibility, check if passed value is env name
and convert it to in_path.
"""
names_or_paths = self.value or []
in_paths = set()
for name_or_path in names_or_paths:
in_path = self._controller.compose_input_file_path(name_or_path)
if os.path.exists(in_path):
in_paths.add(in_path)
else:
in_paths.add(name_or_path)
return in_paths
def on_discover(self, env_confs):
"""Save environment names that need hashing."""
self._hashed_by_reference = set()
for in_path in self.enabled_in_paths:
self._hashed_by_reference.update(
reference_cluster(env_confs, in_path)
)
def _needs_hashes(self, in_path):
assert self._hashed_by_reference is not None
return in_path in self._hashed_by_reference
def pin_options(self, in_path):
"""Return --generate-hashes if env requires it."""
if self._needs_hashes(in_path):
return ['--generate-hashes']
return []
================================================
FILE: pipcompilemulti/features/annotate_index.py
================================================
"""
.. _annotate_index:
Add index URL annotation
========================
Control addition of ``--index-url`` options to the output files.
Corresponds to ``pip-compile``'s ``--emit-index-url / --no-emit-index-url`` flag.
The URL can be defined in the input file as in this example:
.. code-block:: text
--index-url=https://pypi.tuna.tsinghua.edu.cn/simple/
six
click
This option is similar to `extra_index_url`_.
The difference is that URL is saved in the output files,
which means that it doesn't need to be configured in every environment.
.. code-block:: text
--annotate-index / --no-annotate-index
Add index URL to generated files
(default false)
In configuration file, use ``annotate_index`` option::
[requirements]
annotate_index = True
Note: the default behavior is not to add the index, i.e., ``--no-annotate-index``.
"""
from .base import ClickOption
from .forward import ForwardOption
class AnnotateIndex(ForwardOption):
"""Optionally annotate the index URL to the generated files."""
OPTION_NAME = "annotate_index"
CLICK_OPTION = ClickOption(
long_option="--annotate-index/--no-annotate-index",
default=False,
is_flag=True,
help_text="Add the index URL to generated files (default false).",
)
enabled_pin_options = ["--emit-index-url"]
disabled_pin_options = ["--no-emit-index-url"]
================================================
FILE: pipcompilemulti/features/autoresolve.py
================================================
"""
.. _autoresolve:
Autoresolve cross-file conflicts
================================
Compile requirements file, that references all other files first,
and than use it as a constraint.
.. code-block:: text
--autoresolve/--no-autoresolve Automatically resolve
cross-file conflicts.
When using ``requirements`` command, this option is enabled by default.
To disable it, set it to ``False`` in a configuration file::
[requirements]
autoresolve = False
This strategy allows to resolve cross-file conflicts of two types:
1. Files FOO and BAR both have dependency PKG, but compile it to different versions.
2. FOO has PKG resolved to version 3. BAR references FOO, but also
has another dependency, that constraints PKG to version 2.
By compiling all-including file (aka *sink*), that references both FOO and BAR first,
pip-compile-multi generates conflict-free set of versions.
After that, this compiled file is passed as a constraint for compiling
all requirements files.
As the last step, the *sink* file is compiled again preserving reference files
and skipping duplicate packages.
.. note::
This feature works only if your project has a single requirements file,
that references (directly or indirectly) all other files.
"""
from pipcompilemulti.utils import recursive_refs
from .base import BaseFeature, ClickOption
class Autoresolve(BaseFeature):
"""Detect sink file and use it unless the feature is explicitly disabled."""
OPTION_NAME = 'autoresolve'
CLICK_OPTION = ClickOption(
long_option='--autoresolve/--no-autoresolve',
is_flag=True,
default=False,
help_text='Automatically resolve cross-file conflicts.',
)
def __init__(self):
self._sink_path = None
@property
def enabled(self):
"""Whether feature was explicitly disabled or not."""
return self.value
def on_discover(self, env_confs):
"""Save set of all (recursive) included environments."""
self._sink_path = self._find_sink(env_confs)
def sink_path(self):
"""Return sink path if it's enabled. Otherwise None"""
return self._sink_path if self.enabled else None
@staticmethod
def _find_sink(envs):
"""Try to find requirements sink.
Sink is a requirements file that references all other
requirement files.
If no sink exists, return None.
>>> Autoresolve._find_sink([
... {'in_path': 'base', 'refs': set()},
... {'in_path': 'test', 'refs': {'base'}},
... {'in_path': 'local', 'refs': {'test', 'base'}},
... ])
'local'
>>> Autoresolve._find_sink([
... {'in_path': 'base', 'refs': set()},
... {'in_path': 'test', 'refs': {'base'}},
... {'in_path': 'doc', 'refs': set()},
... ])
>>> Autoresolve._find_sink([
... {'in_path': 'base', 'refs': set()},
... {'in_path': 'foo', 'refs': {'base'}},
... {'in_path': 'bar', 'refs': {'base'}},
... {'in_path': 'all', 'refs': {'foo', 'bar'}},
... ])
'all'
"""
all_envs = {env['in_path'] for env in envs}
for env in envs:
included_envs = set(recursive_refs(envs, env['in_path'])) | {env['in_path']}
if all_envs == included_envs:
return env['in_path']
return None
================================================
FILE: pipcompilemulti/features/backtracking.py
================================================
"""
Backtracking resolver
=====================
Pip has an option to enable `backtracking`_ conflict resolution logic,
which can automatically downgrade some dependencies to meet constraints
from other packages.
See also the `a note on resolvers`_ in ``pip-compile`` project.
.. code-block:: text
--backtracking / --no-backtracking
Enable backtracking resolver. Translates to
pip-compile --resolver=backtracking option.
In configuration file, use ``backtracking`` option::
[requirements]
backtracking = True
.. _backtracking: https://pip.pypa.io/en/latest/user_guide/\
#changes-to-the-pip-dependency-resolver-in-20-3-2020
.. _a note on resolvers: https://github.com/jazzband/pip-tools#a-note-on-resolvers
"""
from .base import ClickOption
from .forward import ForwardOption
class Backtracking(ForwardOption):
"""Enable backtracking resolver."""
OPTION_NAME = 'backtracking'
CLICK_OPTION = ClickOption(
long_option='--backtracking/--no-backtracking',
is_flag=True,
default=False,
help_text='Enable backtracking resolver. Translates to '
'pip-compile --resolver=backtracking option.'
)
enabled_pin_options = ['--resolver=backtracking']
disabled_pin_options = ['--resolver=legacy']
================================================
FILE: pipcompilemulti/features/base.py
================================================
"""Common functionality for features activated by command line option."""
import click
from ..options import OPTIONS
class ClickOption:
"""Click option parameters."""
def __init__(self,
long_option='',
short_option='',
*,
is_flag=False,
default=None,
multiple=False,
help_text=''):
self.long_option = long_option
self.short_option = short_option
self.is_flag = is_flag
self.default = default
self.multiple = multiple
self.help_text = help_text
def decorate(self, command):
"""Decorate click command with this option."""
return self.decorator()(command)
def decorator(self):
"""Create click command decorator with this option."""
args = [self.long_option]
kwargs = {
"is_flag": self.is_flag,
"multiple": self.multiple,
"help": self.help_text,
}
if self.short_option:
args.append(self.short_option)
if self.default:
kwargs.update(default=self.default)
return click.option(*args, **kwargs)
@property
def argument_name(self):
"""Generate command argument name from long option.
>>> ClickOption("--param-name").argument_name
'param_name'
>>> ClickOption("--param-name/--no-param-name").argument_name
'param_name'
"""
return self.long_option.lstrip('--').split('/', 1)[0].replace('-', '_')
class BaseFeature:
"""Base class for features."""
OPTION_NAME = None
CLICK_OPTION = None
def bind(self, command):
"""Bind feature's click option to passed command."""
return self.CLICK_OPTION.decorate(command)
def extract_option(self, kwargs):
"""Pop option value from kwargs and save it in OPTIONS.
If option was saved before and new value is the same as default,
then keep previous value.
This allows passing options both before and after ``verify`` command.
"""
new_value = kwargs.pop(self.CLICK_OPTION.argument_name)
if self.OPTION_NAME in OPTIONS and new_value == self.CLICK_OPTION.default:
# Do not overwrite with default if already set.
return
OPTIONS[self.OPTION_NAME] = new_value
@property
def value(self):
"""Option value."""
return OPTIONS.get(self.OPTION_NAME, self.CLICK_OPTION.default)
@value.setter
def value(self, new_value):
OPTIONS[self.OPTION_NAME] = new_value
================================================
FILE: pipcompilemulti/features/base_dir.py
================================================
"""
Requirements Directory
======================
While it's a common practice to put requirements files inside ``requirements``
directory, it's not always the case.
The directory can be overridden with this option:
.. code-block:: text
-d, --directory TEXT Directory path with requirements files
In configuration file, use ``directory`` option. For example, to use project root, specify::
[requirements]
directory = .
"""
import os
from .base import BaseFeature, ClickOption
class BaseDir(BaseFeature):
"""Override input file extension."""
OPTION_NAME = 'directory'
CLICK_OPTION = ClickOption(
long_option='--directory',
short_option='-d',
default="requirements",
is_flag=False,
help_text='Directory path with requirements files.',
)
@property
def path(self):
"""Get the base directory path.
>>> BaseDir().path == 'requirements'
True
"""
return self.value
def file_path(self, file_name):
"""Compose file path for a given file name.
>>> import os.path
>>> expected = os.path.join('requirements', 'base.txt')
>>> expected == BaseDir().file_path('base.txt')
True
"""
return os.path.join(self.value, file_name)
================================================
FILE: pipcompilemulti/features/build_isolation.py
================================================
"""
Build isolation
===============
Allows disabling build isolation through the equivalent ``pip-compile`` flag.
Build isolation is enabled by default.
.. code-block:: text
--build-isolation / --no-build-isolation
Enable isolation when building a modern
source distribution. Build dependencies
specified by PEP 518 must be already
installed if build isolation is disabled.
In configuration file, use ``build_isolation`` option::
[requirements]
build_isolation = False
"""
from .base import ClickOption
from .forward import ForwardOption
class BuildIsolation(ForwardOption):
"""Proxies build isolation flag to pip-compile"""
OPTION_NAME = 'build_isolation'
CLICK_OPTION = ClickOption(
long_option='--build-isolation/--no-build-isolation',
is_flag=True,
default=True,
help_text=(
'Enable isolation when building a modern source distribution. '
'Build dependencies specified by PEP 518 must be already '
'installed if build isolation is disabled.'
),
)
disabled_pin_options = ['--no-build-isolation']
================================================
FILE: pipcompilemulti/features/compatible.py
================================================
"""
Compatible Releases
===================
`PEP-440`_ describes compatible release operator ``~=``.
Sometimes it's useful to have some of the dependencies
pinned using this operator.
For example, rapidly changing internal libraries.
The format for this option is::
-c, --compatible TEXT
where TEXT is a `glob`_ pattern for library name.
This option can be supplied multiple times.
In configuration file, use ``compatible`` option with comma-separated list of package names::
[requirements]
compatible = package1, package2
.. _glob: https://en.wikipedia.org/wiki/Glob_(programming)
.. _PEP-440: https://www.python.org/dev/peps/pep-0440/#compatible-release
"""
from fnmatch import fnmatch
from .base import BaseFeature, ClickOption
class Compatible(BaseFeature):
"""Use ~= for selected packages."""
OPTION_NAME = 'compatible_patterns'
CLICK_OPTION = ClickOption(
long_option='--compatible',
short_option='-c',
multiple=True,
help_text='Glob expression for packages with compatible (~=) '
'version constraint. Can be supplied multiple times.'
)
@property
def patterns(self):
"""Use empty list as the default."""
return self.value or []
def constraint(self, package_name):
"""Return ``~=`` if package_name matches patterns, ``==`` otherwise.
>>> from pipcompilemulti.options import OPTIONS
>>> feature = Compatible()
>>> OPTIONS[feature.OPTION_NAME] = ['xxx']
>>> feature.constraint('package')
'=='
>>> feature.constraint('xxx')
'~='
"""
return '~=' if self.is_matched(package_name) else '=='
def is_matched(self, package_name):
"""Whether package name matches one of configured glob patterns."""
package = package_name.lower()
for pattern in self.patterns:
if fnmatch(package, pattern):
return True
return False
================================================
FILE: pipcompilemulti/features/controller.py
================================================
"""Aggregate all features in a single controller."""
import os
from functools import wraps
from .add_hashes import AddHashes
from .annotate_index import AnnotateIndex
from .autoresolve import Autoresolve
from .backtracking import Backtracking
from .base_dir import BaseDir
from .build_isolation import BuildIsolation
from .compatible import Compatible
from .emit_find_links import EmitFindLinks
from .emit_trusted_host import EmitTrustedHost
from .extra_index_url import ExtraIndexUrl
from .file_extensions import InputExtension, OutputExtension
from .forbid_post import ForbidPost
from .header import CustomHeader
from .limit_in_paths import LimitInPaths
from .live_output import LiveOutput
from .skip_constraint_comments import SkipConstraintComments
from .strip_extras import StripExtras
from .unsafe import AllowUnsafe
from .upgrade import UpgradeAll, UpgradeSelected
from .use_cache import UseCache
from .use_uv import UseUV
class FeaturesController:
"""Gateway to a list of features."""
# pylint: disable=too-many-instance-attributes
def __init__(self):
self.add_hashes = AddHashes(self)
self.allow_unsafe = AllowUnsafe()
self.annotate_index = AnnotateIndex()
self.autoresolve = Autoresolve()
self.backtracking = Backtracking()
self.base_dir = BaseDir()
self.build_isolation = BuildIsolation()
self.compatible = Compatible()
self.emit_find_links = EmitFindLinks()
self.emit_trusted_host = EmitTrustedHost()
self.extra_index_url = ExtraIndexUrl()
self.forbid_post = ForbidPost()
self.header = CustomHeader()
self.input_extension = InputExtension()
self.limit_in_paths = LimitInPaths()
self.live_output = LiveOutput()
self.output_extension = OutputExtension()
self.skip_constraint_comments = SkipConstraintComments()
self.strip_extras = StripExtras()
self.upgrade_all = UpgradeAll(self)
self.upgrade_selected = UpgradeSelected(self)
self.use_cache = UseCache()
self.use_uv = UseUV()
self._features = [
self.add_hashes,
self.allow_unsafe,
self.annotate_index,
self.autoresolve,
self.backtracking,
self.base_dir,
self.build_isolation,
self.compatible,
self.emit_find_links,
self.emit_trusted_host,
self.extra_index_url,
self.forbid_post,
self.header,
self.input_extension,
self.limit_in_paths,
self.live_output,
self.output_extension,
self.skip_constraint_comments,
self.strip_extras,
self.upgrade_all,
self.upgrade_selected,
self.use_cache,
self.use_uv,
]
def bind(self, command):
"""Bind all features to click command."""
@wraps(command)
def save_command_options(*args, **kwargs):
"""Save option values and call original command without it."""
for feature in self._features:
feature.extract_option(kwargs)
return command(*args, **kwargs)
for feature in self._features:
save_command_options = feature.bind(save_command_options)
return save_command_options
def pin_command(self):
"""Return list of pin command parameters."""
if self.use_uv.value:
if not self.use_uv.is_available():
raise RuntimeError(
"UV package is not available. "
"Please install it with: pip install uv"
)
return [
'uv',
'pip',
'compile',
'--no-header',
]
return [
'piptools',
'compile',
'--no-header',
'--verbose',
]
def pin_options(self, in_path):
"""Return list of options to pin command."""
options = []
options.extend(self.add_hashes.pin_options(in_path))
options.extend(self.annotate_index.pin_options())
options.extend(self.build_isolation.pin_options())
options.extend(self.emit_find_links.pin_options())
options.extend(self.extra_index_url.pin_options())
options.extend(self.upgrade_all.pin_options())
options.extend(self.upgrade_selected.pin_options())
if not self.use_uv.value:
options.extend(self.allow_unsafe.pin_options())
options.extend(self.emit_trusted_host.pin_options())
options.extend(self.backtracking.pin_options())
options.extend(self.use_cache.pin_options())
options.extend(self.strip_extras.pin_options())
return options
def compose_input_file_path(self, basename):
"""Return input file path by environment name."""
return self.base_dir.file_path(
self.input_extension.compose_input_file_name(basename)
)
def compose_output_file_path(self, in_path):
"""Return output file path by environment name."""
return self.output_extension.compose_output_file_path(in_path)
def drop_post(self, in_path, package_name, version):
"""Whether post versions are forbidden for passed environment name."""
if self.forbid_post.post_forbidden(in_path):
return self.forbid_post.drop_post(version)
if self.compatible.is_matched(package_name):
return self.forbid_post.drop_post(version)
return version
def constraint(self, package_name):
"""Return ``~=`` if package_name matches patterns, ``==`` otherwise."""
return self.compatible.constraint(package_name)
def on_discover(self, env_confs):
"""Configure features with a list of discovered environments.
Returns a new possibly shorter env list.
"""
self.upgrade_selected.reset()
self.limit_in_paths.on_discover(env_confs)
limited_env_confs = [env for env in env_confs if self.included(env['in_path'])]
self.add_hashes.on_discover(limited_env_confs)
self.autoresolve.on_discover(limited_env_confs)
return limited_env_confs
def affected(self, in_path):
"""Whether environment is affected by upgrade command."""
if self.upgrade_all.enabled:
return True
if self.upgrade_selected.affected(in_path):
return True
return in_path == self.autoresolve.sink_path()
def included(self, in_path):
"""Whether in_path is included directly or by reference."""
return self.limit_in_paths.included(in_path)
def get_header_text(self):
"""Text to put in the beginning of each generated file."""
return self.header.text
def sink_in_path(self):
"""Return input sink path if it's enabled. Otherwise None"""
return self.autoresolve.sink_path()
def sink_out_path(self):
"""Return sink output path if it's enabled and exists. Otherwise None"""
infile = self.autoresolve.sink_path()
if not infile:
return None
outfile = self.compose_output_file_path(infile)
if not os.path.exists(outfile):
return None
return outfile
def process_dependency_comments(self, comment):
"""Process comments of locked dependency (e.g. # via xxx)."""
return self.skip_constraint_comments.process_dependency_comments(comment)
def pipe_arguments(self):
"""Values for stdout and stderr arguments to subprocess.Popen."""
return self.live_output.pipe_arguments()
================================================
FILE: pipcompilemulti/features/emit_find_links.py
================================================
"""
.. _emit_find_links:
Add find-links annotation
=========================
Control addition of ``--find-links`` options to the output files.
Corresponds to ``pip-compile``'s and ``uv pip compile``'s
``--emit-find-links / --no-emit-find-links`` flag.
The URL can be defined in the input file as in this example:
.. code-block:: text
--find-links=https://download.pytorch.org/whl/torch_stable.html
torch
By default, ``--find-links`` entries from input files are preserved
in the generated output files.
Pass ``--no-emit-find-links`` to strip them.
.. code-block:: text
--emit-find-links / --no-emit-find-links
Add find-links to generated files
(default true)
In configuration file, use ``emit_find_links`` option::
[requirements]
emit_find_links = False
Note: ``uv pip compile`` strips ``--find-links`` entries by default,
so this flag is forwarded to both ``pip-tools`` and ``uv`` backends
to keep behavior consistent.
"""
from .base import ClickOption
from .forward import ForwardOption
class EmitFindLinks(ForwardOption):
"""Optionally add the find-links entries to the generated files."""
OPTION_NAME = 'emit_find_links'
CLICK_OPTION = ClickOption(
long_option='--emit-find-links/--no-emit-find-links',
is_flag=True,
default=True,
help_text="Add find-links entries to generated files (default true)",
)
enabled_pin_options = ['--emit-find-links']
disabled_pin_options = ['--no-emit-find-links']
================================================
FILE: pipcompilemulti/features/emit_trusted_host.py
================================================
"""
.. _emit_trusted_host:
Add trusted host annotation
===========================
Control addition of trusted hosts for index URLs to generated files.
Trusted hosts can be defined in ``pip.conf`` file, input requirements file,
or through an argument to ``pip`` command.
Trusted hosts can have invalid HTTPS certificate, or use unencrypted HTTP protocol.
By default, trusted hosts are saved in the generated files.
Pass ``--no-emit-trusted-host`` to remove it.
If no trusted hosts are defined, this flag doesn't have any effect.
.. code-block:: text
--emit-trusted-host / --no-emit-trusted-host
Add trusted host to generated files
(default true)
In configuration file, use ``emit_trusted_host`` option::
[requirements]
emit_trusted_host = False
See also: `annotate_index`_ and `extra_index_url`_ options.
See pip-tools issue `#382`_ for more details.
.. _#382: https://github.com/jazzband/pip-tools/issues/382
"""
from .base import ClickOption
from .forward import ForwardOption
class EmitTrustedHost(ForwardOption):
"""Optionally add the trusted host to the generated files."""
OPTION_NAME = 'emit_trusted_host'
CLICK_OPTION = ClickOption(
long_option='--emit-trusted-host/--no-emit-trusted-host',
is_flag=True,
default=True,
help_text="Add trusted host option to generated file"
)
enabled_pin_options = ['--emit-trusted-host']
disabled_pin_options = ['--no-emit-trusted-host']
================================================
FILE: pipcompilemulti/features/extra_index_url.py
================================================
"""
.. _extra_index_url:
Add additional index URL to search
==================================
Pip accepts URLs for additional package indexes through ``--extra-index-url``.
The same option can be passed to ``pip-compile-multi`` during compilation.
This option is similar to `annotate_index`_.
The difference is that the passed URL is not saved in output files,
which is helpful if the URL contains private credentials.
.. code-block:: text
--extra-index-url TEXT Add additional package index URL to search
for package versions. Can be supplied
multiple times.
In configuration file, use ``extra_index_url`` option with comma-separated list of paths::
[requirements]
extra_index_url = https://pypi.acme.com/simple
.. warning::
Using ``--extra-index-url`` option to search for packages that are not in
the main repository (such as private packages) is unsafe, per a security vulnerability
called `dependency confusion`_: an attacker can claim the package on
the public repository in a way that will ensure it gets chosen over the private package.
Use the ``--index-url`` option in pip’s configuration file or command line
to specify the feed, overriding the default.
Avoid the ``--extra-index-url`` option, which is additive and may lead
to having multiple indexes.
See `pip documentation`_ for details.
.. _pip documentation: https://pip.pypa.io/en/stable/cli/pip_install/#examples
.. _dependency confusion: https://azure.microsoft.com/en-us/resources/\
3-ways-to-mitigate-risk-using-private-package-feeds/
"""
from .base import BaseFeature, ClickOption
class ExtraIndexUrl(BaseFeature):
"""Forward extra index URLs to to pip-compile."""
_OPTION = '--extra-index-url'
OPTION_NAME = 'extra_index_url'
CLICK_OPTION = ClickOption(
long_option=_OPTION,
multiple=True,
help_text=(
'Add additional package index URL to search for package versions. '
'Can be supplied multiple times.'
)
)
def pin_options(self):
"""Pin command options."""
if self.value:
parts = []
for url in self.value:
parts.extend([self._OPTION, url])
return parts
return []
================================================
FILE: pipcompilemulti/features/file_extensions.py
================================================
"""
Requirements Files Extensions
=============================
By default ``pip-compile-multi`` compiles ``*.txt`` from ``*.in`` files.
While this is a common naming pattern, each project can use it's own:
.. code-block:: text
-i, --in-ext TEXT File extension of input files
-o, --out-ext TEXT File extension of output files
In configuration file, use ``in_ext`` and ``out_ext`` options. For example::
[requirements Hashed]
in_ext = txt
out_ext = hash
"""
import os
from .base import BaseFeature, ClickOption
class InputExtension(BaseFeature):
"""Override input file extension."""
OPTION_NAME = 'in_ext'
CLICK_OPTION = ClickOption(
long_option='--in-ext',
short_option='-i',
default="in",
is_flag=False,
help_text='File extension of input files.',
)
def compose_input_file_name(self, base_name):
"""Compose file name given environment name.
>>> InputExtension().compose_input_file_name('base')
'base.in'
"""
return '{0}.{1}'.format(base_name, self.value)
class OutputExtension(BaseFeature):
"""Override output file extension."""
OPTION_NAME = 'out_ext'
CLICK_OPTION = ClickOption(
long_option='--out-ext',
short_option='-o',
default="txt",
is_flag=False,
help_text='File extension of output files.',
)
def compose_output_file_path(self, in_path):
"""Compose file name given environment name.
>>> OutputExtension().compose_output_file_path('sub/base.in')
'sub/base.txt'
"""
return '{0}.{1}'.format(os.path.splitext(in_path)[0], self.value)
================================================
FILE: pipcompilemulti/features/forbid_post.py
================================================
"""
Forbid .postX release
=====================
``pip-compile-multi`` can remove ``.postX`` part of dependencies versions.
.. code-block:: text
-p, --forbid-post TEXT Environment name (base, test, etc) that cannot
have packages with post-release versions
(1.2.3.post777).
Can be supplied multiple times.
In configuration file, use ``forbid_post`` option with comma-separated list of packages::
[requirements]
forbid_post = package1, package2
.. note::
Be careful with this option since different maintainers treat
post releases differently.
"""
from .base import BaseFeature, ClickOption
class ForbidPost(BaseFeature):
"""Truncate postXXX from versions for selected packages."""
OPTION_NAME = 'forbid_post'
CLICK_OPTION = ClickOption(
long_option='--forbid-post',
short_option='-p',
multiple=True,
help_text="Environment name (base, test, etc) that cannot have "
'packages with post-release versions (1.2.3.post777). '
'Can be supplied multiple times.'
)
@property
def enabled_envs(self):
"""Convert to set."""
return set(self.value or [])
@staticmethod
def drop_post(version):
"""Remove .postXXXX postfix from version.
>>> ForbidPost.drop_post('1.2.3.post123')
'1.2.3'
>>> ForbidPost.drop_post('1.2.3')
'1.2.3'
"""
post_index = version.find('.post')
if post_index >= 0:
return version[:post_index]
return version
def post_forbidden(self, env_name):
"""Whether post versions are forbidden for passed environment name."""
return env_name in self.enabled_envs
================================================
FILE: pipcompilemulti/features/forward.py
================================================
"""Base feature for forwarding pip-tools options."""
from .base import BaseFeature
class ForwardOption(BaseFeature):
"""Forward command line option to pip-tools."""
#: Pin command options when feature is enabled.
enabled_pin_options = []
#: Pin command options when feature is disabled.
disabled_pin_options = []
@property
def enabled(self):
"""Is feature enabled."""
return self.value
def pin_options(self):
"""Pin command options."""
if self.enabled:
return self.enabled_pin_options
return self.disabled_pin_options
================================================
FILE: pipcompilemulti/features/header.py
================================================
"""
Custom Header
=============
``pip-compile-multi`` adds a brief header into generated files.
Override it with
.. code-block:: text
-h, --header TEXT File path with custom header text for generated files
In configuration file, use ``header_file`` option::
[requirements]
header_file = requirements/header.txt
"""
from .base import BaseFeature, ClickOption
DEFAULT_HEADER = """
#
# This file was generated by pip-compile-multi.
# To update, run:
#
# requirements upgrade
#
""".lstrip()
class CustomHeader(BaseFeature):
"""Put custom header at the beginning of locked files."""
OPTION_NAME = 'header_file'
CLICK_OPTION = ClickOption(
long_option='--header',
short_option='-h',
default='',
help_text='File path with custom header text for generated files.',
)
def __init__(self):
self._header_text = None
@property
def text(self):
"""Text to put in the beginning of each generated file."""
if self._header_text is None:
if self.value:
self._header_text = self._read_header_text()
else:
self._header_text = DEFAULT_HEADER
return self._header_text
def _read_header_text(self):
with open(self.value, encoding="utf-8") as fp:
return fp.read()
================================================
FILE: pipcompilemulti/features/limit_in_paths.py
================================================
"""
.. _limit-in-files:
Limit input files
=================
By default ``pip-compile-multi`` compiles all ``.in`` files in ``requirements`` directory.
To limit compilation to only a subset, use
.. code-block:: text
-t, --only-path TEXT Compile only for passed input file paths and
their references.
Can be supplied multiple times.
For example, to compile one file under Python2.7 and another under Python3.6, run:
.. code-block:: text
$ pip-compile-multi -t requirements/deps27.in
Locking requirements/deps27.in to requirements/deps27.txt. References: []
$ pip-compile-multi -t requirements/deps36.in
Locking requirements/deps36.in to requirements/deps36.txt. References: []
When using ``requirements`` command, use ``include_in_paths`` configuration
option with a comma-separated list of paths.
For example::
[requirements Python 3.6]
include_in_paths = requirements/deps36.in
"""
from pipcompilemulti.utils import recursive_refs
from .base import BaseFeature, ClickOption
class LimitInPaths(BaseFeature):
"""Limit discovered input files to specified subset.
>>> from pipcompilemulti.options import OPTIONS
>>> feature = LimitInPaths()
>>> OPTIONS[feature.OPTION_NAME] = ['test.in']
>>> feature.on_discover([
... {'in_path': 'base.in', 'refs': []},
... {'in_path': 'test.in', 'refs': ['base.in']},
... {'in_path': 'docs.in', 'refs': []},
... ])
>>> feature.included('base.in')
True
>>> feature.included('test.in')
True
>>> feature.included('docs.in')
False
"""
OPTION_NAME = 'include_in_paths'
CLICK_OPTION = ClickOption(
long_option='--only-path',
short_option='-t',
multiple=True,
help_text='Compile only for passed input paths and their '
'references. Can be supplied multiple times.',
)
def __init__(self):
self._all_envs = None
@property
def direct_envs(self):
"""Set of environments included by command line options."""
return set(self.value or [])
def on_discover(self, env_confs):
"""Save set of all (recursive) included environments."""
if not self.direct_envs:
# No limit means all envs included:
self._all_envs = [env['in_path'] for env in env_confs]
return
transitive_refs = {
ref
for in_path in self.direct_envs
for ref in recursive_refs(env_confs, in_path)
}
self._all_envs = self.direct_envs | transitive_refs
def included(self, in_path):
"""Whether environment is included directly or by reference."""
return in_path in self._all_envs
================================================
FILE: pipcompilemulti/features/live_output.py
================================================
"""
Live output
===========
Print debug output from pip-compile live.
If the option is disabled (by default) the debug output
is printed only in case of failure.
.. code-block:: text
--live / --no-live Print debug output from pip-compile live.
In configuration file, use ``live`` option::
[requirements]
live = True
"""
import subprocess
from .base import BaseFeature, ClickOption
class LiveOutput(BaseFeature):
"""Controls whether stdout and stderr should be printed live or at error."""
OPTION_NAME = 'live'
CLICK_OPTION = ClickOption(
long_option='--live/--no-live',
default=False,
is_flag=True,
help_text='Print debug output from pip-compile live.',
)
def pipe_arguments(self):
"""Values for stdout and stderr arguments to subprocess.Popen."""
if self.value:
return {}
return {
'stdout': subprocess.PIPE,
'stderr': subprocess.PIPE,
}
================================================
FILE: pipcompilemulti/features/skip_constraint_comments.py
================================================
"""
Skip constraints in comments of output files
============================================
When input files contain constraint references (e.g. '-c constraints.in'),
pip-compile adds it to "via" comments. For example::
rsa==3.4.2
# via
# -c constraints.txt
# google-auth
When this option is enabled (default) that snippet will be converted to::
rsa==3.4.2
# via google-auth
Saving two lines in .txt file.
This feature is especially useful in combination with :ref:`autoresolve`.
.. code-block:: text
--skip-constraints / --no-skip-constraints
Remove constraints from "via" comments.
To include constraint comments, set it to ``False`` in a configuration file::
[requirements]
skip_constraints = False
"""
import re
from .base import BaseFeature, ClickOption
class SkipConstraintComments(BaseFeature):
"""Remove lines like ``-c file.txt`` from comments in output files."""
OPTION_NAME = 'skip_constraints'
CLICK_OPTION = ClickOption(
long_option='--skip-constraints/--no-skip-constraints',
is_flag=True,
default=True,
help_text='Remove constraints from "via" comments.',
)
_RE_VIA_COMMENT = re.compile(
r'^\s*# via$'
)
_RE_CONSTRAINT_COMMENT = re.compile(
r'^\s*#\s+-c \S+$'
)
_RE_PACKAGE_COMMENT = re.compile(
r'^\s*#\s+((?:-r )?\S+)$'
)
@property
def enabled(self):
"""Whether feature was explicitly enabled or not."""
return self.value
def process_dependency_comments(self, comment):
"""Remove constraint comments if feature is enabled."""
if self.enabled:
return self._drop_sink_comment(comment)
return comment
def _drop_sink_comment(self, comment):
r"""Erase sink constraint from comments.
>>> feature = SkipConstraintComments()
>>> feature._drop_sink_comment("\n# via\n# -c smth\n# pkg\n")
'\n# via pkg'
>>> feature._drop_sink_comment(" # via pkg")
' # via pkg'
"""
lines = comment.splitlines()
if len(lines) > 2 and self._RE_VIA_COMMENT.match(lines[1]):
result = lines[:2]
for line in lines[2:]:
if self._RE_CONSTRAINT_COMMENT.match(line):
continue
result.append(line)
return "\n".join(self._collapse_single_via(result))
return comment
def _collapse_single_via(self, lines):
r"""Combine via into a single line when it has only two lines.
>>> feature = SkipConstraintComments()
>>> feature._collapse_single_via(["", "# via", "# pkg"])
['', '# via pkg']
>>> feature._collapse_single_via([" # via pkg"])
[' # via pkg']
>>> feature._collapse_single_via(["", "# via", "# -r file"])
['', '# via -r file']
"""
if len(lines) == 3:
matchobj = self._RE_PACKAGE_COMMENT.match(lines[2])
if matchobj:
package = matchobj.group(1)
return [lines[0], lines[1] + ' ' + package]
return lines
================================================
FILE: pipcompilemulti/features/strip_extras.py
================================================
"""
Strip extras
============
Instructs ``pip-compile`` to attempt to omit extras in transient dependencies,
while assuring the constraints compatibility.
.. code-block:: text
--strip-extras Try avoiding use of extras.
In configuration file, use ``strip_extras`` option::
[requirements]
strip_extras = True
"""
from .base import ClickOption
from .forward import ForwardOption
class StripExtras(ForwardOption):
"""Attempt to drop extras"""
OPTION_NAME = 'strip_extras'
CLICK_OPTION = ClickOption(
long_option='--strip-extras',
is_flag=True,
default=False,
help_text='Try avoiding use of extras.'
)
enabled_pin_options = ['--strip-extras']
================================================
FILE: pipcompilemulti/features/unsafe.py
================================================
"""
Allow Unsafe Packages
=====================
If your project depends on packages that include ``setuptools``
or other packages considered "unsafe" by ``pip-tools`` and you
still wish to have them included in the resulting requirements files,
you can pass this option to do so:
.. code-block:: text
-s, --allow-unsafe Whether or not to include 'unsafe' packages
in generated requirements files. Consult
pip-compile --help for more information
In configuration file, use ``allow_unsafe`` option::
[requirements]
allow_unsafe = True
This is commonly used with --generate-hashes to avoid
generating requirements files which `cannot be installed`_.
.. _cannot be installed: https://github.com/jazzband/pip-tools/issues/806
"""
from .base import ClickOption
from .forward import ForwardOption
class AllowUnsafe(ForwardOption):
"""Use pip-tools cache, or rebuild from scratch."""
OPTION_NAME = 'allow_unsafe'
CLICK_OPTION = ClickOption(
long_option='--allow-unsafe',
short_option='-s',
is_flag=True,
default=False,
help_text="Whether or not to include 'unsafe' packages "
'in generated requirements files. '
'Consult pip-compile --help for more information'
)
enabled_pin_options = ['--allow-unsafe']
================================================
FILE: pipcompilemulti/features/upgrade.py
================================================
"""
Disable upgrades
================
When new dependencies are added it's tempting to keep everything else the same.
To recompile ``.txt`` keeping satisfying version use ``--no-upgrade``:
.. code-block:: text
--upgrade / --no-upgrade Upgrade package version (default true)
The option has no effect if there are no existing ``.txt`` files.
When using the ``requirements`` command, disabling upgrades means running ``requirements lock``.
Upgrades are enabled when running ``requirements upgrade``.
Upgrade only selected packages
==============================
To upgrade only one package and keep everything else untouched,
use following option:
.. code-block:: text
-P, --upgrade-package TEXT Only upgrade named package.
Can be supplied multiple times.
When using the ``requirements`` command, to upgrade only selected packages
pass the list of packages to ``requirements upgrade`` command::
requirements upgrade Django cryptography
Under the hood it uses `the same option of pip-compile`_
and runs compilation only for files that have one of the passed packages.
This option implies ``--no-upgrade`` and takes precedence over ``--upgrade``.
Thanks to `Jonathan Rogers <https://github.com/JonathanRRogers>`_.
.. _`the same option of pip-compile`: \
https://github.com/jazzband/pip-tools#updating-requirements
"""
import re
from .base import BaseFeature, ClickOption
from .forward import ForwardOption
class UpgradeAll(ForwardOption):
"""Upgrade all packages in all environments."""
OPTION_NAME = 'upgrade'
CLICK_OPTION = ClickOption(
long_option='--upgrade/--no-upgrade',
default=True,
is_flag=True,
help_text='Upgrade package version (default true)',
)
enabled_pin_options = ['--upgrade']
def __init__(self, controller):
self._controller = controller
@property
def enabled(self):
"""Whether global upgrade is enabled."""
return self.value and not self._controller.upgrade_selected.active
class UpgradeSelected(BaseFeature):
"""Upgrade only specific packages in all environments."""
OPTION_NAME = 'upgrade_packages'
CLICK_OPTION = ClickOption(
long_option='--upgrade-package',
short_option='-P',
multiple=True,
help_text='Only upgrade named package. '
'Can be supplied multiple times.',
)
RE_PACKAGE_NAME = re.compile(
r'(?iu)(?P<package>[a-z0-9-_.]+)',
)
def __init__(self, controller):
self._controller = controller
self.reset()
def reset(self):
"""Clear cached packages."""
self._env_packages_cache = {}
@property
def package_specs(self):
"""List of package specs to upgrade."""
return self.value or []
@property
def package_names(self):
"""List of package names to upgrade."""
def name_from_spec(name):
match = self.RE_PACKAGE_NAME.match(name)
if match is None:
raise ValueError(
f"{name!r} does not appear to be a valid package spec",
)
return match.group(0)
return [name_from_spec(x) for x in self.package_specs]
@property
def active(self):
"""Whether selective upgrade is active."""
return bool(self.package_names)
def pin_options(self):
"""Pin command options for upgrading specific packages."""
return [
'--upgrade-package=' + package
for package in self.package_specs
]
def has_package(self, in_path, package_name):
"""Whether specified package name is already in the outfile."""
return package_name.lower() in self._get_packages(in_path)
def _get_packages(self, in_path):
if in_path not in self._env_packages_cache:
self._env_packages_cache[in_path] = self._read_packages(
self._compose_output_file_path(in_path)
)
return self._env_packages_cache[in_path]
@staticmethod
def _read_packages(outfile):
try:
with open(outfile, encoding="utf-8") as fp:
return set(
line.split('==', 1)[0].lower()
for line in fp
if '==' in line
)
except IOError:
# Act as if file is empty
return set()
def _compose_output_file_path(self, in_path):
return self._controller.compose_output_file_path(in_path)
def affected(self, in_path):
"""Whether environment was affected by upgraded packages."""
if not self.active:
return True
return any(
self.has_package(in_path, package_name)
for package_name in self.package_names
)
================================================
FILE: pipcompilemulti/features/use_cache.py
================================================
"""
Use Cache
=========
By default ``pip-compile-multi`` executes ``pip-compile`` without ``--rebuild`` flag.
``--rebuild`` flag clears any caches upfront and rebuilds from scratch.
Option ``--no-use-cache`` adds ``--rebuild`` flag.
.. code-block:: text
--use-cache / --no-use-cache Use pip-tools cache to speed up compilation
(default true)
In configuration file, use ``use_cache`` option::
[requirements]
use_cache = False
.. note::
But if you run into "magical_" issues,
try rerunning compilation with ``--no-use-cache``.
.. _magical: https://github.com/jazzband/pip-tools/issues?q=--rebuild
"""
from .base import ClickOption
from .forward import ForwardOption
class UseCache(ForwardOption):
"""Use pip-tools cache, or rebuild from scratch."""
OPTION_NAME = 'use_cache'
CLICK_OPTION = ClickOption(
long_option='--use-cache/--no-use-cache',
is_flag=True,
default=True,
help_text='Use pip-tools cache to speed up compilation (default true)',
)
disabled_pin_options = ['--rebuild']
================================================
FILE: pipcompilemulti/features/use_uv.py
================================================
"""
Enable UV
=========
UV is an extremely fast Python package installer and resolver written in Rust.
When enabled, pip-compile-multi will use uv's dependency resolver instead of pip-tools.
.. code-block:: text
--uv / --no-uv Use uv for dependency resolution.
In configuration file, use ``uv`` option::
[requirements]
uv = True
Key differences between uv and pip-tools output:
1. UV is significantly faster at dependency resolution.
Particularly noticeable in large projects with complex dependency trees.
2. UV's resolver is more aggressive at finding newer versions.
To use UV:
- Install ``uv`` (``pip install uv``)
- Pass ``--uv`` flag to ``pip-compile-multi``
or add ``uv = True`` when using ``requirements`` command.
"""
try:
import uv #
del uv
UV_AVAILABLE = True
except ImportError:
UV_AVAILABLE = False
from .base import BaseFeature, ClickOption
class UseUV(BaseFeature):
"""Use uv for dependency resolution.
This feature enables using uv's fast Rust-based dependency resolver
instead of pip-tools. UV must be installed (pip install uv>=0.1.0)
before using this feature.
"""
OPTION_NAME = 'uv'
CLICK_OPTION = ClickOption(
long_option='--uv/--no-uv',
default=False,
is_flag=True,
help_text='Use uv for dependency resolution.',
)
@staticmethod
def is_available():
"""Check if uv package is available"""
return UV_AVAILABLE
================================================
FILE: pipcompilemulti/options.py
================================================
"""Global dictionary holding configuration options."""
OPTIONS = {}
================================================
FILE: pipcompilemulti/utils.py
================================================
"""Functional utilities for lists and dicts manipulation."""
import os
import logging
import itertools
logger = logging.getLogger("pip-compile-multi")
def extract_env_name(file_path):
"""Return environment name for given requirements file path.
>>> extract_env_name("base.in")
'base'
>>> extract_env_name("sub/req.in")
'req'
"""
return os.path.splitext(os.path.basename(file_path))[0]
def fix_reference_path(orig_path, ref_path):
"""Find actual path to reference using relative path to original file.
>>> fix_reference_path("dir/file", "../ref")
'ref'
"""
return os.path.normpath(os.path.join(os.path.dirname(orig_path), ref_path))
def recursive_refs(envs, in_path):
"""Return set of recursive refs for given env name."""
refs_by_in_path = {
os.path.normpath(env['in_path']): {
fix_reference_path(env['in_path'], ref)
for ref in env['refs']
}
for env in envs
}
refs = refs_by_in_path[os.path.normpath(in_path)]
if refs:
indirect_refs = set(
subref
for ref in refs
for subref in recursive_refs(envs, ref)
)
else:
indirect_refs = set()
return set.union(refs, indirect_refs)
def merged_packages(env_packages, names):
"""Return union set of environment packages with given names.
>>> sorted(merged_packages(
... {
... 'a': {'x': 1, 'y': 2},
... 'b': {'y': 2, 'z': 3},
... 'c': {'z': 3, 'w': 4}
... },
... ['a', 'b']
... ).items())
[('x', 1), ('y', 2), ('z', 3)]
"""
combined_packages = sorted(itertools.chain.from_iterable(
env_packages[name].items()
for name in names
))
result = {}
errors = set()
for name, version in combined_packages:
if name in result:
if result[name] != version:
errors.add((name, version, result[name]))
else:
result[name] = version
if errors:
for error in sorted(errors):
logger.error(
"Package %s was resolved to different "
"versions in different environments: %s and %s",
error[0], error[1], error[2],
)
raise RuntimeError(
"Please add constraints for the package version listed above"
)
return result
def reference_cluster(envs, in_path):
"""
Return set of all env in_paths referencing or
referenced by given in_path.
>>> cluster = sorted(reference_cluster([
... {'in_path': 'base', 'refs': []},
... {'in_path': 'test', 'refs': ['base']},
... {'in_path': 'local', 'refs': ['test']},
... ], 'test'))
>>> cluster == ['base', 'local', 'test']
True
"""
edges = [
set([env['in_path'], fix_reference_path(env['in_path'], ref)])
for env in envs
for ref in env['refs']
]
prev, cluster = set(), set([in_path])
while prev != cluster:
# While cluster grows
prev = set(cluster)
to_visit = []
for edge in edges:
if cluster & edge:
# Add adjacent nodes:
cluster |= edge
else:
# Leave only edges that are out
# of cluster for the next round:
to_visit.append(edge)
edges = to_visit
return cluster
================================================
FILE: pipcompilemulti/verify.py
================================================
"""
Check that ``pip-compile-multi`` was run after changes in ``.in`` file
======================================================================
``pip-compile-multi`` adds a special line (before header) at the beginning of each generated file.
This line contains a SHA1 hash of the ``.in`` file's contents.
Command
.. code-block:: shell
$ pip-compile-multi verify
OK - requirements/base.txt was generated from requirements/base.in.
OK - requirements/test.txt was generated from requirements/test.in.
OK - requirements/local.txt was generated from requirements/local.in.
OK - requirements/testwin.txt was generated from requirements/testwin.in.
Or, if using ``requirements`` command:
.. code-block:: shell
$ requirements verify
OK - requirements/base.txt was generated from requirements/base.in.
OK - requirements/test.txt was generated from requirements/test.in.
OK - requirements/local.txt was generated from requirements/local.in.
OK - requirements/testwin.txt was generated from requirements/testwin.in.
OK - requirements/base.hash was generated from requirements/base.txt.
OK - requirements/test.hash was generated from requirements/test.txt.
OK - requirements/local.hash was generated from requirements/local.txt.
OK - requirements/testwin.hash was generated from requirements/testwin.txt.
recalculates hashes for ``.in`` files and compares them with the stored values.
If verification fails, an error message is logged and exit code 1 is returned:
.. code-block:: shell
$ pip-compile-multi verify
ERROR! requirements/base.txt was not regenerated after changes in requirements/base.in.
Expecting: # SHA1:7d82ce5a82b0a6cf91b2c4debe90eb1e5ef37f37
Found: # SHA1:32737333f763ceffd22b7fcb76fbe62a538296fa
OK - requirements/test.txt was generated from requirements/test.in.
OK - requirements/local.txt was generated from requirements/local.in.
OK - requirements/testwin.txt was generated from requirements/testwin.in.
In big teams it might be a good idea to have this check in ``tox.ini``:
.. code-block:: ini
[testenv:verify]
skipsdist = true
skip_install = true
deps = pip-compile-multi
commands = pip-compile-multi verify
whitelist_externals = pip-compile-multi
"""
import hashlib
import logging
from .discover import discover
from .environment import Environment
from .features import FEATURES
logger = logging.getLogger("pip-compile-multi")
def verify_environments():
"""
For each environment verify hash comments and report failures.
If any failure occured, exit with code 1.
"""
env_confs = discover(FEATURES.compose_input_file_path('*'))
success = True
for conf in env_confs:
env = Environment(in_path=conf['in_path'])
current_comment = generate_hash_comment(env.infile)
robust_comment = generate_robust_hash_comment(env.infile)
existing_comment = parse_hash_comment(env.outfile)
if existing_comment in (robust_comment, current_comment):
logger.info("OK - %s was generated from %s.",
env.outfile, env.infile)
else:
logger.error("ERROR! %s was not regenerated after changes in %s.",
env.outfile, env.infile)
logger.error("Expecting: %s", robust_comment.strip())
logger.error("Found: %s", existing_comment.strip())
success = False
return success
def generate_hash_comment(file_path):
"""
Read file with given file_path and return string of format
# SHA1:da39a3ee5e6b4b0d3255bfef95601890afd80709
which is hex representation of SHA1 file content hash
"""
with open(file_path, 'rb') as fp:
hexdigest = hashlib.sha1(fp.read().strip()).hexdigest()
return f"# SHA1:{hexdigest}\n"
def generate_robust_hash_comment(file_path):
"""
Read file with given file_path and return string of format
# SHA1:da39a3ee5e6b4b0d3255bfef95601890afd80709
which is hex representation of SHA1 file content hash.
File content is pre-processed by stripping comments, whitespace and newlines.
"""
with open(file_path, 'rt', encoding="utf-8") as fp:
essense = ''.join(sorted(
line.split('#')[0].strip()
for line in fp
))
hexdigest = hashlib.sha1(essense.encode("utf-8")).hexdigest()
return f"# SHA1:{hexdigest}\n"
def parse_hash_comment(file_path):
"""
Read file with given file_path line by line,
return the first line that starts with "# SHA1:", like this:
# SHA1:da39a3ee5e6b4b0d3255bfef95601890afd80709
"""
with open(file_path, encoding="utf-8") as fp:
for line in fp:
if line.startswith("# SHA1:"):
return line
return ''
================================================
FILE: requirements/base.hash
================================================
# SHA1:9afdc18a682b2ce622499600e46778b76e9a9850
#
# This file was generated by pip-compile-multi.
# To update, run:
#
# requirements upgrade
#
build==1.5.0 \
--hash=sha256:13f3eecb844759ab66efec90ca17639bbf14dc06cb2fdf37a9010322d9c50a6f \
--hash=sha256:302c22c3ba2a0fd5f3911918651341ebb3896176cbdec15bd421f80b1afc7647
# via
# -r requirements/base.txt
# pip-tools
click==8.3.3 \
--hash=sha256:398329ad4837b2ff7cbe1dd166a4c0f8900c3ca3a218de04466f38f6497f18a2 \
--hash=sha256:a2bf429bb3033c89fa4936ffb35d5cb471e3719e1f3c8a7c3fff0b8314305613
# via
# -r requirements/base.txt
# pip-tools
packaging==26.2 \
--hash=sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e \
--hash=sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661
# via
# -r requirements/base.txt
# build
# wheel
pip==26.1.1 \
--hash=sha256:99cb1c2899893b075ff56e4ed0af55669a955b49ad7fb8d8603ecdaf4ed653fb \
--hash=sha256:d36762751d156a4ee895de8af39aa0abeeeb577f93a2eca6ab62467bbf0f8a78
# via
# -r requirements/base.txt
# pip-tools
pip-tools==7.5.3 \
--hash=sha256:3aac0c473240ae90db7213c033401f345b05197293ccbdd2704e52e7a783785e \
--hash=sha256:8fa364779ebc010cbfe17cb9de404457ac733e100840423f28f6955de7742d41
# via -r requirements/base.txt
pyproject-hooks==1.2.0 \
--hash=sha256:1e859bd5c40fae9448642dd871adf459e5e2084186e8d2c2a79a824c970da1f8 \
--hash=sha256:9e5c6bfa8dcc30091c74b0cf803c81fdd29d94f01992a7707bc97babb1141913
# via
# -r requirements/base.txt
# build
# pip-tools
setuptools==82.0.1 \
--hash=sha256:7d872682c5d01cfde07da7bccc7b65469d3dca203318515ada1de5eda35efbf9 \
--hash=sha256:a59e362652f08dcd477c78bb6e7bd9d80a7995bc73ce773050228a348ce2e5bb
# via
# -r requirements/base.txt
# pip-tools
tomli==2.4.1 \
--hash=sha256:01f520d4f53ef97964a240a035ec2a869fe1a37dde002b57ebc4417a27ccd853 \
--hash=sha256:0d85819802132122da43cb86656f8d1f8c6587d54ae7dcaf30e90533028b49fe \
--hash=sha256:136443dbd7e1dee43c68ac2694fde36b2849865fa258d39bf822c10e8068eac5 \
--hash=sha256:1d8591993e228b0c930c4bb0db464bdad97b3289fb981255d6c9a41aedc84b2d \
--hash=sha256:2190f2e9dd7508d2a90ded5ed369255980a1bcdd58e52f7fe24b8162bf9fedbd \
--hash=sha256:2c1c351919aca02858f740c6d33adea0c5deea37f9ecca1cc1ef9e884a619d26 \
--hash=sha256:36d2bd2ad5fb9eaddba5226aa02c8ec3fa4f192631e347b3ed28186d43be6b54 \
--hash=sha256:3d48a93ee1c9b79c04bb38772ee1b64dcf18ff43085896ea460ca8dec96f35f6 \
--hash=sha256:47149d5bd38761ac8be13a84864bf0b7b70bc051806bc3669ab1cbc56216b23c \
--hash=sha256:4ab97e64ccda8756376892c53a72bd1f964e519c77236368527f758fbc36a53a \
--hash=sha256:4b605484e43cdc43f0954ddae319fb75f04cc10dd80d830540060ee7cd0243cd \
--hash=sha256:504aa796fe0569bb43171066009ead363de03675276d2d121ac1a4572397870f \
--hash=sha256:51529d40e3ca50046d7606fa99ce3956a617f9b36380da3b7f0dd3dd28e68cb5 \
--hash=sha256:52c8ef851d9a240f11a88c003eacb03c31fc1c9c4ec64a99a0f922b93874fda9 \
--hash=sha256:559db847dc486944896521f68d8190be1c9e719fced785720d2216fe7022b662 \
--hash=sha256:5a881ab208c0baf688221f8cecc5401bd291d67e38a1ac884d6736cbcd8247e9 \
--hash=sha256:5cb41aa38891e073ee49d55fbc7839cfdb2bc0e600add13874d048c94aadddd1 \
--hash=sha256:5e262d41726bc187e69af7825504c933b6794dc3fbd5945e41a79bb14c31f585 \
--hash=sha256:5ee18d9ebdb417e384b58fe414e8d6af9f4e7a0ae761519fb50f721de398dd4e \
--hash=sha256:7008df2e7655c495dd12d2a4ad038ff878d4ca4b81fccaf82b714e07eae4402c \
--hash=sha256:734e20b57ba95624ecf1841e72b53f6e186355e216e5412de414e3c51e5e3c41 \
--hash=sha256:7c7e1a961a0b2f2472c1ac5b69affa0ae1132c39adcb67aba98568702b9cc23f \
--hash=sha256:7f86fd587c4ed9dd76f318225e7d9b29cfc5a9d43de44e5754db8d1128487085 \
--hash=sha256:7f94b27a62cfad8496c8d2513e1a222dd446f095fca8987fceef261225538a15 \
--hash=sha256:88dceee75c2c63af144e456745e10101eb67361050196b0b6af5d717254dddf7 \
--hash=sha256:8a650c2dbafa08d42e51ba0b62740dae4ecb9338eefa093aa5c78ceb546fcd5c \
--hash=sha256:8d65a2fbf9d2f8352685bc1364177ee3923d6baf5e7f43ea4959d7d8bc326a36 \
--hash=sha256:96481a5786729fd470164b47cdb3e0e58062a496f455ee41b4403be77cb5a076 \
--hash=sha256:a120733b01c45e9a0c34aeef92bf0cf1d56cfe81ed9d47d562f9ed591a9828ac \
--hash=sha256:b1d22e6e9387bf4739fbe23bfa80e93f6b0373a7f1b96c6227c32bef95a4d7a8 \
--hash=sha256:b8c198f8c1805dc42708689ed6864951fd2494f924149d3e4bce7710f8eb5232 \
--hash=sha256:c2541745709bad0264b7d4705ad453b76ccd191e64aa6f0fc66b69a293a45ece \
--hash=sha256:c742f741d58a28940ce01d58f0ab2ea3ced8b12402f162f4d534dfe18ba1cd6a \
--hash=sha256:c7f2c7f2b9ca6bdeef8f0fa897f8e05085923eb091721675170254cbc5b02897 \
--hash=sha256:d312ef37c91508b0ab2cee7da26ec0b3ed2f03ce12bd87a588d771ae15dcf82d \
--hash=sha256:d4d8fe59808a54658fcc0160ecfb1b30f9089906c50b23bcb4c69eddc19ec2b4 \
--hash=sha256:da25dc3563bff5965356133435b757a795a17b17d01dbc0f42fb32447ddfd917 \
--hash=sha256:eab21f45c7f66c13f2a9e0e1535309cee140182a9cdae1e041d02e47291e8396 \
--hash=sha256:eb0dc4e38e6a1fd579e5d50369aa2e10acfc9cace504579b2faabb478e76941a \
--hash=sha256:ec9bfaf3ad2df51ace80688143a6a4ebc09a248f6ff781a9945e51937008fcbc \
--hash=sha256:ede3e6487c5ef5d28634ba3f31f989030ad6af71edfb0055cbbd14189ff240ba \
--hash=sha256:f3c6818a1a86dd6dca7ddcaaf76947d5ba31aecc28cb1b67009a5877c9a64f3f \
--hash=sha256:f758f1b9299d059cc3f6546ae2af89670cb1c4d48ea29c3cacc4fe7de3058257 \
--hash=sha256:f8f0fc26ec2cc2b965b7a3b87cd19c5c6b8c5e5f436b984e85f486d652285c30 \
--hash=sha256:fd0409a3653af6c147209d267a0e4243f0ae46b011aa978b1080359fddc9b6cf \
--hash=sha256:ff18e6a727ee0ab0388507b89d1bc6a22b138d1e2fa56d1ad494586d61d2eae9 \
--hash=sha256:ff2983983d34813c1aeb0fa89091e76c3a22889ee83ab27c5eeb45100560c049
# via
# -r requirements/base.txt
# build
# pip-tools
toposort==1.10 \
--hash=sha256:bfbb479c53d0a696ea7402601f4e693c97b0367837c8898bc6471adfca37a6bd \
--hash=sha256:cbdbc0d0bee4d2695ab2ceec97fe0679e9c10eab4b2a87a9372b929e70563a87
# via -r requirements/base.txt
wheel==0.47.0 \
--hash=sha256:212281cab4dff978f6cedd499cd893e1f620791ca6ff7107cf270781e587eced \
--hash=sha256:cc72bd1009ba0cf63922e28f94d9d83b920aa2bb28f798a31d0691b02fa3c9b3
# via
# -r requirements/base.txt
# pip-tools
================================================
FILE: requirements/base.in
================================================
click
pip-tools>=6.8.0
toposort
================================================
FILE: requirements/base.txt
================================================
# SHA1:32737333f763ceffd22b7fcb76fbe62a538296fa
#
# This file was generated by pip-compile-multi.
# To update, run:
#
# requirements upgrade
#
build==1.5.0
# via pip-tools
click==8.3.3
# via
# -r requirements/base.in
# pip-tools
packaging==26.2
# via
# build
# wheel
pip==26.1.1
# via pip-tools
pip-tools==7.5.3
# via -r requirements/base.in
pyproject-hooks==1.2.0
# via
# build
# pip-tools
setuptools==82.0.1
# via pip-tools
tomli==2.4.1
# via
# build
# pip-tools
toposort==1.10
# via -r requirements/base.in
wheel==0.47.0
# via pip-tools
================================================
FILE: requirements/local.hash
================================================
# SHA1:08b9c753a9730ea1f06fb8c80c0a3fb45797897c
#
# This file was generated by pip-compile-multi.
# To update, run:
#
# requirements upgrade
#
-r test.hash
alabaster==1.0.0 \
--hash=sha256:c00dca57bca26fa62a6d7d0a9fcce65f3e026e9bfe33e9c538fd3fbb2144fd9e \
--hash=sha256:fc6786402dc3fcb2de3cabd5fe455a2db534b371124f1f21de8731783dec828b
# via
# -r requirements/local.txt
# sphinx
astroid==4.0.4 \
--hash=sha256:52f39653876c7dec3e3afd4c2696920e05c83832b9737afc21928f2d2eb7a753 \
--hash=sha256:986fed8bcf79fb82c78b18a53352a0b287a73817d6dbcfba3162da36667c49a0
# via
# -r requirements/local.txt
# pylint
babel==2.18.0 \
--hash=sha256:b80b99a14bd085fcacfa15c9165f651fbb3406e66cc603abf11c5750937c992d \
--hash=sha256:e2b422b277c2b9a9630c1d7903c2a00d0830c409c59ac8cae9081c92f1aeba35
# via
# -r requirements/local.txt
# sphinx
backports-tarfile==1.2.0 \
--hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \
--hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991
# via
# -r requirements/local.txt
# jaraco-context
bump2version==1.0.1 \
--hash=sha256:37f927ea17cde7ae2d7baf832f8e80ce3777624554a653006c9144f8017fe410 \
--hash=sha256:762cb2bfad61f4ec8e2bdf452c7c267416f8c70dd9ecb1653fd0bbb01fa936e6
# via -r requirements/local.txt
cachetools==7.1.1 \
--hash=sha256:0335cd7a0952d2b22327441fb0628139e234c565559eeb91a8a4ac7551c5353d \
--hash=sha256:27bdf856d68fd3c71c26c01b5edc312124ed427524d1ddb31aa2b7746fe20d4b
# via
# -r requirements/local.txt
# tox
certifi==2026.4.22 \
--hash=sha256:3cb2210c8f88ba2318d29b0388d1023c8492ff72ecdde4ebdaddbb13a31b1c4a \
--hash=sha256:8d455352a37b71bf76a79caa83a3d6c25afee4a385d632127b6afb3963f1c580
# via
# -r requirements/local.txt
# requests
cffi==2.0.0 \
--hash=sha256:00bdf7acc5f795150faa6957054fbbca2439db2f775ce831222b66f192f03beb \
--hash=sha256:07b271772c100085dd28b74fa0cd81c8fb1a3ba18b21e03d7c27f3436a10606b \
--hash=sha256:087067fa8953339c723661eda6b54bc98c5625757ea62e95eb4898ad5e776e9f \
--hash=sha256:0a1527a803f0a659de1af2e1fd700213caba79377e27e4693648c2923da066f9 \
--hash=sha256:0cf2d91ecc3fcc0625c2c530fe004f82c110405f101548512cce44322fa8ac44 \
--hash=sha256:0f6084a0ea23d05d20c3edcda20c3d006f9b6f3fefeac38f59262e10cef47ee2 \
--hash=sha256:12873ca6cb9b0f0d3a0da705d6086fe911591737a59f28b7936bdfed27c0d47c \
--hash=sha256:19f705ada2530c1167abacb171925dd886168931e0a7b78f5bffcae5c6b5be75 \
--hash=sha256:1cd13c99ce269b3ed80b417dcd591415d3372bcac067009b6e0f59c7d4015e65 \
--hash=sha256:1e3a615586f05fc4065a8b22b8152f0c1b00cdbc60596d187c2a74f9e3036e4e \
--hash=sha256:1f72fb8906754ac8a2cc3f9f5aaa298070652a0ffae577e0ea9bd480dc3c931a \
--hash=sha256:1fc9ea04857caf665289b7a75923f2c6ed559b8298a1b8c49e59f7dd95c8481e \
--hash=sha256:203a48d1fb583fc7d78a4c6655692963b860a417c0528492a6bc21f1aaefab25 \
--hash=sha256:2081580ebb843f759b9f617314a24ed5738c51d2aee65d31e02f6f7a2b97707a \
--hash=sha256:21d1152871b019407d8ac3985f6775c079416c282e431a4da6afe7aefd2bccbe \
--hash=sha256:24b6f81f1983e6df8db3adc38562c83f7d4a0c36162885ec7f7b77c7dcbec97b \
--hash=sha256:256f80b80ca3853f90c21b23ee78cd008713787b1b1e93eae9f3d6a7134abd91 \
--hash=sha256:28a3a209b96630bca57cce802da70c266eb08c6e97e5afd61a75611ee6c64592 \
--hash=sha256:2c8f814d84194c9ea681642fd164267891702542f028a15fc97d4674b6206187 \
--hash=sha256:2de9a304e27f7596cd03d16f1b7c72219bd944e99cc52b84d0145aefb07cbd3c \
--hash=sha256:38100abb9d1b1435bc4cc340bb4489635dc2f0da7456590877030c9b3d40b0c1 \
--hash=sha256:3925dd22fa2b7699ed2617149842d2e6adde22b262fcbfada50e3d195e4b3a94 \
--hash=sha256:3e17ed538242334bf70832644a32a7aae3d83b57567f9fd60a26257e992b79ba \
--hash=sha256:3e837e369566884707ddaf85fc1744b47575005c0a229de3327f8f9a20f4efeb \
--hash=sha256:3f4d46d8b35698056ec29bca21546e1551a205058ae1a181d871e278b0b28165 \
--hash=sha256:44d1b5909021139fe36001ae048dbdde8214afa20200eda0f64c068cac5d5529 \
--hash=sha256:45d5e886156860dc35862657e1494b9bae8dfa63bf56796f2fb56e1679fc0bca \
--hash=sha256:4647afc2f90d1ddd33441e5b0e85b16b12ddec4fca55f0d9671fef036ecca27c \
--hash=sha256:4671d9dd5ec934cb9a73e7ee9676f9362aba54f7f34910956b84d727b0d73fb6 \
--hash=sha256:53f77cbe57044e88bbd5ed26ac1d0514d2acf0591dd6bb02a3ae37f76811b80c \
--hash=sha256:5eda85d6d1879e692d546a078b44251cdd08dd1cfb98dfb77b670c97cee49ea0 \
--hash=sha256:5fed36fccc0612a53f1d4d9a816b50a36702c28a2aa880cb8a122b3466638743 \
--hash=sha256:61d028e90346df14fedc3d1e5441df818d095f3b87d286825dfcbd6459b7ef63 \
--hash=sha256:66f011380d0e49ed280c789fbd08ff0d40968ee7b665575489afa95c98196ab5 \
--hash=sha256:6824f87845e3396029f3820c206e459ccc91760e8fa24422f8b0c3d1731cbec5 \
--hash=sha256:6c6c373cfc5c83a975506110d17457138c8c63016b563cc9ed6e056a82f13ce4 \
--hash=sha256:6d02d6655b0e54f54c4ef0b94eb6be0607b70853c45ce98bd278dc7de718be5d \
--hash=sha256:6d50360be4546678fc1b79ffe7a66265e28667840010348dd69a314145807a1b \
--hash=sha256:730cacb21e1bdff3ce90babf007d0a0917cc3e6492f336c2f0134101e0944f93 \
--hash=sha256:737fe7d37e1a1bffe70bd5754ea763a62a066dc5913ca57e957824b72a85e205 \
--hash=sha256:74a03b9698e198d47562765773b4a8309919089150a0bb17d829ad7b44b60d27 \
--hash=sha256:7553fb2090d71822f02c629afe6042c299edf91ba1bf94951165613553984512 \
--hash=sha256:7a66c7204d8869299919db4d5069a82f1561581af12b11b3c9f48c584eb8743d \
--hash=sha256:7cc09976e8b56f8cebd752f7113ad07752461f48a58cbba644139015ac24954c \
--hash=sha256:81afed14892743bbe14dacb9e36d9e0e504cd204e0b165062c488942b9718037 \
--hash=sha256:8941aaadaf67246224cee8c3803777eed332a19d909b47e29c9842ef1e79ac26 \
--hash=sha256:89472c9762729b5ae1ad974b777416bfda4ac5642423fa93bd57a09204712322 \
--hash=sha256:8ea985900c5c95ce9db1745f7933eeef5d314f0565b27625d9a10ec9881e1bfb \
--hash=sha256:8eca2a813c1cb7ad4fb74d368c2ffbbb4789d377ee5bb8df98373c2cc0dee76c \
--hash=sha256:92b68146a71df78564e4ef48af17551a5ddd142e5190cdf2c5624d0c3ff5b2e8 \
--hash=sha256:9332088d75dc3241c702d852d4671613136d90fa6881da7d770a483fd05248b4 \
--hash=sha256:94698a9c5f91f9d138526b48fe26a199609544591f859c870d477351dc7b2414 \
--hash=sha256:9a67fc9e8eb39039280526379fb3a70023d77caec1852002b4da7e8b270c4dd9 \
--hash=sha256:9de40a7b0323d889cf8d23d1ef214f565ab154443c42737dfe52ff82cf857664 \
--hash=sha256:a05d0c237b3349096d3981b727493e22147f934b20f6f125a3eba8f994bec4a9 \
--hash=sha256:afb8db5439b81cf9c9d0c80404b60c3cc9c3add93e114dcae767f1477cb53775 \
--hash=sha256:b18a3ed7d5b3bd8d9ef7a8cb226502c6bf8308df1525e1cc676c3680e7176739 \
--hash=sha256:b1e74d11748e7e98e2f426ab176d4ed720a64412b6a15054378afdb71e0f37dc \
--hash=sha256:b21e08af67b8a103c71a250401c78d5e0893beff75e28c53c98f4de42f774062 \
--hash=sha256:b4c854ef3adc177950a8dfc81a86f5115d2abd545751a304c5bcf2c2c7283cfe \
--hash=sha256:b882b3df248017dba09d6b16defe9b5c407fe32fc7c65a9c69798e6175601be9 \
--hash=sha256:baf5215e0ab74c16e2dd324e8ec067ef59e41125d3eade2b863d294fd5035c92 \
--hash=sha256:c649e3a33450ec82378822b3dad03cc228b8f5963c0c12fc3b1e0ab940f768a5 \
--hash=sha256:c654de545946e0db659b3400168c9ad31b5d29593291482c43e3564effbcee13 \
--hash=sha256:c6638687455baf640e37344fe26d37c404db8b80d037c3d29f58fe8d1c3b194d \
--hash=sha256:c8d3b5532fc71b7a77c09192b4a5a200ea992702734a2e9279a37f2478236f26 \
--hash=sha256:cb527a79772e5ef98fb1d700678fe031e353e765d1ca2d409c92263c6d43e09f \
--hash=sha256:cf364028c016c03078a23b503f02058f1814320a56ad535686f90565636a9495 \
--hash=sha256:d48a880098c96020b02d5a1f7d9251308510ce8858940e6fa99ece33f610838b \
--hash=sha256:d68b6cef7827e8641e8ef16f4494edda8b36104d79773a334beaa1e3521430f6 \
--hash=sha256:d9b29c1f0ae438d5ee9acb31cadee00a58c46cc9c0b2f9038c6b0b3470877a8c \
--hash=sha256:d9b97165e8aed9272a6bb17c01e3cc5871a594a446ebedc996e2397a1c1ea8ef \
--hash=sha256:da68248800ad6320861f129cd9c1bf96ca849a2771a59e0344e88681905916f5 \
--hash=sha256:da902562c3e9c550df360bfa53c035b2f241fed6d9aef119048073680ace4a18 \
--hash=sha256:dbd5c7a25a7cb98f5ca55d258b103a2054f859a46ae11aaf23134f9cc0d356ad \
--hash=sha256:dd4f05f54a52fb558f1ba9f528228066954fee3ebe629fc1660d874d040ae5a3 \
--hash=sha256:de8dad4425a6ca6e4e5e297b27b5c824ecc7581910bf9aee86cb6835e6812aa7 \
--hash=sha256:e11e82b744887154b182fd3e7e8512418446501191994dbf9c9fc1f32cc8efd5 \
--hash=sha256:e6e73b9e02893c764e7e8d5bb5ce277f1a009cd5243f8228f75f842bf937c534 \
--hash=sha256:f73b96c41e3b2adedc34a7356e64c8eb96e03a3782b535e043a986276ce12a49 \
--hash=sha256:f93fd8e5c8c0a4aa1f424d6173f14a892044054871c771f8566e4008eaa359d2 \
--hash=sha256:fc33c5141b55ed366cfaad382df24fe7dcbc686de5be719b207bb248e3053dc5 \
--hash=sha256:fc7de24befaeae77ba923797c7c87834c73648a05a4bde34b3b7e5588973a453 \
--hash=sha256:fe562eb1a64e67dd297ccc4f5addea2501664954f2692b69a76449ec7913ecbf
# via
# -r requirements/local.txt
# cryptography
cfgv==3.5.0 \
--hash=sha256:a8dc6b26ad22ff227d2634a65cb388215ce6cc96bbcc5cfde7641ae87e8dacc0 \
--hash=sha256:d5b1034354820651caa73ede66a6294d6e95c1b00acc5e9b098e917404669132
# via
# -r requirements/local.txt
# pre-commit
charset-normalizer==3.4.7 \
--hash=sha256:007d05ec7321d12a40227aae9e2bc6dca73f3cb21058999a1df9e193555a9dcc \
--hash=sha256:03853ed82eeebbce3c2abfdbc98c96dc205f32a79627688ac9a27370ea61a49c \
--hash=sha256:07d9e39b01743c3717745f4c530a6349eadbfa043c7577eef86c502c15df2c67 \
--hash=sha256:08e721811161356f97b4059a9ba7bafb23ea5ee2255402c42881c214e173c6b4 \
--hash=sha256:0c96c3b819b5c3e9e165495db84d41914d6894d55181d2d108cc1a69bfc9cce0 \
--hash=sha256:0ea948db76d31190bf08bd371623927ee1339d5f2a0b4b1b4a4439a65298703c \
--hash=sha256:0f7eb884681e3938906ed0434f20c63046eacd0111c4ba96f27b76084cd679f5 \
--hash=sha256:12a6fff75f6bc66711b73a2f0addfc4c8c15a20e805146a02d147a318962c444 \
--hash=sha256:12d8baf840cc7889b37c7c770f478adea7adce3dcb3944d02ec87508e2dcf153 \
--hash=sha256:14265bfe1f09498b9d8ec91e9ec9fa52775edf90fcbde092b25f4a33d444fea9 \
--hash=sha256:16d971e29578a5e97d7117866d15889a4a07befe0e87e703ed63cd90cb348c01 \
--hash=sha256:177a0ba5f0211d488e295aaf82707237e331c24788d8d76c96c5a41594723217 \
--hash=sha256:1a87ca9d5df6fe460483d9a5bbf2b18f620cbed41b432e2bddb686228282d10b \
--hash=sha256:1c2a768fdd44ee4a9339a9b0b130049139b8ce3c01d2ce09f67f5a68048d477c \
--hash=sha256:1c2aed2e5e41f24ea8ef1590b8e848a79b56f3a5564a65ceec43c9d692dc7d8a \
--hash=sha256:1dc8b0ea451d6e69735094606991f32867807881400f808a106ee1d963c46a83 \
--hash=sha256:1efde3cae86c8c273f1eb3b287be7d8499420cf2fe7585c41d370d3e790054a5 \
--hash=sha256:202389074300232baeb53ae2569a60901f7efadd4245cf3a3bf0617d60b439d7 \
--hash=sha256:203104ed3e428044fd943bc4bf45fa73c0730391f9621e37fe39ecf477b128cb \
--hash=sha256:2257141f39fe65a3fdf38aeccae4b953e5f3b3324f4ff0daf9f15b8518666a2c \
--hash=sha256:298930cec56029e05497a76988377cbd7457ba864beeea92ad7e844fe74cd1f1 \
--hash=sha256:2cd4a60d0e2fb04537162c62bbbb4182f53541fe0ede35cdf270a1c1e723cc42 \
--hash=sha256:2d6eb928e13016cea4f1f21d1e10c1cebd5a421bc57ddf5b1142ae3f86824fab \
--hash=sha256:2fe249cb4651fd12605b7288b24751d8bfd46d35f12a20b1ba33dea122e690df \
--hash=sha256:30b8d1d8c52a48c2c5690e152c169b673487a2a58de1ec7393196753063fcd5e \
--hash=sha256:320ade88cfb846b8cd6b4ddf5ee9e80ee0c1f52401f2456b84ae1ae6a1a5f207 \
--hash=sha256:3534e7dcbdcf757da6b85a0bbf5b6868786d5982dd959b065e65481644817a18 \
--hash=sha256:36836d6ff945a00b88ba1e4572d721e60b5b8c98c155d465f56ad19d68f23734 \
--hash=sha256:38c0109396c4cfc574d502df99742a45c72c08eff0a36158b6f04000043dbf38 \
--hash=sha256:3946fa46a0cf3e4c8cb1cc52f56bb536310d34f25f01ca9b6c16afa767dab110 \
--hash=sha256:3bec022aec2c514d9cf199522a802bd007cd588ab17ab2525f20f9c34d067c18 \
--hash=sha256:3c9a494bc5ec77d43cea229c4f6db1e4d8fe7e1bbffa8b6f0f0032430ff8ab44 \
--hash=sha256:3dce51d0f5e7951f8bb4900c257dad282f49190fdbebecd4ba99bcc41fef404d \
--hash=sha256:3dedcc22d73ec993f42055eff4fcfed9318d1eeb9a6606c55892a26964964e48 \
--hash=sha256:4042d5c8f957e15221d423ba781e85d553722fc4113f523f2feb7b188cc34c5e \
--hash=sha256:481551899c856c704d58119b5025793fa6730adda3571971af568f66d2424bb5 \
--hash=sha256:4dc1e73c36828f982bfe79fadf5919923f8a6f4df2860804db9a98c48824ce8d \
--hash=sha256:4e5163c14bffd570ef2affbfdd77bba66383890797df43dc8b4cc7d6f500bf53 \
--hash=sha256:511ef87c8aec0783e08ac18565a16d435372bc1ac25a91e6ac7f5ef2b0bff790 \
--hash=sha256:532bc9bf33a68613fd7d65e4b1c71a6a38d7d42604ecf239c77392e9b4e8998c \
--hash=sha256:54523e136b8948060c0fa0bc7b1b50c32c186f2fceee897a495406bb6e311d2b \
--hash=sha256:5649fd1c7bade02f320a462fdefd0b4bd3ce036065836d4f42e0de958038e116 \
--hash=sha256:56be790f86bfb2c98fb742ce566dfb4816e5a83384616ab59c49e0604d49c51d \
--hash=sha256:5b77459df20e08151cd6f8b9ef8ef1f961ef73d85c21a555c7eed5b79410ec10 \
--hash=sha256:5ed6ab538499c8644b8a3e18debabcd7ce684f3fa91cf867521a7a0279cab2d6 \
--hash=sha256:6178f72c5508bfc5fd446a5905e698c6212932f25bcdd4b47a757a50605a90e2 \
--hash=sha256:6370e8686f662e6a3941ee48ed4742317cafbe5707e36406e9df792cdb535776 \
--hash=sha256:64f02c6841d7d83f832cd97ccf8eb8a906d06eb95d5276069175c696b024b60a \
--hash=sha256:65bcd23054beab4d166035cabbc868a09c1a49d1efe458fe8e4361215df40265 \
--hash=sha256:66671f93accb62ed07da56613636f3641f1a12c13046ce91ffc923721f23c008 \
--hash=sha256:6696b7688f54f5af4462118f0bfa7c1621eeb87154f77fa04b9295ce7a8f2943 \
--hash=sha256:6785f414ae0f3c733c437e0f3929197934f526d19dfaa75e18fdb4f94c6fb374 \
--hash=sha256:67f6279d125ca0046a7fd386d01b311c6363844deac3e5b069b514ba3e63c246 \
--hash=sha256:6c114670c45346afedc0d947faf3c7f701051d2518b943679c8ff88befe14f8e \
--hash=sha256:6e0d51f618228538a3e8f46bd246f87a6cd030565e015803691603f55e12afb5 \
--hash=sha256:6ed74185b2db44f41ef35fd1617c5888e59792da9bbc9190d6c7300617182616 \
--hash=sha256:708838739abf24b2ceb208d0e22403dd018faeef86ddac04319a62ae884c4f15 \
--hash=sha256:715479b9a2802ecac752a3b0efa2b0b60285cf962ee38414211abdfccc233b41 \
--hash=sha256:733784b6d6def852c814bce5f318d25da2ee65dd4839a0718641c696e09a2960 \
--hash=sha256:750e02e074872a3fad7f233b47734166440af3cdea0add3e95163110816d6752 \
--hash=sha256:752a45dc4a6934060b3b0dab47e04edc3326575f82be64bc4fc293914566503e \
--hash=sha256:7579e913a5339fb8fa133f6bbcfd8e6749696206cf05acdbdca71a1b436d8e72 \
--hash=sha256:7641bb8895e77f921102f72833904dcd9901df5d6d72a2ab8f31d04b7e51e4e7 \
--hash=sha256:7804338df6fcc08105c7745f1502ba68d900f45fd770d5bdd5288ddccb8a42d8 \
--hash=sha256:80d04837f55fc81da168b98de4f4b797ef007fc8a79ab71c6ec9bc4dd662b15b \
--hash=sha256:813c0e0132266c08eb87469a642cb30aaff57c5f426255419572aaeceeaa7bf4 \
--hash=sha256:82b271f5137d07749f7bf32f70b17ab6eaabedd297e75dce75081a24f76eb545 \
--hash=sha256:84c018e49c3bf790f9c2771c45e9313a08c2c2a6342b162cd650258b57817706 \
--hash=sha256:8751d2787c9131302398b11e6c8068053dcb55d5a8964e114b6e196cf16cb366 \
--hash=sha256:8778f0c7a52e56f75d12dae53ae320fae900a8b9b4164b981b9c5ce059cd1fcb \
--hash=sha256:87fad7d9ba98c86bcb41b2dc8dbb326619be2562af1f8ff50776a39e55721c5a \
--hash=sha256:8d828b6667a32a728a1ad1d93957cdf37489c57b97ae6c4de2860fa749b8fc1e \
--hash=sha256:8e385e4267ab76874ae30db04c627faaaf0b509e1ccc11a95b3fc3e83f855c00 \
--hash=sha256:92a0a01ead5e668468e952e4238cccd7c537364eb7d851ab144ab6627dbbe12f \
--hash=sha256:94e1885b270625a9a828c9793b4d52a64445299baa1fea5a173bf1d3dd9a1a5a \
gitextract_snhzpegr/ ├── .coveragerc ├── .editorconfig ├── .github/ │ └── workflows/ │ ├── pipcompilemulti.yml │ ├── python310-windows.yml │ ├── python310.yml │ ├── python311.yml │ ├── python312.yml │ ├── python313.yml │ └── python314.yml ├── .gitignore ├── .pre-commit-config.yaml ├── .pre-commit-hooks.yaml ├── .pylintrc ├── .readthedocs.yml ├── .travis.yml ├── AUTHORS.rst ├── CONTRIBUTING.rst ├── Dockerfile ├── HISTORY.rst ├── LICENSE.txt ├── MANIFEST.in ├── Makefile ├── README.rst ├── docs/ │ ├── Makefile │ ├── _static/ │ │ └── custom.css │ ├── afterword.rst │ ├── boilerplate.rst │ ├── conf.py │ ├── features.rst │ ├── history.rst │ ├── index.rst │ ├── installation.rst │ ├── migration.rst │ ├── precommit.rst │ └── why.rst ├── how-to.md ├── nested/ │ ├── base.in │ ├── base.txt │ ├── diamond.in │ ├── diamond.txt │ ├── subproject/ │ │ ├── base.in │ │ ├── base.txt │ │ ├── sub.in │ │ └── sub.txt │ ├── up.in │ └── up.txt ├── pipcompilemulti/ │ ├── __init__.py │ ├── actions.py │ ├── cli_v1.py │ ├── cli_v2.py │ ├── config.py │ ├── deduplicate.py │ ├── dependency.py │ ├── discover.py │ ├── environment.py │ ├── features/ │ │ ├── __init__.py │ │ ├── add_hashes.py │ │ ├── annotate_index.py │ │ ├── autoresolve.py │ │ ├── backtracking.py │ │ ├── base.py │ │ ├── base_dir.py │ │ ├── build_isolation.py │ │ ├── compatible.py │ │ ├── controller.py │ │ ├── emit_find_links.py │ │ ├── emit_trusted_host.py │ │ ├── extra_index_url.py │ │ ├── file_extensions.py │ │ ├── forbid_post.py │ │ ├── forward.py │ │ ├── header.py │ │ ├── limit_in_paths.py │ │ ├── live_output.py │ │ ├── skip_constraint_comments.py │ │ ├── strip_extras.py │ │ ├── unsafe.py │ │ ├── upgrade.py │ │ ├── use_cache.py │ │ └── use_uv.py │ ├── options.py │ ├── utils.py │ └── verify.py ├── requirements/ │ ├── base.hash │ ├── base.in │ ├── base.txt │ ├── local.hash │ ├── local.in │ ├── local.txt │ ├── test.hash │ ├── test.in │ ├── test.txt │ ├── testwin.hash │ ├── testwin.in │ └── testwin.txt ├── setup.cfg ├── setup.py ├── tests/ │ ├── __init__.py │ ├── configs/ │ │ ├── pyproject.toml │ │ ├── requirements.ini │ │ └── setup.cfg │ ├── conflicting-in-merge/ │ │ ├── base1.in │ │ ├── base1.txt │ │ ├── base2.in │ │ ├── base2.txt │ │ ├── together.in │ │ └── together.txt │ ├── conflicting-in-ref/ │ │ ├── base1.in │ │ ├── base1.txt │ │ ├── base2.in │ │ └── base2.txt │ ├── conftest.py │ ├── sys_platform/ │ │ ├── base.in │ │ ├── base.txt │ │ ├── test.in │ │ └── test.txt │ ├── test_add_hashes.py │ ├── test_cli_v1.py │ ├── test_cli_v2.py │ ├── test_config.py │ ├── test_conflicts.py │ ├── test_deduplicate.py │ ├── test_dependency.py │ ├── test_discover.py │ ├── test_pipcompilemulti.py │ ├── test_skip_constraint_comments.py │ ├── test_upgrade_feature.py │ ├── test_utils.py │ ├── upgrade/ │ │ ├── base.in │ │ └── base.txt │ ├── upgrade-autoresolve-with-range/ │ │ ├── base.in │ │ ├── base.txt │ │ ├── prod.in │ │ └── prod.txt │ ├── upgrade-autoresolve-with-range-expected/ │ │ ├── base.in │ │ ├── base.txt │ │ ├── prod.in │ │ └── prod.txt │ ├── upgrade-expected/ │ │ ├── base.in │ │ └── base.txt │ ├── upgrade-with-range/ │ │ ├── base.in │ │ └── base.txt │ ├── upgrade-with-range-expected/ │ │ ├── base.in │ │ └── base.txt │ └── utils.py └── tox.ini
SYMBOL INDEX (219 symbols across 48 files)
FILE: pipcompilemulti/actions.py
function recompile (line 16) | def recompile():
function compile_topologically (line 34) | def compile_topologically(env_confs, deduplicator):
FILE: pipcompilemulti/cli_v1.py
function cli (line 21) | def cli(ctx):
function verify (line 32) | def verify(ctx):
function exception_hook (line 43) | def exception_hook(exctype, value, traceback):
function trim_traceback (line 48) | def trim_traceback(traceback):
FILE: pipcompilemulti/cli_v2.py
function cli (line 22) | def cli():
function lock (line 28) | def lock():
function upgrade (line 35) | def upgrade(packages):
function verify (line 42) | def verify(ctx):
function skipper (line 53) | def skipper(func):
function run_configurations (line 68) | def run_configurations(callback, sections_reader, **overrides):
FILE: pipcompilemulti/config.py
function read_config (line 11) | def read_config():
function filter_sections (line 19) | def filter_sections(sections):
function read_sections (line 37) | def read_sections():
function _read_cfg_sections (line 45) | def _read_cfg_sections():
function _read_toml_sections (line 61) | def _read_toml_sections():
function _make_toml_scalar (line 93) | def _make_toml_scalar(v: object) -> str:
function _collect_feature_options (line 105) | def _collect_feature_options() -> Dict[str, ClickOption]:
function parse_value (line 116) | def parse_value(key: str, value: str) -> Union[str, List[str], bool]:
function python_version_matchers (line 128) | def python_version_matchers():
FILE: pipcompilemulti/deduplicate.py
class PackageDeduplicator (line 11) | class PackageDeduplicator:
method __init__ (line 14) | def __init__(self):
method on_discover (line 18) | def on_discover(self, env_confs):
method register_packages_for_env (line 22) | def register_packages_for_env(self, in_path, packages):
method ignored_packages (line 26) | def ignored_packages(self, in_path):
method recursive_refs (line 33) | def recursive_refs(self, in_path):
class IgnoredPackages (line 40) | class IgnoredPackages:
method __init__ (line 48) | def __init__(self, package_versions):
method __getitem__ (line 55) | def __getitem__(self, key):
method __contains__ (line 59) | def __contains__(self, key):
method _make_stem (line 63) | def _make_stem(cls, name):
FILE: pipcompilemulti/dependency.py
class Dependency (line 8) | class Dependency(object): # pylint: disable=too-many-instance-attributes
method __init__ (line 87) | def __init__(self, line):
method serialize (line 123) | def serialize(self):
method without_editable (line 161) | def without_editable(cls, line):
method drop_post (line 173) | def drop_post(self, in_path):
method _adjust_span (line 178) | def _adjust_span(span, matchobj):
FILE: pipcompilemulti/discover.py
function discover (line 15) | def discover(glob_pattern):
function order_by_refs (line 58) | def order_by_refs(envs):
FILE: pipcompilemulti/environment.py
class Environment (line 18) | class Environment(object):
method __init__ (line 24) | def __init__(self, in_path, deduplicator=None):
method maybe_create_lockfile (line 35) | def maybe_create_lockfile(self):
method create_lockfile (line 54) | def create_lockfile(self):
method parse_references (line 83) | def parse_references(cls, filename):
method name (line 103) | def name(self):
method infile (line 108) | def infile(self):
method outfile (line 113) | def outfile(self):
method pin_command (line 118) | def pin_command(self):
method fix_lockfile (line 127) | def fix_lockfile(self):
method concatenated (line 143) | def concatenated(fp):
method parse_sections (line 166) | def parse_sections(self, lines):
method fix_pin (line 187) | def fix_pin(self, section):
method add_references (line 217) | def add_references(self, other_in_paths):
method split_header (line 235) | def split_header(fp):
method replace_header (line 251) | def replace_header(self, header_text):
method _read_infile (line 259) | def _read_infile(self):
method _restore_in_file (line 263) | def _restore_in_file(self, content):
method _inject_sink (line 267) | def _inject_sink(self):
FILE: pipcompilemulti/features/add_hashes.py
class AddHashes (line 49) | class AddHashes(BaseFeature):
method __init__ (line 63) | def __init__(self, controller):
method enabled_in_paths (line 68) | def enabled_in_paths(self):
method on_discover (line 84) | def on_discover(self, env_confs):
method _needs_hashes (line 92) | def _needs_hashes(self, in_path):
method pin_options (line 96) | def pin_options(self, in_path):
FILE: pipcompilemulti/features/annotate_index.py
class AnnotateIndex (line 40) | class AnnotateIndex(ForwardOption):
FILE: pipcompilemulti/features/autoresolve.py
class Autoresolve (line 46) | class Autoresolve(BaseFeature):
method __init__ (line 57) | def __init__(self):
method enabled (line 61) | def enabled(self):
method on_discover (line 65) | def on_discover(self, env_confs):
method sink_path (line 69) | def sink_path(self):
method _find_sink (line 74) | def _find_sink(envs):
FILE: pipcompilemulti/features/backtracking.py
class Backtracking (line 30) | class Backtracking(ForwardOption):
FILE: pipcompilemulti/features/base.py
class ClickOption (line 8) | class ClickOption:
method __init__ (line 11) | def __init__(self,
method decorate (line 26) | def decorate(self, command):
method decorator (line 30) | def decorator(self):
method argument_name (line 45) | def argument_name(self):
class BaseFeature (line 56) | class BaseFeature:
method bind (line 62) | def bind(self, command):
method extract_option (line 66) | def extract_option(self, kwargs):
method value (line 80) | def value(self):
method value (line 85) | def value(self, new_value):
FILE: pipcompilemulti/features/base_dir.py
class BaseDir (line 24) | class BaseDir(BaseFeature):
method path (line 37) | def path(self):
method file_path (line 45) | def file_path(self, file_name):
FILE: pipcompilemulti/features/build_isolation.py
class BuildIsolation (line 25) | class BuildIsolation(ForwardOption):
FILE: pipcompilemulti/features/compatible.py
class Compatible (line 31) | class Compatible(BaseFeature):
method patterns (line 44) | def patterns(self):
method constraint (line 48) | def constraint(self, package_name):
method is_matched (line 61) | def is_matched(self, package_name):
FILE: pipcompilemulti/features/controller.py
class FeaturesController (line 29) | class FeaturesController:
method __init__ (line 33) | def __init__(self):
method bind (line 83) | def bind(self, command):
method pin_command (line 96) | def pin_command(self):
method pin_options (line 117) | def pin_options(self, in_path):
method compose_input_file_path (line 135) | def compose_input_file_path(self, basename):
method compose_output_file_path (line 141) | def compose_output_file_path(self, in_path):
method drop_post (line 145) | def drop_post(self, in_path, package_name, version):
method constraint (line 153) | def constraint(self, package_name):
method on_discover (line 157) | def on_discover(self, env_confs):
method affected (line 169) | def affected(self, in_path):
method included (line 177) | def included(self, in_path):
method get_header_text (line 181) | def get_header_text(self):
method sink_in_path (line 185) | def sink_in_path(self):
method sink_out_path (line 189) | def sink_out_path(self):
method process_dependency_comments (line 199) | def process_dependency_comments(self, comment):
method pipe_arguments (line 203) | def pipe_arguments(self):
FILE: pipcompilemulti/features/emit_find_links.py
class EmitFindLinks (line 42) | class EmitFindLinks(ForwardOption):
FILE: pipcompilemulti/features/emit_trusted_host.py
class EmitTrustedHost (line 39) | class EmitTrustedHost(ForwardOption):
FILE: pipcompilemulti/features/extra_index_url.py
class ExtraIndexUrl (line 47) | class ExtraIndexUrl(BaseFeature):
method pin_options (line 61) | def pin_options(self):
FILE: pipcompilemulti/features/file_extensions.py
class InputExtension (line 24) | class InputExtension(BaseFeature):
method compose_input_file_name (line 36) | def compose_input_file_name(self, base_name):
class OutputExtension (line 45) | class OutputExtension(BaseFeature):
method compose_output_file_path (line 57) | def compose_output_file_path(self, in_path):
FILE: pipcompilemulti/features/forbid_post.py
class ForbidPost (line 28) | class ForbidPost(BaseFeature):
method enabled_envs (line 42) | def enabled_envs(self):
method drop_post (line 47) | def drop_post(version):
method post_forbidden (line 60) | def post_forbidden(self, env_name):
FILE: pipcompilemulti/features/forward.py
class ForwardOption (line 6) | class ForwardOption(BaseFeature):
method enabled (line 15) | def enabled(self):
method pin_options (line 19) | def pin_options(self):
FILE: pipcompilemulti/features/header.py
class CustomHeader (line 31) | class CustomHeader(BaseFeature):
method __init__ (line 42) | def __init__(self):
method text (line 46) | def text(self):
method _read_header_text (line 55) | def _read_header_text(self):
FILE: pipcompilemulti/features/limit_in_paths.py
class LimitInPaths (line 37) | class LimitInPaths(BaseFeature):
method __init__ (line 65) | def __init__(self):
method direct_envs (line 69) | def direct_envs(self):
method on_discover (line 73) | def on_discover(self, env_confs):
method included (line 86) | def included(self, in_path):
FILE: pipcompilemulti/features/live_output.py
class LiveOutput (line 23) | class LiveOutput(BaseFeature):
method pipe_arguments (line 34) | def pipe_arguments(self):
FILE: pipcompilemulti/features/skip_constraint_comments.py
class SkipConstraintComments (line 37) | class SkipConstraintComments(BaseFeature):
method enabled (line 58) | def enabled(self):
method process_dependency_comments (line 62) | def process_dependency_comments(self, comment):
method _drop_sink_comment (line 68) | def _drop_sink_comment(self, comment):
method _collapse_single_via (line 87) | def _collapse_single_via(self, lines):
FILE: pipcompilemulti/features/strip_extras.py
class StripExtras (line 21) | class StripExtras(ForwardOption):
FILE: pipcompilemulti/features/unsafe.py
class AllowUnsafe (line 31) | class AllowUnsafe(ForwardOption):
FILE: pipcompilemulti/features/upgrade.py
class UpgradeAll (line 50) | class UpgradeAll(ForwardOption):
method __init__ (line 62) | def __init__(self, controller):
method enabled (line 66) | def enabled(self):
class UpgradeSelected (line 71) | class UpgradeSelected(BaseFeature):
method __init__ (line 87) | def __init__(self, controller):
method reset (line 91) | def reset(self):
method package_specs (line 96) | def package_specs(self):
method package_names (line 101) | def package_names(self):
method active (line 113) | def active(self):
method pin_options (line 117) | def pin_options(self):
method has_package (line 124) | def has_package(self, in_path, package_name):
method _get_packages (line 128) | def _get_packages(self, in_path):
method _read_packages (line 136) | def _read_packages(outfile):
method _compose_output_file_path (line 148) | def _compose_output_file_path(self, in_path):
method affected (line 151) | def affected(self, in_path):
FILE: pipcompilemulti/features/use_cache.py
class UseCache (line 31) | class UseCache(ForwardOption):
FILE: pipcompilemulti/features/use_uv.py
class UseUV (line 39) | class UseUV(BaseFeature):
method is_available (line 56) | def is_available():
FILE: pipcompilemulti/utils.py
function extract_env_name (line 11) | def extract_env_name(file_path):
function fix_reference_path (line 22) | def fix_reference_path(orig_path, ref_path):
function recursive_refs (line 31) | def recursive_refs(envs, in_path):
function merged_packages (line 52) | def merged_packages(env_packages, names):
function reference_cluster (line 90) | def reference_cluster(envs, in_path):
FILE: pipcompilemulti/verify.py
function verify_environments (line 71) | def verify_environments():
function generate_hash_comment (line 95) | def generate_hash_comment(file_path):
function generate_robust_hash_comment (line 108) | def generate_robust_hash_comment(file_path):
function parse_hash_comment (line 126) | def parse_hash_comment(file_path):
FILE: tests/conftest.py
function runner (line 18) | def runner():
function wipe_options (line 24) | def wipe_options():
function test_data_tmpdir (line 30) | def test_data_tmpdir():
FILE: tests/test_add_hashes.py
class AddHashesTestCase (line 9) | class AddHashesTestCase(unittest.TestCase):
method setUp (line 10) | def setUp(self):
method test_pin_options (line 19) | def test_pin_options(self):
class FakeController (line 24) | class FakeController():
method compose_input_file_path (line 25) | def compose_input_file_path(self, name):
FILE: tests/test_cli_v1.py
function requirements_dir (line 18) | def requirements_dir():
function test_v1_command_exits_with_zero (line 26) | def test_v1_command_exits_with_zero(command, monkeypatch):
function test_v1_verify_exits_with_zero (line 62) | def test_v1_verify_exits_with_zero(monkeypatch):
function _load_tree (line 71) | def _load_tree(root, replace_name=None):
function test_package_upgrade (line 85) | def test_package_upgrade(test_data_tmpdir, name, args):
FILE: tests/test_cli_v2.py
function requirements_dir (line 12) | def requirements_dir():
function test_command_exits_with_zero (line 19) | def test_command_exits_with_zero(command):
FILE: tests/test_config.py
function _write_asset (line 20) | def _write_asset(name: str) -> None:
function in_temp_dir (line 25) | def in_temp_dir() -> Iterator[str]:
function test_load_no_configs (line 37) | def test_load_no_configs() -> None:
function test_load_single_config (line 58) | def test_load_single_config(asset_name: str, expected: List) -> None:
function test_load_two_configs (line 65) | def test_load_two_configs() -> None:
function test_load_ini_with_empty_pyproject (line 77) | def test_load_ini_with_empty_pyproject() -> None:
function test_pyproject_without_section_name (line 85) | def test_pyproject_without_section_name() -> None:
FILE: tests/test_conflicts.py
function test_conflict_detected (line 9) | def test_conflict_detected(test_data_tmpdir, conflict):
FILE: tests/test_deduplicate.py
function test_package_deduplicator_handles_delimiters_normalization (line 5) | def test_package_deduplicator_handles_delimiters_normalization():
FILE: tests/test_dependency.py
function test_parse_package_name (line 8) | def test_parse_package_name():
function test_parse_url_without_postfix (line 27) | def test_parse_url_without_postfix():
function test_parse_url_with_postfix (line 51) | def test_parse_url_with_postfix():
function test_parse_at_url_notation (line 80) | def test_parse_at_url_notation():
function test_sanitize_package_version (line 106) | def test_sanitize_package_version():
function test_parse_sys_platform (line 125) | def test_parse_sys_platform():
FILE: tests/test_discover.py
function test_discover_nested (line 12) | def test_discover_nested():
FILE: tests/test_pipcompilemulti.py
function test_fix_compatible_pin (line 23) | def test_fix_compatible_pin():
function test_no_fix_incompatible_pin (line 31) | def test_no_fix_incompatible_pin():
function test_pin_is_ommitted_if_set_to_ignore (line 38) | def test_pin_is_ommitted_if_set_to_ignore():
function test_post_releases_are_kept_by_default (line 51) | def test_post_releases_are_kept_by_default():
function test_forbid_post_releases (line 59) | def test_forbid_post_releases():
function test_parse_references (line 73) | def test_parse_references(in_path, refs):
function test_split_header (line 82) | def test_split_header():
function test_concatenation (line 93) | def test_concatenation():
function test_parse_hashes_with_comment (line 104) | def test_parse_hashes_with_comment():
function test_parse_hashes_without_comment (line 112) | def test_parse_hashes_without_comment():
function test_serialize_hashes (line 121) | def test_serialize_hashes():
function test_reference_cluster (line 134) | def test_reference_cluster():
function test_parse_vcs_dependencies (line 147) | def test_parse_vcs_dependencies():
function test_merged_packages_raise_for_conflict (line 244) | def test_merged_packages_raise_for_conflict():
function test_fix_pin_detects_version_conflict (line 256) | def test_fix_pin_detects_version_conflict():
FILE: tests/test_skip_constraint_comments.py
class SkipConstraintCommentsAlwayOn (line 20) | class SkipConstraintCommentsAlwayOn(SkipConstraintComments):
function test_drop_sink_example (line 25) | def test_drop_sink_example():
FILE: tests/test_upgrade_feature.py
function test_upgrade_package_disables_upgrade (line 7) | def test_upgrade_package_disables_upgrade():
FILE: tests/test_utils.py
function test_recursive_refs (line 11) | def test_recursive_refs():
FILE: tests/utils.py
function temp_dir (line 10) | def temp_dir():
Condensed preview — 146 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (311K chars).
[
{
"path": ".coveragerc",
"chars": 554,
"preview": "# .coveragerc to control coverage.py\n[run]\nbranch = True\n\n[report]\n# Regexes for lines to exclude from consideration\nexc"
},
{
"path": ".editorconfig",
"chars": 291,
"preview": "# http://editorconfig.org\n\nroot = true\n\n[*]\nindent_style = space\nindent_size = 4\ntrim_trailing_whitespace = true\ninsert_"
},
{
"path": ".github/workflows/pipcompilemulti.yml",
"chars": 1172,
"preview": "name: Update Dependencies\non:\n schedule:\n - cron: '15 15 * * 3'\n workflow_dispatch: {}\n\njobs:\n build:\n runs-on:"
},
{
"path": ".github/workflows/python310-windows.yml",
"chars": 479,
"preview": "name: Python 3.10 Windows\n\non:\n push:\n branches: [ master ]\n pull_request:\n branches: [ master ]\n\njobs:\n build:"
},
{
"path": ".github/workflows/python310.yml",
"chars": 453,
"preview": "name: Python 3.10\n\non:\n push:\n branches: [ master ]\n pull_request:\n branches: [ master ]\n\njobs:\n build:\n run"
},
{
"path": ".github/workflows/python311.yml",
"chars": 476,
"preview": "name: Python 3.11\n\non:\n push:\n branches: [ master ]\n pull_request:\n branches: [ master ]\n\njobs:\n build:\n run"
},
{
"path": ".github/workflows/python312.yml",
"chars": 476,
"preview": "name: Python 3.12\n\non:\n push:\n branches: [ master ]\n pull_request:\n branches: [ master ]\n\njobs:\n build:\n run"
},
{
"path": ".github/workflows/python313.yml",
"chars": 476,
"preview": "name: Python 3.13\n\non:\n push:\n branches: [ master ]\n pull_request:\n branches: [ master ]\n\njobs:\n build:\n run"
},
{
"path": ".github/workflows/python314.yml",
"chars": 476,
"preview": "name: Python 3.14\n\non:\n push:\n branches: [ master ]\n pull_request:\n branches: [ master ]\n\njobs:\n build:\n run"
},
{
"path": ".gitignore",
"chars": 557,
"preview": "*.py[cod]\n/.env/\n/.venv*/\n\n# C extensions\n*.so\n\n# Packages\n*.egg\n*.egg-info\ndist\nbuild\neggs\nparts\nbin\nvar\nsdist\ndevelop-"
},
{
"path": ".pre-commit-config.yaml",
"chars": 265,
"preview": "repos:\n - repo: https://github.com/peterdemin/pip-compile-multi\n rev: v2.6.2\n hooks:\n - id: pip-compile-mult"
},
{
"path": ".pre-commit-hooks.yaml",
"chars": 226,
"preview": "- id: pip-compile-multi-verify\n name: pip-compile-multi verify\n language: python\n entry: pip-compile-multi verify\n f"
},
{
"path": ".pylintrc",
"chars": 14261,
"preview": "[MASTER]\n\n# A comma-separated list of package or module names from where C extensions may\n# be loaded. Extensions are lo"
},
{
"path": ".readthedocs.yml",
"chars": 660,
"preview": "# .readthedocs.yml\n# Read the Docs configuration file\n# See https://docs.readthedocs.io/en/stable/config-file/v2.html fo"
},
{
"path": ".travis.yml",
"chars": 1118,
"preview": "sudo: false\nlanguage: python\n\npython:\n - \"3.6\"\n - \"3.7\"\n - \"3.8\"\n\n\nmatrix:\n include:\n - os: linux\n python: 3"
},
{
"path": "AUTHORS.rst",
"chars": 156,
"preview": "=======\nCredits\n=======\n\nDevelopment Lead\n----------------\n\n* Peter Demin <peterdemin@gmail.com>\n\nContributors\n---------"
},
{
"path": "CONTRIBUTING.rst",
"chars": 3246,
"preview": "============\nContributing\n============\n\nContributions are welcome, and they are greatly appreciated! Every\nlittle bit he"
},
{
"path": "Dockerfile",
"chars": 89,
"preview": "FROM python:3.10\n\nRUN apt-get update && apt-get install -y build-essential\n\nWORKDIR /pcm\n"
},
{
"path": "HISTORY.rst",
"chars": 10888,
"preview": "History\n=======\n\n3.3.1 (2025-04-19)\n------------------\n\n* Add support for `--emit-find-links/--no-emit-find-links`.\n (I"
},
{
"path": "LICENSE.txt",
"chars": 1069,
"preview": "MIT License\n\nCopyright (c) [year] [fullname]\n\nPermission is hereby granted, free of charge, to any person obtaining a co"
},
{
"path": "MANIFEST.in",
"chars": 110,
"preview": "include AUTHORS.rst\ninclude CONTRIBUTING.rst\ninclude HISTORY.rst\ninclude LICENSE\ninclude requirements/base.in\n"
},
{
"path": "Makefile",
"chars": 1172,
"preview": ".PHONY: virtual_env_set\nvirtual_env_set:\nifndef VIRTUAL_ENV\n\t$(error VIRTUAL_ENV not set)\nendif\n\n### DEPENDENCIES ###\n.P"
},
{
"path": "README.rst",
"chars": 2625,
"preview": "=================\npip-compile-multi\n=================\n\n.. image:: https://badge.fury.io/py/pip-compile-multi.png\n :ta"
},
{
"path": "docs/Makefile",
"chars": 580,
"preview": "# Minimal makefile for Sphinx documentation\n#\n\n# You can set these variables from the command line.\nSPHINXOPTS =\nSPHI"
},
{
"path": "docs/_static/custom.css",
"chars": 125,
"preview": "body {\n font-family: 'minion pro', 'bell mt', Georgia, 'Hiragino Mincho Pro', serif;\n}\n\ndiv.document {\n width: 71e"
},
{
"path": "docs/afterword.rst",
"chars": 301,
"preview": "Have fun!\n---------\n\nNow that occasional backward incompatible dependency release can't ruin your day,\nyou can **spread "
},
{
"path": "docs/boilerplate.rst",
"chars": 2464,
"preview": "Bonus: boilerplate to put in project's README\n---------------------------------------------\n\nNice way of introducing dep"
},
{
"path": "docs/conf.py",
"chars": 5942,
"preview": "\"\"\"\nConfiguration file for the Sphinx documentation builder.\n\nThis file does only contain a selection of the most common"
},
{
"path": "docs/features.rst",
"chars": 1670,
"preview": "Features\n--------\n\n``pip-compile-multi`` supports many options to customize compilation.\nEach option can be specified in"
},
{
"path": "docs/history.rst",
"chars": 28,
"preview": ".. include:: ../HISTORY.rst\n"
},
{
"path": "docs/index.rst",
"chars": 1337,
"preview": ".. pip-compile-multi documentation master file, created by\n sphinx-quickstart on Thu Aug 8 15:36:37 2019.\n You can "
},
{
"path": "docs/installation.rst",
"chars": 3727,
"preview": "Installation\n------------\n\nPython Version\n==============\n\nWe recommend using the latest version of Python 3.\nPip-compile"
},
{
"path": "docs/migration.rst",
"chars": 2853,
"preview": "How to start using pip-compile-multi on existing project\n--------------------------------------------------------\n\nIniti"
},
{
"path": "docs/precommit.rst",
"chars": 427,
"preview": "Verify as pre-commit hook\n=========================\n\nTo verify that ``pip-compile-multi`` has been run after changing ``"
},
{
"path": "docs/why.rst",
"chars": 7813,
"preview": "Motivation\n----------\n\nI will start from the very basics of dependency management and will go very slow,\nso if you feel "
},
{
"path": "how-to.md",
"chars": 6075,
"preview": "# Managing dependencies in multi-platform Python project\n\nI'm going to describe the setup I made for my Python\n[project]"
},
{
"path": "nested/base.in",
"chars": 21,
"preview": "-r subproject/sub.in\n"
},
{
"path": "nested/base.txt",
"chars": 167,
"preview": "# SHA1:6a7fb5b91341e12583307fb55069246d43096b44\n#\n# This file is autogenerated by pip-compile-multi\n# To update, run:\n#\n"
},
{
"path": "nested/diamond.in",
"chars": 33,
"preview": "-r base.in\n-r subproject/base.in\n"
},
{
"path": "nested/diamond.txt",
"chars": 180,
"preview": "# SHA1:1301840447a3dce149692692b41974b7668ba22e\n#\n# This file is autogenerated by pip-compile-multi\n# To update, run:\n#\n"
},
{
"path": "nested/subproject/base.in",
"chars": 12,
"preview": "-r ../up.in\n"
},
{
"path": "nested/subproject/base.txt",
"chars": 158,
"preview": "# SHA1:e975ad2caba20cf1afaea2cd3626c6c566edfacf\n#\n# This file is autogenerated by pip-compile-multi\n# To update, run:\n#\n"
},
{
"path": "nested/subproject/sub.in",
"chars": 11,
"preview": "-r base.in\n"
},
{
"path": "nested/subproject/sub.txt",
"chars": 157,
"preview": "# SHA1:a87fd594461015e819a1f468943967d12880b85d\n#\n# This file is autogenerated by pip-compile-multi\n# To update, run:\n#\n"
},
{
"path": "nested/up.in",
"chars": 4,
"preview": "six\n"
},
{
"path": "nested/up.txt",
"chars": 183,
"preview": "# SHA1:bec9703f7a456cd2b4ab5fb3220ae016e3e394e3\n#\n# This file is autogenerated by pip-compile-multi\n# To update, run:\n#\n"
},
{
"path": "pipcompilemulti/__init__.py",
"chars": 126,
"preview": "\"\"\"Pip compile multi aka requirements\"\"\"\n\n__author__ = 'Peter Demin'\n__email__ = 'peterdemin@gmail.com'\n__version__ = '3"
},
{
"path": "pipcompilemulti/actions.py",
"chars": 1424,
"preview": "#!/usr/bin/env python\n\"\"\"High level actions to be called from CLI\"\"\"\n\nimport logging\n\nfrom .discover import discover\nfro"
},
{
"path": "pipcompilemulti/cli_v1.py",
"chars": 1586,
"preview": "\"\"\"The current stable version of command line interface.\"\"\"\n\nimport os\nimport sys\nimport logging\nfrom traceback import p"
},
{
"path": "pipcompilemulti/cli_v2.py",
"chars": 2687,
"preview": "\"\"\"Human-friendly interface to pip-compile-multi\"\"\"\nimport functools\nimport logging\n\nimport click\n\nfrom .actions import "
},
{
"path": "pipcompilemulti/config.py",
"chars": 4096,
"preview": "\"\"\"Get tasks options from INI file\"\"\"\nimport sys\nimport os\nimport configparser\nfrom typing import Dict, List, Union\nfrom"
},
{
"path": "pipcompilemulti/deduplicate.py",
"chars": 1997,
"preview": "\"\"\"Remove packages included in referenced environments.\"\"\"\n\nimport logging\n\nfrom pipcompilemulti.utils import recursive_"
},
{
"path": "pipcompilemulti/dependency.py",
"chars": 6218,
"preview": "\"\"\"Dependency class\"\"\"\n\nimport re\n\nfrom .features import FEATURES\n\n\nclass Dependency(object): # pylint: disable=too-man"
},
{
"path": "pipcompilemulti/discover.py",
"chars": 2210,
"preview": "\"\"\"Environment discovery\"\"\"\n\nimport os\nimport glob\nfrom collections import deque\n\nfrom toposort import toposort_flatten\n"
},
{
"path": "pipcompilemulti/environment.py",
"chars": 9516,
"preview": "\"\"\"Environment class\"\"\"\n\nimport os\nimport re\nimport sys\nimport logging\nimport subprocess\n\nfrom .dependency import Depend"
},
{
"path": "pipcompilemulti/features/__init__.py",
"chars": 105,
"preview": "\"\"\"Features as modules.\"\"\"\n\nfrom .controller import FeaturesController\n\n\nFEATURES = FeaturesController()\n"
},
{
"path": "pipcompilemulti/features/add_hashes.py",
"chars": 3319,
"preview": "\"\"\"\nGenerate hashes\n===============\n\nPut package hash after pinned version for additional security.\nFormat for this opti"
},
{
"path": "pipcompilemulti/features/annotate_index.py",
"chars": 1462,
"preview": "\"\"\"\n.. _annotate_index:\n\nAdd index URL annotation\n========================\n\nControl addition of ``--index-url`` options "
},
{
"path": "pipcompilemulti/features/autoresolve.py",
"chars": 3425,
"preview": "\"\"\"\n.. _autoresolve:\n\nAutoresolve cross-file conflicts\n================================\n\nCompile requirements file, that"
},
{
"path": "pipcompilemulti/features/backtracking.py",
"chars": 1347,
"preview": "\"\"\"\nBacktracking resolver\n=====================\n\nPip has an option to enable `backtracking`_ conflict resolution logic,\n"
},
{
"path": "pipcompilemulti/features/base.py",
"chars": 2624,
"preview": "\"\"\"Common functionality for features activated by command line option.\"\"\"\n\nimport click\n\nfrom ..options import OPTIONS\n\n"
},
{
"path": "pipcompilemulti/features/base_dir.py",
"chars": 1300,
"preview": "\"\"\"\nRequirements Directory\n======================\n\nWhile it's a common practice to put requirements files inside ``requi"
},
{
"path": "pipcompilemulti/features/build_isolation.py",
"chars": 1249,
"preview": "\"\"\"\nBuild isolation\n===============\n\nAllows disabling build isolation through the equivalent ``pip-compile`` flag.\nBuild"
},
{
"path": "pipcompilemulti/features/compatible.py",
"chars": 1976,
"preview": "\"\"\"\nCompatible Releases\n===================\n\n`PEP-440`_ describes compatible release operator ``~=``.\nSometimes it's use"
},
{
"path": "pipcompilemulti/features/controller.py",
"chars": 7704,
"preview": "\"\"\"Aggregate all features in a single controller.\"\"\"\n\nimport os\nfrom functools import wraps\n\nfrom .add_hashes import Add"
},
{
"path": "pipcompilemulti/features/emit_find_links.py",
"chars": 1562,
"preview": "\"\"\"\n.. _emit_find_links:\n\nAdd find-links annotation\n=========================\n\nControl addition of ``--find-links`` opti"
},
{
"path": "pipcompilemulti/features/emit_trusted_host.py",
"chars": 1534,
"preview": "\"\"\"\n.. _emit_trusted_host:\n\nAdd trusted host annotation\n===========================\n\nControl addition of trusted hosts f"
},
{
"path": "pipcompilemulti/features/extra_index_url.py",
"chars": 2348,
"preview": "\"\"\"\n.. _extra_index_url:\n\nAdd additional index URL to search\n==================================\n\nPip accepts URLs for ad"
},
{
"path": "pipcompilemulti/features/file_extensions.py",
"chars": 1685,
"preview": "\"\"\"\nRequirements Files Extensions\n=============================\n\nBy default ``pip-compile-multi`` compiles ``*.txt`` fro"
},
{
"path": "pipcompilemulti/features/forbid_post.py",
"chars": 1814,
"preview": "\"\"\"\nForbid .postX release\n=====================\n\n``pip-compile-multi`` can remove ``.postX`` part of dependencies versio"
},
{
"path": "pipcompilemulti/features/forward.py",
"chars": 608,
"preview": "\"\"\"Base feature for forwarding pip-tools options.\"\"\"\n\nfrom .base import BaseFeature\n\n\nclass ForwardOption(BaseFeature):\n"
},
{
"path": "pipcompilemulti/features/header.py",
"chars": 1344,
"preview": "\"\"\"\nCustom Header\n=============\n\n``pip-compile-multi`` adds a brief header into generated files.\nOverride it with\n\n.. co"
},
{
"path": "pipcompilemulti/features/limit_in_paths.py",
"chars": 2779,
"preview": "\"\"\"\n.. _limit-in-files:\n\nLimit input files\n=================\n\nBy default ``pip-compile-multi`` compiles all ``.in`` file"
},
{
"path": "pipcompilemulti/features/live_output.py",
"chars": 993,
"preview": "\"\"\"\nLive output\n===========\n\nPrint debug output from pip-compile live.\nIf the option is disabled (by default) the debug "
},
{
"path": "pipcompilemulti/features/skip_constraint_comments.py",
"chars": 3188,
"preview": "\"\"\"\nSkip constraints in comments of output files\n============================================\n\nWhen input files contain "
},
{
"path": "pipcompilemulti/features/strip_extras.py",
"chars": 719,
"preview": "\"\"\"\nStrip extras\n============\n\nInstructs ``pip-compile`` to attempt to omit extras in transient dependencies,\nwhile assu"
},
{
"path": "pipcompilemulti/features/unsafe.py",
"chars": 1389,
"preview": "\"\"\"\nAllow Unsafe Packages\n=====================\n\nIf your project depends on packages that include ``setuptools``\nor othe"
},
{
"path": "pipcompilemulti/features/upgrade.py",
"chars": 4848,
"preview": "\"\"\"\nDisable upgrades\n================\n\nWhen new dependencies are added it's tempting to keep everything else the same.\nT"
},
{
"path": "pipcompilemulti/features/use_cache.py",
"chars": 1101,
"preview": "\"\"\"\nUse Cache\n=========\n\nBy default ``pip-compile-multi`` executes ``pip-compile`` without ``--rebuild`` flag.\n``--rebui"
},
{
"path": "pipcompilemulti/features/use_uv.py",
"chars": 1472,
"preview": "\"\"\"\nEnable UV\n=========\n\nUV is an extremely fast Python package installer and resolver written in Rust.\nWhen enabled, pi"
},
{
"path": "pipcompilemulti/options.py",
"chars": 69,
"preview": "\"\"\"Global dictionary holding configuration options.\"\"\"\n\nOPTIONS = {}\n"
},
{
"path": "pipcompilemulti/utils.py",
"chars": 3439,
"preview": "\"\"\"Functional utilities for lists and dicts manipulation.\"\"\"\n\nimport os\nimport logging\nimport itertools\n\n\nlogger = loggi"
},
{
"path": "pipcompilemulti/verify.py",
"chars": 4827,
"preview": "\"\"\"\nCheck that ``pip-compile-multi`` was run after changes in ``.in`` file\n============================================="
},
{
"path": "requirements/base.hash",
"chars": 6422,
"preview": "# SHA1:9afdc18a682b2ce622499600e46778b76e9a9850\n#\n# This file was generated by pip-compile-multi.\n# To update, run:\n#\n# "
},
{
"path": "requirements/base.in",
"chars": 32,
"preview": "click\npip-tools>=6.8.0\ntoposort\n"
},
{
"path": "requirements/base.txt",
"chars": 631,
"preview": "# SHA1:32737333f763ceffd22b7fcb76fbe62a538296fa\n#\n# This file was generated by pip-compile-multi.\n# To update, run:\n#\n# "
},
{
"path": "requirements/local.hash",
"chars": 54825,
"preview": "# SHA1:08b9c753a9730ea1f06fb8c80c0a3fb45797897c\n#\n# This file was generated by pip-compile-multi.\n# To update, run:\n#\n# "
},
{
"path": "requirements/local.in",
"chars": 180,
"preview": "-r test.in\n\ntox\ntwine\nwheel\nbump2version\nflake8\ncollective.checkdocs\npygments\npre-commit\npipdeptree\npylint\npep8-naming\np"
},
{
"path": "requirements/local.txt",
"chars": 3400,
"preview": "# SHA1:e617f45ac71d5408e21d0e8432038f3deb4c1e2d\n#\n# This file was generated by pip-compile-multi.\n# To update, run:\n#\n# "
},
{
"path": "requirements/test.hash",
"chars": 13087,
"preview": "# SHA1:1ecb652424b040b181164313dd881f4f3ee1477f\n#\n# This file was generated by pip-compile-multi.\n# To update, run:\n#\n# "
},
{
"path": "requirements/test.in",
"chars": 99,
"preview": "-r base.in\n\npytest\npytest-cov\nmock\nmore-itertools\nuv>=0.1.0 # Astral's faster dependency resolver\n"
},
{
"path": "requirements/test.txt",
"chars": 688,
"preview": "# SHA1:93b27adc99cd695162fe4433a52e3d0639d8b812\n#\n# This file was generated by pip-compile-multi.\n# To update, run:\n#\n# "
},
{
"path": "requirements/testwin.hash",
"chars": 688,
"preview": "# SHA1:a053c9dcc1ff686538ecf3dda5723ad59a09a7cd\n#\n# This file is autogenerated by pip-compile-multi\n# To update, run:\n#\n"
},
{
"path": "requirements/testwin.in",
"chars": 34,
"preview": "-r test.in\n\ncolorama\natomicwrites\n"
},
{
"path": "requirements/testwin.txt",
"chars": 364,
"preview": "# SHA1:ab0b9abf8863d5dced78a6f5664a664d8d2488cd\n#\n# This file is autogenerated by pip-compile-multi\n# To update, run:\n#\n"
},
{
"path": "setup.cfg",
"chars": 774,
"preview": "[bumpversion]\ncurrent_version = 3.3.1\ncommit = True\ntag = True\n\n[wheel]\nuniversal = 1\n\n[bdist_wheel]\nuniversal = 1\n\n[bum"
},
{
"path": "setup.py",
"chars": 2249,
"preview": "\"\"\"Package configuration\"\"\"\n\nimport os\nfrom setuptools import setup, find_packages\n\n\nVERSION = \"3.3.1\"\n\n\nREADME = \"\"\"\npi"
},
{
"path": "tests/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "tests/configs/pyproject.toml",
"chars": 84,
"preview": "[tool.requirements.one]\nuv=true\n\n[tool.requirements.two]\ngenerate_hashes=\"file.txt\"\n"
},
{
"path": "tests/configs/requirements.ini",
"chars": 62,
"preview": "[requirements]\nallow_unsafe = True\nuse_cache = True\nuv = True\n"
},
{
"path": "tests/configs/setup.cfg",
"chars": 43,
"preview": "[requirements:Python 3]\nautoresolve = True\n"
},
{
"path": "tests/conflicting-in-merge/base1.in",
"chars": 10,
"preview": "pytz<2018\n"
},
{
"path": "tests/conflicting-in-merge/base1.txt",
"chars": 207,
"preview": "# SHA1:07973b0a74462beb75c2bb65cae3db0b53237103\n#\n# This file is autogenerated by pip-compile-multi\n# To update, run:\n#\n"
},
{
"path": "tests/conflicting-in-merge/base2.in",
"chars": 5,
"preview": "pytz\n"
},
{
"path": "tests/conflicting-in-merge/base2.txt",
"chars": 207,
"preview": "# SHA1:fe99a8ac13be13203ced26880be5ac493ae359ab\n#\n# This file is autogenerated by pip-compile-multi\n# To update, run:\n#\n"
},
{
"path": "tests/conflicting-in-merge/together.in",
"chars": 24,
"preview": "-r base1.in\n-r base2.in\n"
},
{
"path": "tests/conflicting-in-merge/together.txt",
"chars": 171,
"preview": "# SHA1:08d9c33c7a4a6c4972d515c176318068baf08856\n#\n# This file is autogenerated by pip-compile-multi\n# To update, run:\n#\n"
},
{
"path": "tests/conflicting-in-ref/base1.in",
"chars": 15,
"preview": "Django==5.2.12\n"
},
{
"path": "tests/conflicting-in-ref/base1.txt",
"chars": 345,
"preview": "# SHA1:b6c95e2580609775eb2b7302ad50c369810daef6\n#\n# This file is autogenerated by pip-compile-multi\n# To update, run:\n#\n"
},
{
"path": "tests/conflicting-in-ref/base2.in",
"chars": 26,
"preview": "-r base1.in\n\nopal==0.21.0\n"
},
{
"path": "tests/conflicting-in-ref/base2.txt",
"chars": 1705,
"preview": "amqp==5.0.9\n # via kombu\nbilliard==3.6.4.0\n # via celery\ncached-property==1.5.2\n # via kombu\ncelery==5.0.2\n "
},
{
"path": "tests/conftest.py",
"chars": 941,
"preview": "\"\"\"Pytest configuration.\"\"\"\n\nimport shutil\nimport pathlib\nimport os.path\nimport tempfile\nimport contextlib\n\nimport pytes"
},
{
"path": "tests/sys_platform/base.in",
"chars": 349,
"preview": "torch==1.12.0; sys_platform=='darwin' and python_version > '3.6' # Mac OS\ntorch==1.11.0; sys_platform=='linux' # Linux\np"
},
{
"path": "tests/sys_platform/base.txt",
"chars": 4509,
"preview": "# SHA1:854aa2388eff8ba43b94aee23c77726c4fa6a05d\n#\n# This file is autogenerated by pip-compile-multi\n# To update, run:\n#\n"
},
{
"path": "tests/sys_platform/test.in",
"chars": 44,
"preview": "-r base.in\n\ntorchmetrics # Metrics comment\n"
},
{
"path": "tests/sys_platform/test.txt",
"chars": 2486,
"preview": "# SHA1:97dcc9f96f311da386d4460771c4ecb4a47dfe6d\n#\n# This file is autogenerated by pip-compile-multi\n# To update, run:\n#\n"
},
{
"path": "tests/test_add_hashes.py",
"chars": 881,
"preview": "# pylint: disable=too-few-public-methods,missing-module-docstring\n# pylint: disable=missing-function-docstring,missing-c"
},
{
"path": "tests/test_cli_v1.py",
"chars": 3220,
"preview": "\"\"\"End to end tests for CLI v1\"\"\"\n\nfrom pathlib import Path\n\nimport pytest\nfrom click.testing import CliRunner\n\nfrom pip"
},
{
"path": "tests/test_cli_v2.py",
"chars": 566,
"preview": "\"\"\"End to end tests for CLI v2\"\"\"\n\nfrom click.testing import CliRunner\n\nimport pytest\n\nfrom pipcompilemulti.cli_v2 impor"
},
{
"path": "tests/test_config.py",
"chars": 2451,
"preview": "\"\"\"Tests for config loading for CLI v2\"\"\"\nimport os\nimport pathlib\nimport shutil\nimport tempfile\nfrom typing import Iter"
},
{
"path": "tests/test_conflicts.py",
"chars": 797,
"preview": "\"\"\"End to end tests checking conflicts detection\"\"\"\n\nimport pytest\nfrom click.testing import CliRunner\nfrom pipcompilemu"
},
{
"path": "tests/test_deduplicate.py",
"chars": 720,
"preview": "\"\"\"Package name deduplication tests\"\"\"\nfrom pipcompilemulti.deduplicate import PackageDeduplicator\n\n\ndef test_package_de"
},
{
"path": "tests/test_dependency.py",
"chars": 4541,
"preview": "\"\"\"Dependency parser tests\"\"\"\n\nfrom pipcompilemulti.dependency import Dependency\nfrom pipcompilemulti.features import FE"
},
{
"path": "tests/test_discover.py",
"chars": 1187,
"preview": "\"\"\"Environment discovery tests.\"\"\"\n\nimport os\nimport sys\n\nimport pytest\n\nfrom pipcompilemulti.discover import discover\n\n"
},
{
"path": "tests/test_pipcompilemulti.py",
"chars": 10478,
"preview": "\"\"\"Tests for pip-compile-multi\"\"\"\n\nimport os\ntry:\n from unittest import mock\nexcept ImportError:\n import mock\n\nimp"
},
{
"path": "tests/test_skip_constraint_comments.py",
"chars": 749,
"preview": "\"\"\"Tests for Skip constraints in comments of output files feature.\"\"\"\nfrom textwrap import dedent\nfrom pipcompilemulti.f"
},
{
"path": "tests/test_upgrade_feature.py",
"chars": 367,
"preview": "\"\"\"Test upgrade feature.\"\"\"\n\nfrom pipcompilemulti.features import FEATURES\nfrom pipcompilemulti.options import OPTIONS\n\n"
},
{
"path": "tests/test_utils.py",
"chars": 775,
"preview": "\"\"\"Utils tests.\"\"\"\n\nimport sys\n\nimport pytest\n\nfrom pipcompilemulti.utils import recursive_refs\n\n\n@pytest.mark.skipif(sy"
},
{
"path": "tests/upgrade/base.in",
"chars": 110,
"preview": "# Set an upper bound for test stability\nmarkupsafe<2.1.3\n\n# Another package which shouldn't be upgraded\nwheel\n"
},
{
"path": "tests/upgrade/base.txt",
"chars": 248,
"preview": "# SHA1:361e74487f9a6061b75ff3372ad729f74dcfd875\n#\n# This file was generated by pip-compile-multi.\n# To update, run:\n#\n# "
},
{
"path": "tests/upgrade-autoresolve-with-range/base.in",
"chars": 11,
"preview": "markupsafe\n"
},
{
"path": "tests/upgrade-autoresolve-with-range/base.txt",
"chars": 286,
"preview": "# SHA1:2f781f20c9698050fd9afac72c0364db5deacb05\n#\n# This file is autogenerated by pip-compile-multi\n# To update, run:\n#\n"
},
{
"path": "tests/upgrade-autoresolve-with-range/prod.in",
"chars": 11,
"preview": "-r base.in\n"
},
{
"path": "tests/upgrade-autoresolve-with-range/prod.txt",
"chars": 157,
"preview": "# SHA1:a87fd594461015e819a1f468943967d12880b85d\n#\n# This file is autogenerated by pip-compile-multi\n# To update, run:\n#\n"
},
{
"path": "tests/upgrade-autoresolve-with-range-expected/base.in",
"chars": 11,
"preview": "markupsafe\n"
},
{
"path": "tests/upgrade-autoresolve-with-range-expected/base.txt",
"chars": 231,
"preview": "# SHA1:2f781f20c9698050fd9afac72c0364db5deacb05\n#\n# This file was generated by pip-compile-multi.\n# To update, run:\n#\n# "
},
{
"path": "tests/upgrade-autoresolve-with-range-expected/prod.in",
"chars": 11,
"preview": "-r base.in\n"
},
{
"path": "tests/upgrade-autoresolve-with-range-expected/prod.txt",
"chars": 158,
"preview": "# SHA1:a87fd594461015e819a1f468943967d12880b85d\n#\n# This file was generated by pip-compile-multi.\n# To update, run:\n#\n# "
},
{
"path": "tests/upgrade-expected/base.in",
"chars": 110,
"preview": "# Set an upper bound for test stability\nmarkupsafe<2.1.3\n\n# Another package which shouldn't be upgraded\nwheel\n"
},
{
"path": "tests/upgrade-expected/base.txt",
"chars": 266,
"preview": "# SHA1:361e74487f9a6061b75ff3372ad729f74dcfd875\n#\n# This file was generated by pip-compile-multi.\n# To update, run:\n#\n# "
},
{
"path": "tests/upgrade-with-range/base.in",
"chars": 11,
"preview": "markupsafe\n"
},
{
"path": "tests/upgrade-with-range/base.txt",
"chars": 210,
"preview": "# SHA1:2f781f20c9698050fd9afac72c0364db5deacb05\n#\n# This file was generated by pip-compile-multi.\n# To update, run:\n#\n# "
},
{
"path": "tests/upgrade-with-range-expected/base.in",
"chars": 11,
"preview": "markupsafe\n"
},
{
"path": "tests/upgrade-with-range-expected/base.txt",
"chars": 219,
"preview": "# SHA1:2f781f20c9698050fd9afac72c0364db5deacb05\n#\n# This file was generated by pip-compile-multi.\n# To update, run:\n#\n# "
},
{
"path": "tests/utils.py",
"chars": 473,
"preview": "\"\"\"Test utilities.\"\"\"\n\nimport os\nimport tempfile\nimport shutil\nimport contextlib\n\n\n@contextlib.contextmanager\ndef temp_d"
},
{
"path": "tox.ini",
"chars": 1246,
"preview": "[tox]\nenvlist = py{39,310,311,312,313,314,py3}-{linux,windows,darwin}, lint, checkdocs, verify\nskip_missing_interpreters"
}
]
About this extraction
This page contains the full source code of the peterdemin/pip-compile-multi GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 146 files (284.1 KB), approximately 96.9k tokens, and a symbol index with 219 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.