Showing preview only (450K chars total). Download the full file or copy to clipboard to get everything.
Repository: openai/chz
Branch: main
Commit: b01c082aa50b
Files: 48
Total size: 431.1 KB
Directory structure:
gitextract_3v4reuym/
├── .gitignore
├── CHANGELOG.md
├── LICENSE
├── README.md
├── chz/
│ ├── __init__.py
│ ├── blueprint/
│ │ ├── __init__.py
│ │ ├── _argmap.py
│ │ ├── _argv.py
│ │ ├── _blueprint.py
│ │ ├── _entrypoint.py
│ │ ├── _lazy.py
│ │ └── _wildcard.py
│ ├── data_model.py
│ ├── factories.py
│ ├── field.py
│ ├── mungers.py
│ ├── py.typed
│ ├── tiepin.py
│ ├── universal.py
│ ├── util.py
│ └── validators.py
├── docs/
│ ├── 01_quickstart.md
│ ├── 02_object_model.md
│ ├── 03_validation.md
│ ├── 04_command_line.md
│ ├── 05_blueprint.md
│ ├── 06_serialisation.md
│ ├── 21_post_init.md
│ ├── 22_field_api.md
│ ├── 91_philosophy.md
│ ├── 92_alternatives.md
│ └── 93_testimonials.md
├── pyproject.toml
└── tests/
├── test_blueprint.py
├── test_blueprint_cast.py
├── test_blueprint_errors.py
├── test_blueprint_meta_factory.py
├── test_blueprint_methods.py
├── test_blueprint_reference.py
├── test_blueprint_root_polymorphism.py
├── test_blueprint_unit.py
├── test_blueprint_variadic.py
├── test_data_model.py
├── test_factories.py
├── test_munge.py
├── test_tiepin.py
├── test_todo.py
└── test_validate.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
__pycache__/
*.py[cod]
.DS_Store
.env
.venv
env/
venv/
build/
dist/
*.egg-info/
.tox/
.mypy_cache/
================================================
FILE: CHANGELOG.md
================================================
# Changelog
## November 2025
- fix most tests on Python 3.14
- support cast to `datetime.datetime`
- improve `is_subtype` for `TypedDict`s
- add `Computed` reference type, thanks sfitzgerald!
- support int dict keys in blueprint, thanks hessam!
- fix subparam mutation in the "template thing", thanks tz!
- improve docs, thanks awei!
- require newer `typing-extensions`
## September 2025
- add `dispatch_entrypoint`
- several changes to optimise argmap lookups by collapsing and consolidating layers
- this is >10x speedup for some use cases
- always print additional diagnostics for extraneous args, thanks camillo!
- add `skip_default` arg to `beta_to_blueprint_values`, thanks charlieb!
- add special casing for tuples in `beta_argv_arg_to_string`, thanks tz!
- testing improvements
## August 2025
- mention the value of the closest ancestor for extraneous args to help with polymorphism confusion
- improve extraneous arg error message
- add `exclude` param to `asdict`, thanks andrey!
- fix subtype check in the "template thing", thanks tz!
- some cleanup of the "template thing"
## July 2025
- changes to add the "template thing" to blueprint, thanks xintao!
- this feature is not available in the open source version and I plan to attempt to remove it from the internal version
- differentiate between untyped and zero length tuple in sequence param collection, thanks elwong!
- fix `beta_argv_arg_to_string` behaviour for list elements that are strings containing commas
## June 2025
- better error if annotation eval fails, thanks jelle!
- add `ge` and `le` validators, thanks cassirer!
- special casing to make `beta_argv_arg_to_string` handle dicts, thanks yjiao!
## May 2025
- error for duplicate class when name is ambiguous
- fix defaulting special case for nested args
- add `chz.traverse`, thanks hessam!
- better handling of type variables and meta factory casting
- special casing to make `beta_argv_arg_to_string` involving lists more compact
- improve `freeze_dict` munger static typing for optionals, thanks camillo!
- add `include_type` param to `asdict`, thanks wenda and andrei!
- internal refactoring
## March 2025
Improvements:
- add "universal CLI" via `python -m chz.universal`
- add `shallow` param to `asdict` to prevent deep copying, thanks wenda!
- look at `builtins` and `__main__` to find object factories
- support `*args` and `**kwargs` collection in blueprint
- support type variables in `is_subtype`
- fix variadics that match wildcards in more than one literal location
- fix blueprint apply to subpath with empty key, thanks hessam!
- suppert converter argument in field, thanks camillo!
- refactor param collection in blueprint
- various docs improvements, thanks andrey, csh, mtli!
Error messages:
- better error for a value with subparams specified
- improve error for blueprint type mismatch
- fix bug in `simplistic_type_of_value`
- include Python's native suggestions for `AttributeError` in blueprint attribute access, thanks yifan!
## February 2025
Improvements:
- revamp the docs
- improve casting for callables
- blindly trust explicit inheritance from protocol
- record `meta_factory_value` for non castable factory
- add `__eq__` to `castable`
- fix quoting in the `beta_blueprint_to_argv` thing
- expose the `beta_argv_arg_to_string` thing
Performance:
- add an optimisation when constructing large variadics for 6x speedup on some workloads
- rewrite the `beta_blueprint_to_argv` thing so it's now 40x faster
- make it easier to reuse `MakeResult` to save repeated blueprint make
- lru cache `inspect.getmembers_static` to speed up repeated construction
- refactoring to make optimisation easier
Error messages:
- show the full path more often when errors occur during blueprint construction, thanks mlim and gross!
- add error for case where you have duplicate classes due to `__main__` confusion
- improve error message when constructing ambiguous or mistyped callables
- minor improvements to error messages
## January 2025
Improvements:
- add basic support for functools.partial in blueprints
- allow parametrising the entrypoint in chz blueprints. this allows for a "universal" cli
- rewrie `beta_to_blueprint_values` to better support nesting and polymorphism
- improve interaction between type checking and munger
- ignore self references more consistently when there is a default value available
- better error when self references are missing a default value
- add `freeze_dict` munger, thanks camillo!
- use post init field value in hash, thanks camillo!
- prevent parameterisation of enum classes
- colourise and improve alignment of `--help` output
- various refactoring
Typing improvements:
- implement callable subtyping (especially useful for substructural typing)
- improve `is_subtype_instance` of protocols
- improve `is_subtype_instance` of None, thanks tongzhou!
- improve `is_subtype` handling of unions, thanks tongzhou!
- improve `is_subtype` handling of literals and `types.NoneType`
- better signature subtyping
- better casting for dict
## December 2024
Improvements:
- expand the error for wildcard matching variadic defaults, preventing a footgun
- optimise blueprint construction with large variadics, making a use case 2.7x faster
- add support for protocol subtyping for vitchyr use case
- pass field metadata through blueprint, for use in custom tools
- allow custom root for consistency in tree, thanks ignasi!
- fix `beta_blueprint_to_argv` with None args, thanks tongzhou!
- add test for unspecified `type(None)` trick to avoid instantiating defaulted class
- simplify some `meta_factory` logic
- fix standard `meta_factory` for `type[specialform]`
Error messages:
- mention layer name for extraneous arguments, so you know where the arg comes from
- reorder logic in cast for better errors
- more helpful error message when disallowing `__init__`, `__post_init__`, etc. thanks ebrevdo!
- other misc error message improvements
- misc internal docs
## November 2024
Two headline features for this month: references and `meta_factory` unification:
- references allow for deduplication of parameters and allow introducing indirection where some config is controlled by other teams
- `meta_factory` unification makes chz’s polymorphism more consistent and more powerful
Features:
- core of `meta_factory` unification, change default `meta_factory`
- infra for references, expose references
- use `X_values` from pre-init in `beta_to_blueprint_values`, thanks guillaume!
- give users access to methods_entrypoint blueprint
- add strict option to `Blueprint.apply`, thanks menick!
- add subpath to apply
- add override validators, thanks vineet!
- allow default values in nested_entrypoint
- make (wildcard) references not self-reference when defaulted
- recurse into dict in pretty_format
- make `meta_factory` lambda logic more robust
- support for python3.12 and 3.13
Typing features:
- basic pep 692 support
- add typeddict total=False and pep 655 support, thanks alec!
- add subtype support for pep 655 / required
- parse objects as literals
- allow casting to iterable
- allow ast eval of tuple for sequence
- better casting rules for list
- support casting pathlib
- add typeddict and callable tests
Error messages:
- improve two issues with `--help` in polymorphic command lines
- better error when we choose not to cast due to subparams
- batch errors for invalid ref targets
- improve error with reference cycles
- improve error message during blueprint evaluation
- include previous valid parent for non wildcard extraneous
- improve error mentioning closest matching parent for extraneous argument
- improve error messages on failure to interpret argument
- special case representation of objects from typing module
Internal:
- many refactoring changes and clean up, including large refactor of blueprint and changes for open source
## October 2024
- finally land support for variadic typeddicts
- add ability to attach user metadata to fields
- add better support for NewType, LiteralString, NamedTuple and other niche typing features
- add some support for PEP 646 unpacking of tuples
- add native support for casting fractions
- steps towards `meta_factory` unification. these changes make chz's polymorphism more powerful and more consistent
- allow disallowing `meta_factory`, useful in niche cases
- fix static typing of runtime typing to allow better downstream type checking
## September 2024
- add `blueprint_unspecified` to field, as generalisation of `chz.field(meta_factory=chz.factories.subclass(annot, default_cls=...))`. thanks to vitchyr for helping with this
- use `__orig_class__` to type check user defined generics, if possible
- add `chz.chz_fields` helper to access `__chz_fields__` attribute
- better error if there are no params and extraneous args
## August 2024
- add `check_field_consistency_in_tree` validator, as a way to help ensure your wildcards are doing what you want them to do
- use stdout for `--help`
- allow parsing empty tuple
- add a `const_default` validator for constant fields
## July 2024
- improvements to static types, thanks lmetz and wenda
- quick follow ups to `beta_blueprint_to_argv`, thanks hunter and noah
- improve `type_repr`, thanks davis
- minor error improvements
## June 2024
- support for polymorphic variadic generics
- fix some issues with pydantic support
- add `x_type` to improve static type checking of mungers
- add `beta_blueprint_to_argv`, thanks hunter
- fix callable subtyping with future annotations
- various improvements to `pretty_repr`, make dunder pure
- allow use of chz with abc
- add some special casing to avoid false positives with the conservative check against wildcard default factory interaction
- improve error message when validating types against a `Literal`
- improve error message when hashing chz class with unhashable fields, thanks alexk
- improve error message for unparseable type, thanks andmis
- fix typo in error message, thanks sean
- make various error messages more concise
## May 2024
- show default values in `--help`, includes some fancy logic around lambdas
- show values from unspecified_factory in `--help`, to make polymorphic construction easier to understand
- add `chz.methods_entrypoint` for easily make cli's from classes
- support mapping and sequence variadics
- basic support for pydantic validation during runtime type checking, thanks camillo
- better handling of runtime contexts for future annotations support
- support for nested classes when `meta_factory` turns strings into classes
- better support for polymorphism in `beta_blueprint_to_values`, thanks wenda
- only error for variadic failure if variadic param specified
- more docs, more tests, cleaner help output, cleaner tracebacks
## ???
Established in 2022
================================================
FILE: LICENSE
================================================
MIT License
Copyright (c) 2024 OpenAI
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: README.md
================================================
# 🪤 chz
*(pronounced "चीज़")*
`chz` helps you manage configuration, particularly from the command line.
`chz` is available on [PyPI](https://pypi.org/project/chz/).
To click the links below, please visit [Github](https://github.com/openai/chz).
Overview:
- [Quickstart](docs/01_quickstart.md)
- [Declarative object model](docs/02_object_model.md)
- [Immutability](docs/02_object_model.md#immutability)
- [Validation](docs/03_validation.md)
- [Type checking](docs/03_validation.md#type-checking)
- [Command line parsing](docs/04_command_line.md)
- [Discoverability](docs/04_command_line.md#discoverability---help-and-errors)
- [Partial application](docs/05_blueprint.md)
- [Presets or shared configuration](docs/05_blueprint.md#presets-or-shared-configuration)
- [Serialisation and deserialisation](docs/06_serialisation.md)
More details:
- [Post init](docs/21_post_init.md)
- [Field API](docs/22_field_api.md)
- [Philosophy](docs/91_philosophy.md)
- [Alternatives](docs/92_alternatives.md)
- [Testimonials](docs/93_testimonials.md)
Please let @shantanu know if you have feedback!
================================================
FILE: chz/__init__.py
================================================
from typing import TYPE_CHECKING, Callable, TypeVar, overload
from . import blueprint, factories, mungers, tiepin, validators
from .blueprint import (
Blueprint,
Castable,
dispatch_entrypoint,
entrypoint,
get_nested_target,
methods_entrypoint,
nested_entrypoint,
)
from .data_model import (
asdict,
beta_to_blueprint_values,
chz_fields,
chz_make_class,
init_property,
is_chz,
replace,
traverse,
)
from .field import field
from .validators import validate
__all__ = [
"Blueprint",
"asdict",
"chz",
"is_chz",
"chz_fields",
"entrypoint",
"field",
"get_nested_target",
"init_property",
"methods_entrypoint",
"nested_entrypoint",
"replace",
"beta_to_blueprint_values",
"traverse",
"validate",
"validators",
"mungers",
"Castable",
# are the following public?
"blueprint",
"factories",
"tiepin",
]
def _chz(cls=None, *, version: str | None = None, typecheck: bool | None = None):
if cls is None:
return lambda cls: chz_make_class(cls, version=version, typecheck=typecheck)
return chz_make_class(cls, version=version, typecheck=typecheck)
if TYPE_CHECKING:
_TypeT = TypeVar("_TypeT", bound=type)
from typing_extensions import dataclass_transform
@dataclass_transform(kw_only_default=True, frozen_default=True, field_specifiers=(field,))
@overload
def chz(version: str = ..., typecheck: bool = ...) -> Callable[[type], type]: ...
@overload
def chz(cls: _TypeT, /) -> _TypeT: ...
def chz(*a, **k):
raise NotImplementedError
else:
chz = _chz
================================================
FILE: chz/blueprint/__init__.py
================================================
from chz.blueprint._argv import argv_to_blueprint_args as argv_to_blueprint_args
from chz.blueprint._argv import beta_argv_arg_to_string as beta_argv_arg_to_string
from chz.blueprint._argv import beta_blueprint_to_argv as beta_blueprint_to_argv
from chz.blueprint._blueprint import Blueprint as Blueprint
from chz.blueprint._blueprint import Castable as Castable
from chz.blueprint._blueprint import Reference as Reference
from chz.blueprint._entrypoint import ConstructionException as ConstructionException
from chz.blueprint._entrypoint import EntrypointHelpException as EntrypointHelpException
from chz.blueprint._entrypoint import ExtraneousBlueprintArg as ExtraneousBlueprintArg
from chz.blueprint._entrypoint import InvalidBlueprintArg as InvalidBlueprintArg
from chz.blueprint._entrypoint import MissingBlueprintArg as MissingBlueprintArg
from chz.blueprint._entrypoint import dispatch_entrypoint as dispatch_entrypoint
from chz.blueprint._entrypoint import entrypoint as entrypoint
from chz.blueprint._entrypoint import exit_on_entrypoint_error as exit_on_entrypoint_error
from chz.blueprint._entrypoint import get_nested_target as get_nested_target
from chz.blueprint._entrypoint import methods_entrypoint as methods_entrypoint
from chz.blueprint._entrypoint import nested_entrypoint as nested_entrypoint
================================================
FILE: chz/blueprint/_argmap.py
================================================
from __future__ import annotations
import bisect
import re
from dataclasses import dataclass
from typing import TYPE_CHECKING, AbstractSet, Any, Iterator, Mapping
from chz.blueprint._entrypoint import ExtraneousBlueprintArg
from chz.blueprint._wildcard import wildcard_key_approx, wildcard_key_to_regex
if TYPE_CHECKING:
from chz.blueprint._blueprint import _MakeResult
class Layer:
def __init__(self, args: Mapping[str, Any], layer_name: str | None):
self._args = args
self.layer_name = layer_name
# Computed from the above
self.qualified = {}
self.wildcard = {}
self._to_regex = {}
# Match more specific wildcards first
for k, v in sorted(args.items(), key=lambda kv: -len(kv[0])):
if "..." in k:
self.wildcard[k] = v
self._to_regex[k] = wildcard_key_to_regex(k)
else:
self.qualified[k] = v
def get_kv(self, exact_key: str) -> tuple[str, Any, str | None] | None:
# TODO: remove this method
if exact_key in self.qualified:
return exact_key, self.qualified[exact_key], self.layer_name
for wildcard_key, value in self.wildcard.items():
if self._to_regex[wildcard_key].fullmatch(exact_key):
return wildcard_key, value, self.layer_name
return None
def iter_keys(self) -> Iterator[tuple[str, bool]]:
yield from ((k, False) for k in self.qualified)
yield from ((k, True) for k in self.wildcard)
def nest_subpath(self, subpath: str | None) -> Layer:
if subpath is None:
return self
return Layer(
{join_arg_path(subpath, k): v for k, v in self._args.items()},
self.layer_name,
)
def __repr__(self) -> str:
return f"<Layer {self.layer_name} {self.qualified | self.wildcard}>"
@dataclass(frozen=True)
class _FoundArgument:
key: str
value: Any
layer_index: int
layer_name: str | None
def _valid_parent(parts: list[str], param_paths: AbstractSet[str]) -> str | None:
for i in reversed(range(1, len(parts))):
parent = ".".join(parts[:i])
if parent in param_paths:
return parent
return None
class ArgumentMap:
def __init__(self, layers: list[Layer]) -> None:
self._layers = layers
self.consolidated = False
self.consolidated_qualified: dict[str, tuple[Any, int]] = {}
self.consolidated_qualified_sorted: list[str] = []
self.consolidated_wildcard: list[tuple[str, re.Pattern[str], Any, int]] = []
def add_layer(self, layer: Layer) -> None:
self._layers.append(layer)
self.consolidated = False
def consolidate(self) -> None:
self.consolidated_qualified = {}
for i, layer in enumerate(self._layers):
for key, value in layer.qualified.items():
self.consolidated_qualified[key] = (value, i)
self.consolidated_qualified_sorted = sorted(self.consolidated_qualified.keys())
self.consolidated_wildcard = []
for i, layer in reversed(list(enumerate(self._layers))):
for wildcard_key, value in layer.wildcard.items():
self.consolidated_wildcard.append(
(wildcard_key, layer._to_regex[wildcard_key], value, i)
)
self.consolidated = True
def subpaths(self, path: str, strict: bool = False) -> list[str]:
"""Returns the suffix of arguments this contains that would match a subpath of path.
The invariant is that for each element `suffix` in the returned list, `path + suffix`
would match an argument in this map.
Args:
strict: Whether to avoid returning arguments that match path exactly.
"""
assert self.consolidated, "ArgumentMap must be consolidated before calling subpaths"
assert not path.endswith(".")
wildcard_literal = path.split(".")[-1]
# note path may be the empty string
assert path.endswith(wildcard_literal)
path_plus_dot = path + "."
ret = []
if not strict and path in self.consolidated_qualified:
ret.append("")
if not path:
ret.extend([k for k in self.consolidated_qualified_sorted if k])
index = bisect.bisect_left(self.consolidated_qualified_sorted, path_plus_dot)
for i in range(index, len(self.consolidated_qualified_sorted)):
key = self.consolidated_qualified_sorted[i]
if not key.startswith(path_plus_dot):
break
ret.append(key.removeprefix(path_plus_dot))
assert key == join_arg_path(path, ret[-1])
for key, pattern, _value, _index in self.consolidated_wildcard:
if not path:
ret.append(key)
continue
# If it's not a wildcard, the logic is straightforward. But doing the equivalent
# for wildcards is tricky!
i = key.rfind(wildcard_literal)
if i == -1:
continue
# The not strict case is not complicated, we just regex match
if pattern.fullmatch(path):
if not strict:
ret.append("")
assert pattern.fullmatch(path + ret[-1])
continue
# This needs a little thinking about.
# Say path is "foo.bar" and key is "...bar...baz"
# Then wildcard_literal is "bar" and we check if "...bar" matches "foo.bar"
# Since it does, we append "...baz"
while i != -1:
if (
i + len(wildcard_literal) < len(key)
and key[i + len(wildcard_literal)] == "."
and wildcard_key_to_regex(key[: i + len(wildcard_literal)]).fullmatch(path)
):
assert i == 0 or key[i - 1] == "."
suffix = key[i + len(wildcard_literal) :]
if not suffix.startswith("..."):
suffix = suffix.removeprefix(".")
ret.append(suffix)
assert pattern.fullmatch(join_arg_path(path, ret[-1]))
break
i_next = key.rfind(wildcard_literal, 0, i)
assert i_next < i, "Infinite loop"
i = i_next
return ret
def get_kv(self, exact_key: str, *, ignore_wildcards: bool = False) -> _FoundArgument | None:
assert self.consolidated, "ArgumentMap must be consolidated before calling get_kv"
lookup = self.consolidated_qualified.get(exact_key)
if not ignore_wildcards:
lookup_index = lookup[1] if lookup is not None else -1
for wildcard_key, pattern, value, index in self.consolidated_wildcard:
if index <= lookup_index:
break
if pattern.fullmatch(exact_key):
layer_name = self._layers[index].layer_name
return _FoundArgument(wildcard_key, value, index, layer_name=layer_name)
if lookup is not None:
value, lookup_index = lookup
layer_name = self._layers[lookup_index].layer_name
return _FoundArgument(exact_key, value, lookup_index, layer_name=layer_name)
return None
def check_extraneous(
self,
used_args: set[tuple[str, int]],
param_paths: AbstractSet[str],
make_result: _MakeResult,
*,
entrypoint_repr: str,
) -> None:
for index in range(len(self._layers)):
layer = self._layers[index]
for key, is_wildcard in layer.iter_keys():
# If something is not in used_args, it means it was either extraneous or it got
# clobbered because something in a higher layer matched it
if (key, index) in used_args:
continue
if (
# It's easy to check if a non-wildcard arg was clobbered. We just check if
# there was a param with that name (that we should have matched if not for
# presumed clobbering)
(not is_wildcard and key not in param_paths)
# For wildcards, we need to match against all param paths
or (
is_wildcard
and not any(layer._to_regex[key].fullmatch(p) for p in param_paths)
)
):
# Okay, we have an extraneous argument. We're going to error, but we should
# helpfully try to figure out what the user wanted
extra = ""
if layer.layer_name:
extra += f" (from {layer.layer_name})"
ratios = {p: wildcard_key_approx(key, p) for p in param_paths}
if ratios:
max_option = max(ratios, key=lambda v: ratios[v][0])
if ratios[max_option][0] > 0.1:
extra = f"\nDid you mean {ratios[max_option][1]!r}?"
if not is_wildcard:
nested_pattern = wildcard_key_to_regex("..." + key)
found_key = next(
(p for p in param_paths if nested_pattern.fullmatch(p)), None
)
if found_key is not None:
extra += (
f"\nDid you get the nesting wrong, maybe you meant {found_key!r}?"
)
if key.startswith("--"):
extra += "\nDid you mean to use allow_hyphens=True in your entrypoint?"
if not is_wildcard:
parts = key.split(".")
if len(parts) >= 2:
valid_parent = _valid_parent(parts, param_paths)
if valid_parent is None:
extra += f"\nNo param found matching {parts[0]!r}"
else:
from chz.blueprint._blueprint import _found_arg_desc
extra += f"\n\nParam {valid_parent!r} is closest valid ancestor"
parent_found_arg = self.get_kv(valid_parent)
param = make_result.all_params[valid_parent]
desc = _found_arg_desc(
make_result,
parent_found_arg,
param_path=valid_parent,
param=param,
omit_redundant=False,
)
invalid_part = (
".".join(parts).removeprefix(valid_parent + ".").split(".")[0]
)
extra += f"\nParam {valid_parent!r} is set to {desc}"
extra += f"\nSubparam {invalid_part!r} does not exist on it"
raise ExtraneousBlueprintArg(
f"Extraneous argument {key!r} to Blueprint for {entrypoint_repr}"
+ extra
+ "\nAppend --help to your command to see valid arguments"
)
def __repr__(self) -> str:
return "ArgumentMap(\n" + "\n".join(" " + repr(layer) for layer in self._layers) + "\n)"
def join_arg_path(parent: str, child: str) -> str:
if not parent:
return child
if child.startswith(".") or child == "":
return parent + child
return parent + "." + child
================================================
FILE: chz/blueprint/_argv.py
================================================
from __future__ import annotations
import itertools
import types
from typing import Any, TypeVar
import chz.blueprint
from chz.blueprint._argmap import Layer
from chz.blueprint._wildcard import wildcard_key_to_regex
from chz.tiepin import type_repr
_T = TypeVar("_T")
def argv_to_blueprint_args(
argv: list[str], *, allow_hyphens: bool = False
) -> dict[str, chz.blueprint.Castable | chz.blueprint.Reference]:
# TODO: allow stuff like model[family=linear n_layers=1]
ret: dict[str, chz.blueprint.Castable | chz.blueprint.Reference] = {}
for arg in argv:
try:
key, value = arg.split("=", 1)
except ValueError:
raise ValueError(
f"Invalid argument {arg!r}. Specify arguments in the form key=value"
) from None
if allow_hyphens:
key = key.lstrip("-")
# parse key@=reference syntax (note =@ would be ambiguous)
if key.endswith("@"):
ret[key.removesuffix("@")] = chz.blueprint.Reference(value)
else:
ret[key] = chz.blueprint.Castable(value)
return ret
def beta_argv_arg_to_string(key: str, value: Any) -> list[str]:
if isinstance(value, chz.blueprint.Castable):
return [f"{key}={value.value}"]
if isinstance(value, chz.blueprint.Reference):
return [f"{key}@={value.ref}"]
if isinstance(value, (types.FunctionType, type)):
return [f"{key}={type_repr(value)}"]
if isinstance(value, str):
return [f"{key}={value}"]
if isinstance(value, (int, float, bool)) or value is None:
return [f"{key}={repr(value)}"]
if isinstance(value, (list, tuple)):
if all(isinstance(e, str) for e in value):
if not any("," in e for e in value):
return [f"{key}={','.join(value)}"]
args_list = []
for i, e in enumerate(value):
args_list.extend(beta_argv_arg_to_string(f"{key}.{i}", e))
return args_list
elif all(isinstance(e, (int, float, bool)) or e is None for e in value):
return [f"{key}={','.join(map(str, value))}"]
if isinstance(value, dict):
args_list = []
for k, v in value.items():
args_list.extend(beta_argv_arg_to_string(f"{key}.{k}", v))
return args_list
# Probably safe to use repr here, but I'm curious to see how people end up using this
raise NotImplementedError(
f"TODO: beta_blueprint_to_argv does not currently convert {value!r} of "
f"type {type(value)} to string"
)
def beta_blueprint_to_argv(blueprint: chz.Blueprint[_T]) -> list[str]:
"""Returns a list of arguments that would recreate the given blueprint.
Please do not use this function without asking @shantanu, it is slow and not fully robust,
and more importantly, there may well be a better way to accomplish your goal.
"""
ret = [
arg
for key, value in _collapse_layers(blueprint)
for arg in beta_argv_arg_to_string(key, value)
]
return ret
def _collapse_layer(
ordered_args: list[tuple[str, Any]], ordered_arg_keys: set[str], layer: Layer
) -> None:
"""Collapses `layer` into `ordered_args`, overriding any old keys as necessary."""
layer_args: list[tuple[str, Any]] = []
keys_to_remove: set[str] = set()
for key, value in itertools.chain(layer.qualified.items(), layer.wildcard.items()):
# Remove any previous args that would be overwritten by this one.
wildcard = wildcard_key_to_regex(key) if "..." in key else None
if wildcard:
for prev_key in ordered_arg_keys:
# TODO(shantanu): usually this regex is only matched against concrete keys
# However, here we're matching against other wildcards
if wildcard.fullmatch(prev_key):
keys_to_remove.add(prev_key)
else:
if key in ordered_arg_keys:
keys_to_remove.add(key)
layer_args.append((key, value))
# Commit the new layer
ordered_args[:] = [arg for arg in ordered_args if arg[0] not in keys_to_remove] + layer_args
ordered_arg_keys.difference_update(keys_to_remove)
ordered_arg_keys.update(key for key, _ in layer_args)
def _collapse_layers(blueprint: chz.Blueprint[_T]) -> list[tuple[str, Any]]:
"""Collapses the layers of a blueprint into a list of key-value pairs.
These could be applied as a single layer to a new blueprint to recreate the original.
"""
ordered_args: list[tuple[str, Any]] = []
ordered_arg_keys: set[str] = set()
for layer in blueprint._arg_map._layers:
_collapse_layer(ordered_args, ordered_arg_keys, layer)
return ordered_args
================================================
FILE: chz/blueprint/_blueprint.py
================================================
from __future__ import annotations
import ast
import collections.abc
import dataclasses
import functools
import inspect
import io
import sys
import textwrap
import typing
from dataclasses import dataclass
from typing import Any, Callable, Final, Generic, Mapping, Protocol
from typing_extensions import TypeVar
import chz
from chz.blueprint._argmap import ArgumentMap, Layer, _FoundArgument, join_arg_path
from chz.blueprint._argv import argv_to_blueprint_args
from chz.blueprint._entrypoint import (
ConstructionException,
EntrypointHelpException,
ExtraneousBlueprintArg,
InvalidBlueprintArg,
MissingBlueprintArg,
)
from chz.blueprint._lazy import (
Evaluatable,
ParamRef,
Thunk,
Value,
check_reference_targets,
evaluate,
)
from chz.field import Field
from chz.tiepin import (
CastError,
_simplistic_try_cast,
_simplistic_type_of_value,
eval_in_context,
is_kwargs_unpack,
is_subtype_instance,
is_typed_dict,
type_repr,
)
from chz.util import MISSING, MISSING_TYPE
_T = TypeVar("_T")
_T_cov_def = TypeVar("_T_cov_def", covariant=True, default=Any)
class SpecialArg: ...
class Castable(SpecialArg):
"""A wrapper class for str if you want a Blueprint value to be magically type aware casted."""
def __init__(self, value: str) -> None:
self.value = value
def __repr__(self) -> str:
return f"Castable({self.value!r})"
def __hash__(self) -> int:
return hash(self.value)
def __eq__(self, other: object) -> bool:
if not isinstance(other, Castable):
try:
return _simplistic_try_cast(self.value, type(other)) == other
except CastError:
return False
return self.value == other.value
class Reference(SpecialArg):
"""A reference to another parameter in a Blueprint."""
def __init__(self, ref: str) -> None:
if "..." in ref:
raise ValueError("Cannot use wildcard as a reference target")
self.ref = ref
def __repr__(self) -> str:
return f"Reference({self.ref!r})"
@dataclass(frozen=True, kw_only=True)
class Computed(SpecialArg):
"""A parameter computed from other parameters in a Blueprint."""
src: dict[str, Reference]
compute: Callable[..., Any]
def __repr__(self) -> str:
arg_str = ", ".join(f"{k}@={v.ref}" for k, v in self.src.items())
return f"Computed({arg_str})"
@dataclass(frozen=True)
class _MakeResult:
# `value_mapping` is a dictionary mapping from parameter paths to Evaluatable values.
# This ultimately contains all the kinds of values we will use in instantiation.
# See chz.blueprint._lazy.evaluate for an example of using Evaluatable.
value_mapping: dict[str, Evaluatable]
# `all_params` is a dictionary containing all parameters we discover, mapping from that param
# path to the parameter. Note what parameters we discover will depend on polymorphic
# construction via meta_factories. We use all_params to provide a useful --help (and various
# other things, e.g. detect clobbering when checking for extraneous arguments)
all_params: dict[str, _Param]
# `used_args` is a set of (key, layer_index) tuples that we use to track which arguments from
# arg_map we've used. We use this to check for extraneous arguments.
used_args: set[tuple[str, int]]
# `meta_factory_value` records what meta_factory we're using. This makes --help more
# understandable in the presence of polymorphism, especially when factories come from
# blueprint_unspecified. It's conceptually the same information as in Thunk.fn in value_mapping,
# but preserves user input for variadics or generics (instead of being a constructor function)
meta_factory_value: dict[str, Any]
# `missing_params` is a list of parameters we know need are required but haven't been
# specified. In theory, this is unnecessary because `__init__` will raise an error if
# a required param is missing, but this improves diagnostics.
missing_params: list[str]
def _entrypoint_caster(o: str) -> object:
raise chz.tiepin.CastError("Will not interpret entrypoint as a value")
def _found_arg_desc(
r: _MakeResult,
found_arg: _FoundArgument | None,
*,
param_path: str,
param: _Param,
omit_redundant: bool = True,
color: bool = False,
) -> str:
if found_arg is None:
if param_path in r.meta_factory_value:
found_arg_str = type_repr(r.meta_factory_value[param_path])
if color:
found_arg_str += " \033[90m(meta_factory)\033[0m"
else:
found_arg_str += " (meta_factory)"
elif param.default is not None:
found_arg_str = param.default.to_help_str()
if color:
found_arg_str += " \033[90m(default)\033[0m"
else:
found_arg_str += " (default)"
elif (
param.meta_factory is not None
and (factory := param.meta_factory.unspecified_factory()) is not None
and (factory is not param.type or not omit_redundant)
):
if getattr(factory, "__name__", None) == "<lambda>":
found_arg_str = _lambda_repr(factory) or type_repr(factory)
else:
found_arg_str = type_repr(factory)
if color:
found_arg_str += " \033[90m(blueprint_unspecified)\033[0m"
else:
found_arg_str += " (blueprint_unspecified)"
else:
found_arg_str = "-"
else:
if isinstance(found_arg.value, Castable):
found_arg_str = repr(found_arg.value.value)[1:-1]
elif isinstance(found_arg.value, Reference):
found_arg_str = f"@={found_arg.value.ref}"
elif isinstance(found_arg.value, Computed):
arg_str = ", ".join(f"{k}@={v.ref}" for k, v in found_arg.value.src.items())
found_arg_str = f"f({arg_str})"
else:
found_arg_str = type_repr(found_arg.value)
if found_arg.layer_name:
if color:
found_arg_str += f" \033[90m(from \033[94m{found_arg.layer_name}\033[90m)\033[0m"
else:
found_arg_str += f" (from {found_arg.layer_name})"
return found_arg_str
class Blueprint(Generic[_T_cov_def]):
def __init__(
self, target: chz.factories.MetaFactory | type[_T_cov_def] | Callable[..., _T_cov_def]
) -> None:
"""Instantiate a Blueprint.
Args:
target: The target object or callable we will instantiate or call.
"""
self.target = target
if isinstance(target, chz.factories.MetaFactory):
self.meta_factory = target
if isinstance(target, chz.factories.standard):
entrypoint_type = target.annotation
entrypoint_doc = getattr(entrypoint_type, "__doc__", "")
else:
entrypoint_type = object
entrypoint_doc = ""
self.entrypoint_repr = type_repr(entrypoint_type)
else:
self.meta_factory = chz.factories.standard(annotation=target)
entrypoint_type = target
if self.meta_factory.unspecified_factory() is None:
if not callable(target):
raise ValueError(f"{target} is not callable")
self.meta_factory = chz.factories.standard(annotation=object, unspecified=target)
entrypoint_type = object
self.entrypoint_repr = type_repr(target)
entrypoint_doc = getattr(target, "__doc__", "")
self.param = _Param(
name="",
type=entrypoint_type,
meta_factory=self.meta_factory,
default=None,
doc=entrypoint_doc.strip() if entrypoint_doc else "",
blueprint_cast=_entrypoint_caster,
metadata={},
)
self._arg_map = ArgumentMap([])
def clone(self) -> Blueprint[_T_cov_def]:
"""Make a copy of this Blueprint."""
return Blueprint(self.target).apply(self)
def apply(
self,
values: Blueprint[_T_cov_def] | Mapping[str, Any],
layer_name: str | None = None,
*,
subpath: str | None = None,
strict: bool = False,
) -> Blueprint[_T_cov_def]:
"""Modify this Blueprint by partially applying some arguments.
Args:
values: The values to partially apply to this Blueprint
layer_name: The name of the layer to add (allows identification of the source of values)
subpath: A subpath to nest the argument names under
strict: Whether to eagerly check for extraneous arguments. This may not work well in
cases where a polymorphic field is applied later.
"""
if isinstance(values, Mapping):
self._arg_map.add_layer(Layer(values, layer_name).nest_subpath(subpath))
elif isinstance(values, Blueprint):
if subpath is None:
if values.target is not self.target:
raise ValueError(
f"Cannot apply Blueprint for {type_repr(values.target)} to Blueprint for "
f"{type_repr(self.target)}"
)
for layer in values._arg_map._layers:
self._arg_map.add_layer(layer.nest_subpath(subpath))
else:
raise TypeError(f"Expected dict or Blueprint, got {type(values)}")
if strict:
r = self._make_lazy()
self._arg_map.check_extraneous(
r.used_args,
r.all_params.keys(),
make_result=r,
entrypoint_repr=self.entrypoint_repr,
)
return self
def apply_from_argv(
self, argv: list[str], allow_hyphens: bool = False, layer_name: str = "command line"
) -> Blueprint[_T_cov_def]:
"""Apply arguments from argv to this Blueprint."""
values = argv_to_blueprint_args(
[a for a in argv if a != "--help"], allow_hyphens=allow_hyphens
)
self.apply(values, layer_name=layer_name)
if "--help" in argv:
argv.remove("--help")
raise EntrypointHelpException(self.get_help(color=sys.stdout.isatty()))
return self
def _make_lazy(self) -> _MakeResult:
all_params: dict[str, _Param] = {}
used_args: set[tuple[str, int]] = set()
meta_factory_value: dict[str, Any] = {}
missing_params: list[str] = []
self._arg_map.consolidate()
value_mapping: dict[str, Evaluatable] | ConstructionIssue | None
value_mapping = _construct_param(
self.param,
"",
self._arg_map,
all_params=all_params,
used_args=used_args,
meta_factory_value=meta_factory_value,
missing_params=missing_params,
)
if isinstance(value_mapping, ConstructionIssue):
raise ConstructionException(value_mapping.issue)
if value_mapping is None:
# value_mapping is None if _construct_param / _construct_unspecified_param
# wants us to fallback to the default or thinks we're missing required arguments
# This is sort of equivalent to what happens in _construct_factory
unspecified_factory = self.meta_factory.unspecified_factory()
if unspecified_factory is None:
raise ConstructionException(
f"Cannot construct {self.target} because it has no unspecified factory"
)
value_mapping = {"": Thunk(unspecified_factory, {})}
if "" in missing_params:
missing_params.remove("")
return _MakeResult(
value_mapping=value_mapping,
all_params=all_params,
used_args=used_args,
meta_factory_value=meta_factory_value,
missing_params=missing_params,
)
def _make_from_make_result(self, r: _MakeResult) -> _T_cov_def:
self._arg_map.check_extraneous(
r.used_args,
r.all_params.keys(),
make_result=r,
entrypoint_repr=self.entrypoint_repr,
)
check_reference_targets(r.value_mapping, r.all_params.keys())
# Note we check for extraneous args first, so we get better errors for typos
if r.missing_params:
raise MissingBlueprintArg(
f"Missing required arguments for parameter(s): {', '.join(r.missing_params)}"
)
# __chz_blueprint__
return evaluate(r.value_mapping)
def make(self) -> _T_cov_def:
"""Instantiate or call the target object or callable."""
r = self._make_lazy()
return self._make_from_make_result(r)
def make_from_argv(
self, argv: list[str] | None = None, allow_hyphens: bool = False
) -> _T_cov_def:
"""Like make, but suitable for command line entrypoints.
This will apply arguments from argv to this Blueprint before attempting to make the target.
If "--help" is in argv, this will print help text and exit.
"""
if argv is None:
argv = sys.argv[1:]
self.apply_from_argv(argv, allow_hyphens=allow_hyphens)
return self.make()
def get_help(self, *, color: bool = False) -> str:
"""Get help text for this Blueprint.
Note that applied arguments may affect output here, e.g. in case of polymorphically
constructed fields.
"""
r = self._make_lazy()
f = io.StringIO()
output = functools.partial(print, file=f)
try:
self._arg_map.check_extraneous(
r.used_args,
r.all_params.keys(),
make_result=r,
entrypoint_repr=self.entrypoint_repr,
)
except ExtraneousBlueprintArg as e:
output(f"WARNING: {e}\n")
try:
check_reference_targets(r.value_mapping, r.all_params.keys())
except InvalidBlueprintArg as e:
output(f"WARNING: {e}\n")
if r.missing_params:
output(
f"WARNING: Missing required arguments for parameter(s): {', '.join(r.missing_params)}\n"
)
output(f"Entry point: {self.entrypoint_repr}")
output()
if self.param.doc:
output(textwrap.indent(self.param.doc, " "))
output()
param_output = []
for param_path, param in r.all_params.items():
# TODO: cast or meta_factory info, not just type
found_arg = self._arg_map.get_kv(param_path)
if (
not isinstance(self.target, chz.factories.MetaFactory)
and param_path == ""
and found_arg is None
):
# If we're not using root polymorphism, skip this param
continue
found_arg_str = _found_arg_desc(
r, found_arg, param_path=param_path, param=param, color=color
)
param_output.append(
(param_path or "<entrypoint>", type_repr(param.type), found_arg_str, param.doc)
)
clip = 40
lens = tuple(min(clip, max(map(len, column))) for column in zip(*param_output))
output("Arguments:")
for p, typ, arg, doc in param_output:
col = 0
target_col = 0
line = io.StringIO()
add = functools.partial(print, file=line, end="")
raw_string = p
add(" ")
if color:
add(f"\033[1m{raw_string}\033[0m")
else:
add(raw_string)
col += 2 + len(raw_string)
target_col += 2 + lens[0]
pad = (target_col - col) if col <= target_col else (-len(raw_string)) % 20
add(" " * pad)
col += pad
raw_string = typ
add(" ")
add(raw_string)
col += 2 + len(raw_string)
target_col += 2 + lens[1]
pad = (target_col - col) if col <= target_col else (-len(raw_string)) % 20
add(" " * pad)
col += pad
raw_string = arg
add(" ")
add(raw_string)
col += 2 + len(raw_string)
target_col += 2 + lens[2]
pad = (target_col - col) if col <= target_col else (-len(raw_string)) % 20
add(" " * pad)
col += pad
raw_string = doc
add(" ")
if color:
add(f"\033[90m{raw_string}\033[0m")
else:
add(raw_string)
output(line.getvalue().rstrip())
return f.getvalue()
def _lambda_repr(fn) -> str | None:
try:
src = inspect.getsource(fn).strip()
tree = ast.parse(src)
nodes = [node for node in ast.walk(tree) if isinstance(node, ast.Lambda)]
if len(nodes) != 1:
return None
return ast.unparse(nodes[0])
except Exception:
return None
@dataclass(frozen=True, kw_only=True)
class _Default:
value: Any | MISSING_TYPE
factory: Callable[..., Any] | MISSING_TYPE
def to_help_str(self) -> str:
if self.factory is not MISSING:
if getattr(self.factory, "__name__", None) == "<lambda>":
return f"({_lambda_repr(self.factory)})()" or "<default>"
# type_repr works reasonably well for functions too
return f"{type_repr(self.factory)}()"
ret = repr(self.value)
if len(ret) > 40:
return "<default>"
return ret
def instantiate(self) -> Any:
if not isinstance(self.factory, MISSING_TYPE):
return self.factory()
return self.value
@classmethod
def from_field(cls, field: Field) -> _Default | None:
if field._default is MISSING and field._default_factory is MISSING:
return None
return _Default(value=field._default, factory=field._default_factory)
@classmethod
def from_inspect_param(cls, sigparam: inspect.Parameter) -> _Default | None:
if sigparam.default is sigparam.empty:
return None
return _Default(value=sigparam.default, factory=MISSING)
@dataclass(frozen=True, kw_only=True)
class _Param:
name: str
type: Any
meta_factory: chz.factories.MetaFactory | None
default: _Default | None
doc: str
blueprint_cast: Callable[[str], object] | None
metadata: dict[str, Any]
def cast(self, value: str) -> object:
# If we have a field-level cast, always use that!
if self.blueprint_cast is not None:
return self.blueprint_cast(value)
# If we have a meta_factory, route casting through it. This allows user expectations
# of types that result from casting to better match their expectations from polymorphic
# construction (e.g. using default_cls from chz.factories.subclass)
if self.meta_factory is not None:
return self.meta_factory.perform_cast(value)
# TODO: maybe MetaFactory should have default impl and this should be:
# return chz.factories.MetaFactory().perform_cast(value, self.type)
return _simplistic_try_cast(value, self.type)
def _get_variadic_elements(obj_path: str, arg_map: ArgumentMap) -> set[str]:
elements = set()
for subpath in arg_map.subpaths(obj_path, strict=True):
assert subpath
if subpath[0] != ".":
element = subpath.split(".")[0]
assert element
elements.add(element)
return elements
def _collect_params_from_chz(
obj: Any, obj_path: str, arg_map: ArgumentMap
) -> tuple[list[_Param], Callable[..., Any], list[Any]]:
obj_origin = getattr(obj, "__origin__", obj)
params = []
for field in chz.chz_fields(obj_origin).values():
params.append(
_Param(
name=field.logical_name,
type=field.x_type,
meta_factory=field.meta_factory,
default=_Default.from_field(field),
doc=field._doc,
blueprint_cast=field._blueprint_cast,
metadata=(field.metadata or {}),
)
)
return params, obj, []
def _collect_params_from_sequence(
obj: Any, obj_path: str, arg_map: ArgumentMap
) -> tuple[list[_Param], Callable[..., Any], list[Any]]:
elements = _get_variadic_elements(obj_path, arg_map)
max_element = max((int(e) for e in elements), default=-1)
obj_origin = getattr(obj, "__origin__", obj)
obj_type_construct = obj_origin
type_for_index: Callable[[int], type]
if obj_origin is list:
element_type = getattr(obj, "__args__", [object])[0]
type_for_index = lambda i: element_type
variadic_types = [element_type]
elif obj_origin is collections.abc.Sequence:
element_type = getattr(obj, "__args__", [object])[0]
type_for_index = lambda i: element_type
variadic_types = [element_type]
obj_type_construct = tuple
elif obj_origin is tuple:
args: tuple[Any, ...] | None = getattr(obj, "__args__", None)
if args is None:
args = (Any, ...)
if len(args) == 2 and args[-1] is ...:
# homogeneous tuple
type_for_index = lambda i: args[0]
variadic_types = [args[0]]
else:
# heterogeneous tuple
if max_element >= len(args):
raise TypeError(
f"Tuple type {obj} for {obj_path!r} must take {len(args)} items; "
f"arguments for index {max_element} were specified"
+ (
f". Homogeneous tuples should be typed as tuple[{type_repr(args[0])}, ...] not tuple[{type_repr(args[0])}]"
if len(args) == 1
else ""
)
)
type_for_index = lambda i: args[i]
variadic_types = list(args)
else:
raise AssertionError
params = []
for i in range(max_element + 1):
element_type = type_for_index(i)
params.append(
_Param(
name=str(i),
type=element_type,
meta_factory=chz.factories.standard(annotation=element_type),
default=None,
doc="",
blueprint_cast=None,
metadata={},
)
)
def sequence_constructor(**kwargs):
return obj_type_construct(kwargs[str(i)] for i in range(max_element + 1))
obj_constructor = sequence_constructor
return params, obj_constructor, variadic_types
def _collect_params_from_mapping(
obj: Any, obj_path: str, arg_map: ArgumentMap
) -> tuple[list[_Param], Callable[..., Any], list[Any]] | ConstructionIssue:
elements = _get_variadic_elements(obj_path, arg_map)
args: tuple[Any, ...] = getattr(obj, "__args__", ())
if len(args) == 0:
element_type = object
key_type = str
elif len(args) == 2:
if args[0] not in (str, int):
if elements:
raise TypeError(
f"Variadic dict type must take str or int keys, not {type_repr(args[0])}"
)
return ConstructionIssue(
f"Variadic dict type must take str or int keys, not {type_repr(args[0])}"
)
key_type = args[0]
element_type = args[1]
else:
raise TypeError(f"Dict type {obj} must take 0 or 2 items")
params = []
for element in elements:
params.append(
_Param(
name=element,
type=element_type,
meta_factory=chz.factories.standard(annotation=element_type),
default=None,
doc="",
blueprint_cast=None,
metadata={},
)
)
def _dict(**kwargs) -> dict[int | str, Any]:
return {key_type(k): v for k, v in kwargs.items()}
return params, _dict, [element_type]
def _collect_params_from_typed_dict(
obj: Any, obj_path: str, arg_map: ArgumentMap
) -> tuple[list[_Param], Callable[..., Any], list[Any]]:
obj_origin = getattr(obj, "__origin__", obj)
params = []
variadic_types = []
for key, annotation in typing.get_type_hints(obj_origin).items():
required = key in obj_origin.__required_keys__
params.append(
_Param(
name=key,
type=annotation,
meta_factory=chz.factories.standard(annotation=annotation),
# Mark the default as NotRequired to improve --help output
# We don't actually use the default values in Blueprint since we let
# instantiation handle insertion of default values
default=(None if required else _Default(value=typing.NotRequired, factory=MISSING)),
doc="",
blueprint_cast=None,
metadata={},
)
)
variadic_types.append(annotation)
return params, obj_origin, variadic_types
def _collect_params_from_callable(
obj: Any, obj_path: str, arg_map: ArgumentMap
) -> tuple[list[_Param], Callable[..., Any], list[Any]] | ConstructionIssue:
# Note you probably don't want to call this if obj is a primitive
try:
signature = inspect.signature(obj)
except ValueError:
return ConstructionIssue(f"Failed to get signature for {obj.__name__}")
obj_constructor = obj
has_pos_only = False
has_pos_or_kwarg = False
elements: set[str] | None = None
params = []
for i, (name, sigparam) in enumerate(signature.parameters.items()):
param_annot: Any
if sigparam.annotation is sigparam.empty:
if i == 0 and "." in obj.__qualname__:
# potentially first parameter of a method, default the annotation to the class
try:
cls = getattr(inspect.getmodule(obj), obj.__qualname__.rsplit(".", 1)[0])
param_annot = cls
except Exception:
param_annot = object
else:
param_annot = object
else:
param_annot = sigparam.annotation
if isinstance(param_annot, str):
try:
param_annot = eval_in_context(param_annot, obj)
except Exception as e:
raise ValueError(
f"Failed to evaluate parameter {name}: {param_annot} in signature {signature} of object {obj}"
) from e
if sigparam.kind == sigparam.POSITIONAL_ONLY:
has_pos_only = True
name = str(i)
if sigparam.kind == sigparam.POSITIONAL_OR_KEYWORD:
has_pos_or_kwarg = True
if sigparam.kind == sigparam.VAR_POSITIONAL:
if elements is None:
elements = _get_variadic_elements(obj_path, arg_map)
max_element = max((int(e) for e in elements if e.isdigit()), default=-1)
if has_pos_or_kwarg and max_element >= 0:
return ConstructionIssue(
"Cannot collect parameters with both positional-or-keyword and variadic positional parameters"
)
has_pos_only = True
for j in range(i, max_element + 1):
params.append(
_Param(
name=str(j),
type=param_annot,
meta_factory=chz.factories.standard(annotation=param_annot),
default=None,
doc="",
blueprint_cast=None,
metadata={},
)
)
continue
if sigparam.kind == sigparam.VAR_KEYWORD:
if is_kwargs_unpack(param_annot):
if len(param_annot.__args__) != 1 or not is_typed_dict(param_annot.__args__[0]):
return ConstructionIssue(
f"Cannot collect parameters from {obj.__name__}, expected Unpack[TypedDict], not {param_annot}"
)
for key, annotation in typing.get_type_hints(param_annot.__args__[0]).items():
# TODO: handle total=False and PEP 655
params.append(
_Param(
name=key,
type=annotation,
meta_factory=chz.factories.standard(annotation=annotation),
default=None,
doc="",
blueprint_cast=None,
metadata={},
)
)
else:
if elements is None:
elements = _get_variadic_elements(obj_path, arg_map)
for element in elements - {p.name for p in params}:
params.append(
_Param(
name=element,
type=param_annot,
meta_factory=chz.factories.standard(annotation=param_annot),
default=None,
doc="",
blueprint_cast=None,
metadata={},
)
)
continue
# It could be interesting to let function defaults be chz.Field :-)
# TODO: could be fun to parse function docstring
params.append(
_Param(
name=name,
type=param_annot,
meta_factory=chz.factories.standard(annotation=param_annot),
default=_Default.from_inspect_param(sigparam),
doc="",
blueprint_cast=None,
metadata={},
)
)
if has_pos_only:
def positional_constructor(**kwargs):
a = []
kw = {}
for k, v in kwargs.items():
if k.isdigit():
a.append((int(k), v))
else:
kw[k] = v
a = [v for _, v in sorted(a)]
return obj(*a, **kw)
obj_constructor = positional_constructor
# Note params may be empty here if obj doesn't take any parameters.
# This is usually okay, but has some interaction with fully defaulted subcomponents.
# See test_nested_all_defaults and variants
return params, obj_constructor, []
def _collect_params(
obj: Any, obj_path: str, arg_map: ArgumentMap
) -> (
ConstructionIssue
| tuple[
list[_Param], # params discovered
Callable[..., Any], # constructor to call
list[Any], # vaguely like [p.type for p in params], used only for sanity checking
]
):
obj_origin = getattr(obj, "__origin__", obj)
if chz.is_chz(obj_origin):
return _collect_params_from_chz(obj, obj_path, arg_map)
if isinstance(obj, functools.partial) and chz.is_chz(obj.func):
if obj.args:
return ConstructionIssue(
f"Cannot collect parameters from partial function of chz class "
f"{type_repr(obj.func)} with positional arguments"
)
result = _collect_params(obj.func, obj_path, arg_map)
if isinstance(result, ConstructionIssue):
return result
params, _constructor, variadic_types = result
new_params = []
for param in params:
if param.name in obj.keywords:
# The actual value of the default should only matter for --help output
param = dataclasses.replace(
param, default=_Default(value=obj.keywords[param.name], factory=MISSING)
)
new_params.append(param)
return new_params, obj, variadic_types
if obj_origin in {list, tuple, collections.abc.Sequence}:
return _collect_params_from_sequence(obj, obj_path, arg_map)
if obj_origin in {dict, collections.abc.Mapping}:
return _collect_params_from_mapping(obj, obj_path, arg_map)
if is_typed_dict(obj_origin):
return _collect_params_from_typed_dict(obj, obj_path, arg_map)
if obj_origin in {bool, int, float, str, bytes, None, type(None)}:
return ConstructionIssue("Cannot collect parameters from primitive")
if "enum" in sys.modules:
import enum
if type(obj) is enum.EnumMeta:
return ConstructionIssue("Cannot collect parameters from Enum class")
if callable(obj):
return _collect_params_from_callable(obj, obj_path, arg_map)
return ConstructionIssue(
f"Could not collect parameters to construct {obj} of type {type_repr(obj)}"
)
_K = TypeVar("_K")
_V = TypeVar("_V", contravariant=True)
class _WriteOnlyMapping(Generic[_K, _V], Protocol):
def __setitem__(self, __key: _K, __value: _V, /) -> None: ...
def update(self, __m: Mapping[_K, _V], /) -> None: ...
class ConstructionIssue:
def __init__(self, issue: str) -> None:
self.issue = issue
def _construct_factory(
obj: Callable[..., _T],
obj_path: str,
arg_map: ArgumentMap,
*,
# Output parameters, do not use within this function
# Typing these as write-only should help prevent accidental unsound use within this function
# See _MakeResult for docs about these parameters
all_params: _WriteOnlyMapping[str, _Param],
used_args: set[tuple[str, int]],
meta_factory_value: _WriteOnlyMapping[str, Any],
missing_params: list[str],
) -> dict[str, Evaluatable] | ConstructionIssue:
result = _collect_params(obj, obj_path, arg_map)
del obj
if isinstance(result, ConstructionIssue):
return result
params, obj_constructor, _ = result
# Ideas:
# TODO: Allow automatically accessing any attribute on parent for factories?
# This eases the responsibility of getting the right structure for the config and could mean
# we don't need to support wildcards? Reduces problems of something not getting specified
# correctly.
# "If you need a value, move it one level up."
# TODO: Allow factories to return blueprints? This would allow for better presets, e.g. you
# could do model=d4-dev model.n_layers=5
kwargs: dict[str, ParamRef] = {}
value_mapping: dict[str, Evaluatable] = {}
for param in params:
sub_value_mapping = _construct_param(
param,
obj_path,
arg_map,
all_params=all_params,
used_args=used_args,
meta_factory_value=meta_factory_value,
missing_params=missing_params,
)
if isinstance(sub_value_mapping, ConstructionIssue):
return sub_value_mapping
if sub_value_mapping is None:
continue
param_path = (obj_path + "." if obj_path else "") + param.name
value_mapping.update(sub_value_mapping)
kwargs[param.name] = ParamRef(param_path)
value_mapping[obj_path] = Thunk(obj_constructor, kwargs)
return value_mapping
def _construct_unspecified_param(
param: _Param,
*,
param_path: str,
arg_map: ArgumentMap,
# Output parameters, do not use within this function
# See _MakeResult for docs about these parameters
all_params: _WriteOnlyMapping[str, _Param],
used_args: set[tuple[str, int]],
meta_factory_value: _WriteOnlyMapping[str, Any],
missing_params: list[str],
) -> dict[str, Evaluatable] | ConstructionIssue | None:
if (
param.meta_factory is not None
and (factory := param.meta_factory.unspecified_factory()) is not None
):
assert callable(factory)
sub_all_params: dict[str, _Param] = {}
sub_missing_params: list[str] = []
sub_used_args: set[tuple[str, int]] = set()
sub_meta_factory_value: dict[str, Any] = {}
value_mapping = _construct_factory(
factory,
param_path,
arg_map,
all_params=sub_all_params,
used_args=sub_used_args,
meta_factory_value=sub_meta_factory_value,
missing_params=sub_missing_params,
)
all_params.update(sub_all_params)
used_args.update(sub_used_args) # TODO: should this be gated by use?
meta_factory_value.update(sub_meta_factory_value)
if isinstance(value_mapping, ConstructionIssue):
if param_path == "" and param.default is None:
# This is a little bit special case-y. But if we have a construction issue with
# the root param, it's much better to forward it than for the user to get an error
# about a missing required root argument.
return value_mapping
else:
thunk = value_mapping[param_path]
assert isinstance(thunk, Thunk)
# Only do this if we have subcomponents specified (including wildcards)
if thunk.kwargs:
meta_factory_value[param_path] = factory
missing_params.extend(sub_missing_params)
return value_mapping
# Alternatively, if a) we do not have a default, b) we're making a chz object
# c) we know instantiation would always work, that's fine too.
# A little special-case-y, but somewhat sane. It turns something that would
# error due to lack of default into something reasonable.
# See test_nested_all_defaults and variants
if (
param.default is None
and (
chz.is_chz(factory)
or (isinstance(factory, functools.partial) and chz.is_chz(factory.func))
)
and (
all(
p.default is not None
for path, p in sub_all_params.items()
if "." not in path.removeprefix(param_path + ".")
)
)
):
assert not sub_missing_params
meta_factory_value[param_path] = factory
return value_mapping
# If we have a default, make sure we don't extend missing_params
if param.default is None:
if sub_missing_params:
missing_params.extend(sub_missing_params)
else:
# Happens if we collect no params, like non-chz field or variadics
missing_params.append(param_path)
else:
# If we have a default, do some validation about wildcards + variadics
_check_for_wildcard_matching_variadic_top_level(factory, param, param_path, arg_map)
return None
assert not sub_missing_params
# If we have no subcomponents specified or we have no factory, we don't add any kwargs
# When the object is created, this will be equivalent to:
# `attr = default` or `attr = default_factory()`
# If there is no default, we will raise MissingBlueprintArg, instead of relying on the
# normal Python error during instantiation. We also rely on raising ExtraneousBlueprintArg
# if there are arguments that go unused.
# (In the case of Blueprint implementation bugs, if we're missing a param, __init__ will
# have our back, but the extraneous logic has no backup)
if param.default is None:
missing_params.append(param_path)
return None
def _construct_param(
param: _Param,
obj_path: str,
arg_map: ArgumentMap,
*,
# Output parameters, do not use within this function
# See _MakeResult for docs about these parameters
all_params: _WriteOnlyMapping[str, _Param],
used_args: set[tuple[str, int]],
meta_factory_value: _WriteOnlyMapping[str, Any],
missing_params: list[str],
) -> dict[str, Evaluatable] | ConstructionIssue | None:
# Returns None if we don't need to pass any value. This doesn't mean there's an error,
# we might simply want the default or default_factory value.
param_path: Final = (obj_path + "." if obj_path else "") + param.name
all_params[param_path] = param
found_arg = arg_map.get_kv(param_path)
# If nothing is specified, check if we have a factory we can feed subcomponents to and if there
# are specified subcomponents we could feed to it. Otherwise, if a default or default_factory
# exists, we'll use that.
if found_arg is None:
return _construct_unspecified_param(
param,
param_path=param_path,
arg_map=arg_map,
all_params=all_params,
used_args=used_args,
meta_factory_value=meta_factory_value,
missing_params=missing_params,
)
used_args.add((found_arg.key, found_arg.layer_index))
spec: object = found_arg.value
# Something is specified, so we must either add something to kwargs or error out
# If something is specified, and is of the expected type, we just assign it:
# `attr = spec`
if not isinstance(spec, SpecialArg) and is_subtype_instance(spec, param.type):
# (ignore SpecialArg's here, in case param.type is object)
if not (param.meta_factory is not None and arg_map.subpaths(param_path, strict=True)):
# TODO: deep copy?
return {param_path: Value(spec)}
# ..or if it's a Reference to some other parameter
if isinstance(spec, Reference):
if spec.ref == param_path:
# If it's a self reference, treat it as if it were unspecified
value_mapping = _construct_unspecified_param(
param,
param_path=param_path,
arg_map=arg_map,
all_params=all_params,
used_args=used_args,
meta_factory_value=meta_factory_value,
missing_params=missing_params,
)
if isinstance(value_mapping, ConstructionIssue):
return value_mapping
if value_mapping is None and param.default is not None:
# See test_blueprint_reference_wildcard_default
# TODO: this is the only place we instantiate a default
default = param.default.instantiate()
return {param_path: Value(default)}
return value_mapping
return {param_path: ParamRef(spec.ref)}
elif isinstance(spec, Computed):
# If it inherits from a set of other parameters
if param_path in {spec.ref for spec in spec.src.values()}:
# Same as the unspecified param case
return _construct_unspecified_param(
param,
param_path=param_path,
arg_map=arg_map,
all_params=all_params,
used_args=used_args,
meta_factory_value=meta_factory_value,
missing_params=missing_params,
)
else:
kwargs = {k: ParamRef(v.ref) for k, v in spec.src.items()}
return {param_path: Thunk(kwargs=kwargs, fn=spec.compute)}
# Otherwise, we see if we can cast it to the expected type:
# `attr = trycast(spec.value, param.type)`
if isinstance(spec, Castable):
# If we have a meta_factory and we have args that are prefixed with the param path, we
# will always want to construct that (if we successfully casted here when subcomponents
# are specified, we'd just fail later because those subcomponents would be extraneous)
if not (param.meta_factory is not None and arg_map.subpaths(param_path, strict=True)):
try:
casted_spec = param.cast(spec.value)
return {param_path: Value(casted_spec)}
except CastError:
pass
# ..or if it's a Reference to some other parameter
if isinstance(spec, Reference):
if spec.ref == param_path:
# If it's a self reference, treat it as if it were unspecified
value_mapping = _construct_unspecified_param(
param,
param_path=param_path,
arg_map=arg_map,
all_params=all_params,
used_args=used_args,
meta_factory_value=meta_factory_value,
missing_params=missing_params,
)
if isinstance(value_mapping, ConstructionIssue):
return value_mapping
if value_mapping is None and param.default is not None:
# See test_blueprint_reference_wildcard_default
# TODO: this is the only place we instantiate a default
default = param.default.instantiate()
return {param_path: Value(default)}
return value_mapping
return {param_path: ParamRef(spec.ref)}
# Otherwise, see if it's something that can construct the expected type. For instance,
# maybe it's a subclass of param.type, or more generally any `Callable[..., param.type]`,
# in which case we do:
# `attr = spec(...)`
factory: Callable[..., Any]
if is_subtype_instance(spec, Callable[..., param.type]):
assert callable(spec)
factory = spec
value_mapping = _construct_factory(
factory,
param_path,
arg_map,
all_params=all_params,
used_args=used_args,
meta_factory_value=meta_factory_value,
missing_params=missing_params,
)
if isinstance(value_mapping, ConstructionIssue):
return value_mapping
meta_factory_value[param_path] = factory
return value_mapping
# Otherwise, see if it's something that can be casted into something that can construct
# the expected type. For instance, maybe it's a string that's the name of a subclass of
# param.type or "module:func" where module.func is a `func: Callable[..., param.type]`.
# `attr = trycast(spec, constructor_type)(...)`
if isinstance(spec, Castable):
if param.meta_factory is not None:
try:
factory = param.meta_factory.from_string(spec.value)
except chz.factories.MetaFromString as e:
cast_error = None
try:
param.cast(spec.value)
except CastError as e2:
cast_error = str(e2)
if cast_error is None:
subpaths = arg_map.subpaths(param_path, strict=True)
assert subpaths
cast_error = f"Not a value, since subparameters were provided (e.g. {join_arg_path(param_path, subpaths[0])!r})"
raise InvalidBlueprintArg(
f"Could not interpret argument {spec.value!r} provided for param {param_path!r}...\n\n"
f"- Failed to interpret it as a value:\n{cast_error}\n\n"
f"- Failed to interpret it as a factory for polymorphic construction:\n{e}"
) from None
assert callable(factory)
value_mapping = _construct_factory(
factory,
param_path,
arg_map,
all_params=all_params,
used_args=used_args,
meta_factory_value=meta_factory_value,
missing_params=missing_params,
)
if isinstance(value_mapping, ConstructionIssue):
return value_mapping
meta_factory_value[param_path] = factory
return value_mapping
# This cast is just to raise the error we caught previously
try:
param.cast(spec.value)
except CastError as e:
raise InvalidBlueprintArg(
f"Could not cast {spec.value!r} to {type_repr(param.type)}:\n{e}"
) from e
# This next line should be unreachable...
raise TypeError(
f"Expected {param_path!r} to be castable to {type_repr(param.type)}, got {spec.value!r}"
)
if not isinstance(spec, SpecialArg) and is_subtype_instance(spec, param.type):
if param.meta_factory is not None:
subpaths = arg_map.subpaths(param_path, strict=True)
if subpaths:
raise InvalidBlueprintArg(
f"Could not interpret {spec!r} provided for param {param_path!r} "
f"as a value, since subparameters were provided "
f"(e.g. {join_arg_path(param_path, subpaths[0])!r})"
)
raise TypeError(
f"Expected {param_path!r} to be {type_repr(param.type)}, got {type_repr(_simplistic_type_of_value(spec))}"
)
def _check_for_wildcard_matching_variadic_top_level(
obj: object, param: _Param, obj_path: str, arg_map: ArgumentMap
):
assert param.default is not None
if (
type(param.default.value) is tuple and param.default.value == ()
) or param.default.factory in {tuple, list, dict}:
return
result = _collect_params(obj, obj_path, arg_map)
if isinstance(result, ConstructionIssue):
return
variadic_params, _, variadic_types = result
if variadic_params:
return
if isinstance(param.default.value, (tuple, list)):
variadic_types = list(
set(variadic_types) | {type(element) for element in param.default.value}
)
elif isinstance(param.default.value, dict):
variadic_types = list(
set(variadic_types) | {type(element) for element in param.default.value.values()}
)
if not variadic_types:
return
# The case we're checking here is if we:
# 1) have a param with a default
# 1.5) the default is not an empty tuple or list or dict
# 2) have a variadic factory for that param
# 3) we do not find any variadic params
# Then we check if any wildcards would have matched a param if we had one, since it can be
# unintuitive that the default will not be affected by the wildcard (default / default_factory
# are opaque and have no interaction with wildcards beyond their presence or absence).
# See test_variadic_default_wildcard_error
for element_type in variadic_types:
result = _collect_params(element_type, obj_path + ".__chz_empty_variadic", arg_map)
if isinstance(result, ConstructionIssue):
continue
subparams, _, _ = result
for subparam in subparams:
param_path = obj_path + ".__chz_empty_variadic." + subparam.name
found_arg = arg_map.get_kv(param_path)
param_path = obj_path + ".(variadic)." + subparam.name
if found_arg is not None:
raise ConstructionException(
f"\n\nYou've hit an interesting case.\n\n"
f'The parameter "{obj_path}" is variadic ({type_repr(obj)}), but no '
"parametrisation was found (either variadic subparameters or a polymorphic "
"parametrisation).\n"
f'This is fine in theory, because "{obj_path}" has a '
f"default value.\n\n"
f'However, you also specified the wildcard "{found_arg.key}" and you may '
f'have expected it to modify the value of "{param_path}".\n'
"This is not possible -- default values / default_factory results are "
"opaque to chz. "
"The only way in which default / default_factory interact with Blueprint "
"is presence / absence. So out of caution, here's an error!\n\n"
"If this error is a false positive, consider scoping the wildcard more "
"narrowly or using exact keys. As always, appending --help to a chz command "
"will show you what gets mapped to which param."
)
================================================
FILE: chz/blueprint/_entrypoint.py
================================================
from __future__ import annotations
import functools
import inspect
import io
import os
import sys
from typing import Any, Callable, TypeVar
import chz
from chz.tiepin import eval_in_context, type_repr
_T = TypeVar("_T")
_F = TypeVar("_F", bound=Callable[..., Any])
class EntrypointException(Exception): ...
class EntrypointHelpException(EntrypointException): ...
class ExtraneousBlueprintArg(EntrypointException): ...
class InvalidBlueprintArg(EntrypointException): ...
class MissingBlueprintArg(EntrypointException): ...
class ConstructionException(EntrypointException): ...
def exit_on_entrypoint_error(fn: _F) -> _F:
@functools.wraps(fn)
def inner(*args, **kwargs):
try:
return fn(*args, **kwargs)
except EntrypointException as e:
if isinstance(e, EntrypointHelpException):
print(e, end="" if e.args[0][-1] == "\n" else "\n")
else:
print("Error:", file=sys.stderr)
print(e, end="" if e.args[0][-1] == "\n" else "\n", file=sys.stderr)
if "PYTEST_VERSION" in os.environ:
raise
sys.exit(1)
return inner # type: ignore[return-value]
@exit_on_entrypoint_error
def entrypoint(
target: Callable[..., _T], *, argv: list[str] | None = None, allow_hyphens: bool = False
) -> _T:
"""Easy way to create a script entrypoint using chz.
For example, if you wish to run a function:
```
def do_something(alpha: int, beta: str, gamma: bytes) -> None:
...
if __name__ == "__main__":
chz.entrypoint(do_something)
```
It also works for instantiating objects:
```
@chz.chz
class Run:
name: str
def launch(self) -> None: ...
if __name__ == "__main__":
run = chz.entrypoint(Run)
run.launch()
```
"""
# This function should be easily forkable, so do not make it more complicated
return chz.Blueprint(target).make_from_argv(argv, allow_hyphens=allow_hyphens)
@exit_on_entrypoint_error
def nested_entrypoint(
main: Callable[[Any], _T], *, argv: list[str] | None = None, allow_hyphens: bool = False
) -> _T:
"""Easy way to create a script entrypoint using chz for functions that take a chz object.
For example:
```
@chz.chz
class Run:
name: str
def main(run: Run) -> None:
...
if __name__ == "__main__":
chz.nested_entrypoint(main)
```
Tip: If your `main` function is `async`, you can just do
`asyncio.run(chz.nested_entrypoint(main))`.
"""
# This function should be easily forkable, so do not make it more complicated
target = get_nested_target(main)
value = chz.Blueprint(target).make_from_argv(argv, allow_hyphens=allow_hyphens)
return main(value)
@exit_on_entrypoint_error
def methods_entrypoint(
target: type[_T],
*,
argv: list[str] | None = None,
transform: Callable[[chz.Blueprint[Any], Any, str], chz.Blueprint[Any]] | None = None,
) -> _T:
"""Easy way to create a script entrypoint using chz for methods on a class.
For example, given main.py:
```
@chz.chz
class Run:
name: str
def launch(self, cluster: str):
"Launch a job on a cluster"
return ("launch", self, cluster)
if __name__ == "__main__":
print(chz.methods_entrypoint(Run))
```
Try out the following command line invocations:
```
python main.py launch self.name=job cluster=owl
python main.py launch --help
python main.py --help
```
Note that you can rename the `self` argument in your method to something else.
"""
if argv is None:
argv = sys.argv[1:]
is_help = not argv or argv[0] == "--help"
is_valid = not argv or (argv[0].isidentifier() and hasattr(target, argv[0]))
if is_help or not is_valid:
f = io.StringIO()
output = functools.partial(print, file=f)
if not is_valid:
output(f"WARNING: {argv[0]} is not a valid method")
output(f"Entry point: methods of {type_repr(target)}")
output()
output("Available methods:")
for name in dir(target):
meth = getattr(target, name)
if not name.startswith("_") and callable(meth):
meth_doc = getattr(meth, "__doc__", "") or ""
meth_doc = meth_doc.strip().split("\n", 1)[0]
output(f" {name} {meth_doc}".rstrip())
raise EntrypointHelpException(f.getvalue())
blueprint = chz.Blueprint(getattr(target, argv[0]))
if transform is not None:
blueprint = transform(blueprint, target, argv[0])
return blueprint.make_from_argv(argv[1:])
@exit_on_entrypoint_error
def dispatch_entrypoint(
targets: dict[str, Callable[..., _T]], *, argv: list[str] | None = None
) -> _T:
"""Easy way to create a script entrypoint using chz for dispatching to different functions.
Conceptually, this is strictly a subset of the universal `python -m chz.universal` entrypoint.
Compared to that, or methods_entrypoint, this basically just lets you flatten args one level.
```
def say_hello(name: str) -> None:
print(f"Hello, {name}!")
def say_goodbye(name: str) -> None:
print(f"Goodbye, {name}!")
chz.dispatch_entrypoint({
"hello": say_hello,
"goodbye": say_goodbye,
})
```
"""
if argv is None:
argv = sys.argv[1:]
is_help = not argv or argv[0] == "--help"
is_valid = not argv or (argv[0].isidentifier() and argv[0] in targets)
if is_help or not is_valid:
f = io.StringIO()
output = functools.partial(print, file=f)
if not is_valid:
output(f"WARNING: {argv[0]} is not a valid entrypoint")
output("Available entrypoints:")
for name in targets:
meth = targets[name]
meth_doc = getattr(meth, "__doc__", "") or ""
meth_doc = meth_doc.strip().split("\n", 1)[0]
output(f" {name} {meth_doc}".rstrip())
raise EntrypointHelpException(f.getvalue())
return chz.Blueprint(targets[argv[0]]).make_from_argv(argv[1:])
def _resolve_annotation(annotation: Any, func: Any) -> Any:
"""Resolves a type annotation against the globals of the target function."""
if annotation is inspect.Parameter.empty:
return None
if isinstance(annotation, str):
return eval_in_context(annotation, func)
return annotation
def get_nested_target(main: Callable[[_T], object]) -> type[_T]:
"""Returns the type of the first argument of a function.
For example:
```
def main(run: Run) -> None: ...
assert chz.get_nested_target(main) is Run
```
"""
params = list(inspect.signature(main).parameters.values())
if not params or params[0].annotation == inspect.Parameter.empty:
raise ValueError("Nested entrypoints must take a type annotated argument")
if any(p.default is p.empty for p in params[1:]):
raise ValueError("Nested entrypoints must take at most one argument without a default")
return _resolve_annotation(params[0].annotation, main)
================================================
FILE: chz/blueprint/_lazy.py
================================================
import collections
from typing import AbstractSet, Any, Callable, TypeVar
from chz.blueprint._entrypoint import InvalidBlueprintArg
from chz.blueprint._wildcard import wildcard_key_approx, wildcard_key_to_regex
from chz.tiepin import type_repr
T = TypeVar("T")
class Evaluatable: ...
class Value(Evaluatable):
def __init__(self, value: Any) -> None:
self.value = value
def __repr__(self) -> str:
return f"Value({self.value})"
class ParamRef(Evaluatable):
def __init__(self, ref: str) -> None:
self.ref = ref
def __repr__(self) -> str:
return f"ParamRef({self.ref})"
class Thunk(Evaluatable):
def __init__(self, fn: Callable[..., Any], kwargs: dict[str, ParamRef]) -> None:
self.fn = fn
self.kwargs = kwargs
def __repr__(self) -> str:
return f"Thunk({type_repr(self.fn)}, {self.kwargs})"
def evaluate(value_mapping: dict[str, Evaluatable]) -> Any:
assert "" in value_mapping
refs_in_progress = collections.OrderedDict[str, None]()
def inner(ref: str) -> Any:
if ref in refs_in_progress:
cycle = " -> ".join(list(refs_in_progress.keys())[1:] + [ref])
raise RecursionError(f"Detected cyclic reference: {cycle}")
refs_in_progress[ref] = None
try:
value = value_mapping[ref]
assert isinstance(value, Evaluatable)
if isinstance(value, Value):
return value.value
if isinstance(value, ParamRef):
try:
ret = inner(value.ref)
except Exception as e:
e.add_note(f" (when dereferencing {ref!r})")
raise
assert not isinstance(ret, Evaluatable)
value_mapping[ref] = Value(ret)
return ret
if isinstance(value, Thunk):
kwargs = {}
for k, v in value.kwargs.items():
assert isinstance(v, ParamRef)
try:
kwargs[k] = inner(v.ref)
except Exception as e:
e.add_note(f" (when evaluating argument {k!r} for {type_repr(value.fn)})")
raise
ret = value.fn(**kwargs)
return ret
finally:
item = refs_in_progress.popitem()
assert item[0] == ref
raise AssertionError
return inner("")
def check_reference_targets(
value_mapping: dict[str, Evaluatable], param_paths: AbstractSet[str]
) -> None:
invalid_references: dict[str, list[str]] = {}
def record_invalid(ref: str, referrer: str) -> None:
if not referrer:
return
if ref not in param_paths:
referrers = invalid_references.setdefault(ref, [])
if referrer not in referrers:
referrers.append(referrer)
def walk(value: Evaluatable, referrer: str) -> None:
if isinstance(value, ParamRef):
record_invalid(value.ref, referrer)
elif isinstance(value, Thunk):
for param_ref in value.kwargs.values():
walk(param_ref, referrer)
for param_path, value in value_mapping.items():
walk(value, param_path)
if invalid_references:
errors = []
for reference, referrers in invalid_references.items():
ratios = {p: wildcard_key_approx(reference, p) for p in param_paths}
extra = ""
if ratios:
max_option = max(ratios, key=lambda v: ratios[v][0])
if ratios[max_option][0] > 0.1:
extra = f"\nDid you mean {ratios[max_option][1]!r}?"
nested_pattern = wildcard_key_to_regex("..." + reference)
found_key = next((p for p in param_paths if nested_pattern.fullmatch(p)), None)
if found_key is not None:
extra += f"\nDid you get the nesting wrong, maybe you meant {found_key!r}?"
if len(referrers) > 1:
referrers_str = "params " + ", ".join(referrers)
else:
referrers_str = f"param {referrers[0]}"
errors.append(f"Invalid reference target {reference!r} for {referrers_str}" + extra)
raise InvalidBlueprintArg("\n\n".join(errors))
================================================
FILE: chz/blueprint/_wildcard.py
================================================
import re
_FUZZY_SIMILARITY = 0.6
def wildcard_key_to_regex_str(key: str) -> str:
if key.endswith("..."):
raise ValueError("Wildcard not allowed at end of key")
pattern = r"(.*\.)?" if key.startswith("...") else ""
pattern += r"\.(.*\.)?".join(map(re.escape, key.removeprefix("...").split("...")))
return pattern
def wildcard_key_to_regex(key: str) -> re.Pattern[str]:
return re.compile(wildcard_key_to_regex_str(key))
def _wildcard_key_match(key: str, target_str: str) -> bool:
# This is what the regex does; currently unused (but is tested)
if key.endswith("..."):
raise ValueError("Wildcard not allowed at end of key")
pattern = ["..."] if key.startswith("...") else []
pattern += [x for x in re.split(r"(\.\.\.)|\.", key.removeprefix("...")) if x is not None]
target = target_str.split(".")
_grid: dict[tuple[int, int], bool] = {}
def _match(i: int, j: int) -> bool:
if i == len(pattern):
return j == len(target)
if j == len(target):
return False
if (i, j) in _grid:
return _grid[i, j]
if pattern[i] == "...":
ret = _match(i, j + 1) or _match(i + 1, j)
_grid[i, j] = ret
return ret
ret = pattern[i] == target[j] and _match(i + 1, j + 1)
_grid[i, j] = ret
return ret
return _match(0, 0)
def wildcard_key_approx(key: str, target_str: str) -> tuple[float, str]:
"""
Returns a score and a string representing what the key should have been to match the target.
Currently only used in error messages.
"""
if key.endswith("..."):
raise ValueError("Wildcard not allowed at end of key")
pattern = ["..."] if key.startswith("...") else []
pattern += [x for x in re.split(r"(\.\.\.)|\.", key.removeprefix("...")) if x is not None]
target = target_str.split(".")
import difflib
_grid: dict[tuple[int, int], tuple[float, tuple[str, ...]]] = {}
def _match(i, j) -> tuple[float, tuple[str, ...]]:
if i == len(pattern):
return (1, ()) if j == len(target) else (0, ())
if j == len(target):
return (0, ())
if (i, j) in _grid:
return _grid[i, j]
if pattern[i] == "...":
with_wildcard = _match(i, j + 1)
without_wildcard = _match(i + 1, j)
if with_wildcard[0] * _FUZZY_SIMILARITY > without_wildcard[0]:
score, value = with_wildcard
score *= _FUZZY_SIMILARITY
else:
score, value = without_wildcard
if value and value[0] != "...":
value = ("...",) + value
ret = (score, value)
_grid[i, j] = ret
return ret
ratio = difflib.SequenceMatcher(a=pattern[i], b=target[j]).ratio()
if ratio >= _FUZZY_SIMILARITY:
score, value = _match(i + 1, j + 1)
score *= ratio
if value and value[0] != "...":
value = (target[j] + ".",) + value
else:
value = (target[j],) + value
ret = (score, value)
_grid[i, j] = ret
return ret
return 0, ()
score, value = _match(0, 0)
return score, "".join(value)
================================================
FILE: chz/data_model.py
================================================
"""
This is the core implementation of the chz class. It's based off of the implementation of
dataclasses, but is somewhat simpler. I also fixed a couple minor issues in dataclasses when
writing this :-)
Some non-exhaustive reasons why chz's feature set isn't built on top of dataclasses / attrs:
- dataclasses is a general purpose class replacement, chz isn't. This lets us establish intention,
have better defaults, make different tradeoffs, better errors in various places
- Ability to have custom logic in chz.field
- Clearer handling of type annotation evaluation and scopes
- chz needs keyword-only arguments for various reasons (dataclasses acquired this only later)
- Cool data model tricks like munging and init_property
- Many small things
"""
import builtins
import copy
import dataclasses
import functools
import hashlib
import inspect
import sys
import types
import typing
from collections.abc import Collection, Mapping
from typing import TYPE_CHECKING, Any, Callable, Iterable, TypeVar
import typing_extensions
from chz.field import Field
from chz.tiepin import type_repr
from chz.util import MISSING
FrozenInstanceError = dataclasses.FrozenInstanceError
_T = TypeVar("_T")
_INIT_ALTERNATIVES: str = (
"For validation, see @chz.validate decorators. "
"For per-field defaults, see `default` and `default_factory` options in chz.field. "
"To perform post-initialization rewrites of field values, use `munger` option in chz.field "
"or add an `init_property` to the class.\n"
"See the docs for more details."
)
def _create_fn(
name: str, args: list[str], body: list[str], *, locals: dict[str, Any], globals: dict[str, Any]
):
args_str = ",".join(args)
body_str = "\n".join(f" {b}" for b in body)
# Compute the text of the entire function.
txt = f" def {name}({args_str}):\n{body_str}"
# Free variables in exec are resolved in the global namespace.
# The global namespace we have is user-provided, so we can't modify it for
# our purposes. So we put the things we need into locals and introduce a
# scope to allow the function we're creating to close over them.
local_vars = ", ".join(locals.keys())
txt = f"def __create_fn__({local_vars}):\n{txt}\n return {name}"
ns: Any = {}
exec(txt, globals, ns)
return ns["__create_fn__"](**locals)
# ==============================
# Method synthesis
# ==============================
def _synthesise_field_init(f: Field, out_vars: dict[str, Any]) -> tuple[str, str]:
# This function modifies out_vars
var_type = f"__chz_{f.logical_name}"
out_vars[var_type] = f._raw_type
var_default = f"__chz_dflt_{f.logical_name}"
if f._default_factory is not MISSING:
out_vars[var_default] = f._default_factory
value = f"{var_default}() if {f.logical_name} is __chz_MISSING else {f.logical_name}"
dflt_expr = " = __chz_MISSING"
elif f._default is not MISSING:
out_vars[var_default] = f._default
# Is it ever useful to explicitly pass MISSING?
# value = f"{var_default} if {f.logical_name} is __chz_MISSING else {f.logical_name}"
value = f.logical_name
dflt_expr = f" = {var_default}"
else:
value = f.logical_name
dflt_expr = ""
arg = f"{f.logical_name}: {var_type}{dflt_expr}"
body = f"__chz_builtins.object.__setattr__(self, {f.x_name!r}, {value})"
return arg, body
def _synthesise_init(fields: Collection[Field], user_globals: dict[str, Any]) -> Callable[..., Any]:
varlocals = {"__chz_MISSING": MISSING, "__chz_builtins": builtins}
# __chz_args is not strictly necessary, but makes for better errors
args = ["self", "*__chz_args"]
body = [
"if __chz_args:",
" raise __chz_builtins.TypeError(f'{self.__class__.__name__}.__init__ only takes keyword arguments')",
"if '__chz_fields__' not in __chz_builtins.type(self).__dict__:",
" raise __chz_builtins.TypeError(f'{self.__class__.__name__} is not decorated with @chz.chz')",
]
for field in fields:
if field.logical_name.startswith("__chz") or field.logical_name == "self":
raise ValueError(f"Field name {field.logical_name!r} is reserved")
_arg, _body = _synthesise_field_init(field, varlocals)
args.append(_arg)
body.append(_body)
# Note it's important we validate before we check all init_property
body.append("self.__chz_validate__()")
body.append("self.__chz_init_property__()")
return _create_fn("__init__", args, body, locals=varlocals, globals=user_globals)
def __setattr__(self, name, value):
raise FrozenInstanceError(f"Cannot modify field {name!r}")
def __delattr__(self, name):
raise FrozenInstanceError(f"Cannot delete field {name!r}")
def _recursive_repr(user_function):
import threading
repr_running = set()
@functools.wraps(user_function)
def wrapper(self):
key = id(self), threading.get_ident()
if key in repr_running:
return "..."
repr_running.add(key)
try:
result = user_function(self)
finally:
repr_running.discard(key)
return result
return wrapper
def __repr__(self) -> str:
def field_repr(field: Field) -> str:
# use x_name so that repr can be copy-pasted to create the same object
if callable(field._repr):
return field._repr(getattr(self, field.x_name))
assert isinstance(field._repr, bool)
if field._repr:
return repr(getattr(self, field.x_name))
return "..."
contents = ", ".join(
f"{field.logical_name}={field_repr(field)}" for field in self.__chz_fields__.values()
)
return self.__class__.__qualname__ + f"({contents})"
def __eq__(self, other):
if self.__class__ is not other.__class__:
return NotImplemented
return all(getattr(self, name) == getattr(other, name) for name in self.__chz_fields__)
def __hash__(self) -> int:
try:
return hash(tuple((name, getattr(self, name)) for name in self.__chz_fields__))
except TypeError as e:
for name in self.__chz_fields__:
value = getattr(self, name)
try:
hash(value)
except TypeError:
raise TypeError(
f"Cannot hash chz field: {type(self).__name__}.{name}={value}"
) from e
raise e
def __chz_validate__(self) -> None:
for field in self.__chz_fields__.values():
if field._munger is None:
for validator in field._validator:
# So without mungers, we always run validators against the raw value
# There is currently code that relies on not running validator against a potential
# user-specified init_property
# TODO: is it unfortunate that x_name appears in error messages?
validator(self, field.x_name)
else:
# With mungers, we run validators against both the munged and unmunged value
# I'm willing to reconsider this, but want to be conservative for now
for validator in field._validator:
validator(self, field.logical_name)
validator(self, field.x_name)
for validator in getattr(self, "__chz_validators__", []):
validator(self)
@functools.lru_cache()
def _get_init_properties(cls: type) -> list[str]:
return [
name
for name, _obj in inspect.getmembers_static(cls, lambda o: isinstance(o, init_property))
]
def __chz_init_property__(self) -> None:
for name in _get_init_properties(self.__class__):
getattr(self, name)
def pretty_format(obj: Any, colored: bool = True) -> str:
"""Format a chz object for human readability."""
bold = "\033[1m" if colored else ""
blue = "\033[34m" if colored else ""
grey = "\033[90m" if colored else ""
reset = "\033[0m" if colored else ""
space = " " * 4
if isinstance(obj, (list, tuple)):
if not obj or all(not is_chz(x) for x in obj):
return repr(obj)
a, b = ("[", "]") if isinstance(obj, list) else ("(", ")")
items = [pretty_format(x, colored).replace("\n", "\n" + space) for x in obj]
items_str = f",\n{space}".join(items)
return f"{a}\n{space}{items_str},\n{b}"
if isinstance(obj, dict):
if not obj or all(not is_chz(x) for x in obj.values()):
return repr(obj)
items = []
for k, v in obj.items():
k_str = pretty_format(k, colored).replace("\n", "\n" + space)
v_str = pretty_format(v, colored).replace("\n", "\n" + space)
items.append(f"{k_str}: {v_str}")
items_str = f",\n{space}".join(items)
return f"{{\n{space}{items_str},\n}}"
if not is_chz(obj):
return repr(obj)
cls_name = obj.__class__.__qualname__
out = f"{bold}{cls_name}({reset}\n"
def field_repr(field: Field) -> str:
# use x_name so that repr can be copy-pasted to create the same object
if field._repr is False:
return "..."
if callable(field._repr):
r = field._repr
else:
assert field._repr is True
r = lambda o: pretty_format(o, colored=colored)
x_val = getattr(obj, field.x_name)
val = getattr(obj, field.logical_name)
if x_val is val:
return r(val)
return f"{grey}{r(x_val)} # {reset}{r(val)}{grey} (after init){reset}"
field_reprs: dict[bool, list[str]] = {True: [], False: []}
for field in sorted(obj.__chz_fields__.values(), key=lambda f: f.logical_name):
if field._default is not MISSING:
matches_default = field._default is getattr(obj, field.x_name)
elif field._default_factory is not MISSING:
matches_default = field._default_factory() == getattr(obj, field.x_name)
else:
matches_default = False
val_str = field_repr(field).replace("\n", "\n" + space)
field_str = f"{space}{blue}{field.logical_name}={reset}{val_str},\n"
field_reprs[matches_default].append(field_str)
out += "".join(field_reprs[False])
if field_reprs[True]:
out += f"{space}{bold}# Fields where pre-init value matches default:{reset}\n"
out += "".join(field_reprs[True])
out += f"{bold}){reset}"
return out
def _repr_pretty_(self, p, cycle: bool) -> None:
# for nice ipython printing
p.text(pretty_format(self))
def __chz_pretty__(self, colored: bool = True) -> str:
"""Print a chz object for human readability."""
return pretty_format(self, colored=colored)
# ==============================
# Construction
# ==============================
def _is_classvar_annotation(annot: str | Any) -> bool:
if isinstance(annot, str):
# TODO: use better dataclass logic?
return annot.startswith(("typing.ClassVar[", "ClassVar["))
return annot is typing.ClassVar or (
type(annot) is typing._GenericAlias # type: ignore[attr-defined]
and annot.__origin__ is typing.ClassVar
)
def _is_property_like(obj: Any) -> bool:
# TODO: the semantics implied here could be more crisply defined and maybe generalised to
# more descriptors
return isinstance(obj, (property, init_property, functools.cached_property))
def chz_make_class(cls, version: str | None, typecheck: bool | None) -> type:
if cls.__class__ is not type:
if cls.__class__ is typing._ProtocolMeta:
if typing_extensions.is_protocol(cls):
raise TypeError("chz class cannot itself be a Protocol)")
else:
import abc
if cls.__class__ is not abc.ABCMeta:
raise TypeError("Cannot use custom metaclass")
user_module = cls.__module__
cls_annotations = typing_extensions.get_annotations(cls)
fields: dict[str, Field] = {}
# Collect fields from parent classes
for b in reversed(cls.__mro__):
if hasattr(b, "__dataclass_fields__"):
raise ValueError("Cannot mix chz with dataclasses")
# Only process classes that have been processed by our decorator
base_fields: dict[str, Field] | None = getattr(b, "__chz_fields__", None)
if base_fields is None:
continue
for f in base_fields.values():
if (
f.logical_name in cls.__dict__
and f.logical_name not in cls_annotations
and not _is_property_like(getattr(cls, f.logical_name))
):
# Do an LSP check against parent fields (for non-property-like members)
raise ValueError(
f"Cannot override field {f.logical_name!r} with a non-field member; "
f"maybe you're missing a type annotation?"
)
else:
fields[f.logical_name] = f
# Collect fields from the current class
for name, annotation in cls_annotations.items():
if _is_classvar_annotation(annotation):
continue
# Find the field specification from the class __dict__
value = cls.__dict__.get(name, MISSING)
if value is MISSING:
field = Field(name=name, raw_type=annotation)
elif isinstance(value, Field):
field = value
field._name = name
field._raw_type = annotation
delattr(cls, name)
else:
if _is_property_like(value) or (
isinstance(value, types.FunctionType)
and value.__name__ != "<lambda>"
and value.__qualname__.startswith(cls.__qualname__)
):
# It's problematic to redefine the field in the same class, because it means we
# lose any field specification or default value
raise ValueError(f"Field {name!r} is clobbered by {type_repr(type(value))}")
field = Field(name=name, raw_type=annotation, default=value)
delattr(cls, name)
field._user_module = user_module
# Do a basic LSP check for new fields
parent_value = getattr(cls, name, MISSING) # note the delattr above
if parent_value is not MISSING and not (
field.logical_name in fields and isinstance(parent_value, init_property)
):
raise ValueError(
f"Cannot define field {name!r} because it conflicts with something defined on a "
f"superclass: {parent_value!r}"
)
other_name = field.logical_name if name != field.logical_name else field.x_name
parent_value = getattr(cls, other_name, MISSING)
if (
parent_value is not MISSING
and not (field.logical_name in fields and isinstance(parent_value, init_property))
and other_name not in cls.__dict__
):
raise ValueError(
f"Cannot define field {name!r} because it conflicts with something defined on a "
f"superclass: {parent_value!r}"
)
if (
name == field.logical_name
and name not in cls.__dict__
and name in fields
and fields[name]._name != name
):
raise ValueError(
"I'm a little unsure of what the semantics should be here. "
"See test_conflicting_superclass_x_field_in_base. "
"Please let @shantanu know if you hit this. "
f"You can also just rename the field in the subclass to X_{name}."
)
# Create a default init_property for the field that accesses the raw X_ field
munger: Any = field.get_munger()
if munger is not None:
if field.logical_name in cls.__dict__:
raise ValueError(
f"Cannot define {field.logical_name!r} in class when the associated field "
f"has a munger"
)
munger.__name__ = field.logical_name
munger = init_property(munger)
munger.__set_name__(cls, field.logical_name)
setattr(cls, field.logical_name, munger)
if (
# but don't clobber existing definitions...
field.logical_name not in cls.__dict__ # ...if something is already there in class
and field.logical_name not in fields # ...if a parent has defined the field
):
fn: Any = lambda self, x_name=field.x_name: getattr(self, x_name)
fn.__name__ = field.logical_name
fn = init_property(fn)
fn.__set_name__(cls, field.logical_name)
setattr(cls, field.logical_name, fn)
fields[field.logical_name] = field
for name, value in cls.__dict__.items():
if isinstance(value, Field) and name not in cls_annotations:
raise TypeError(f"{name!r} has no type annotation")
# Mark the class as having been processed by our decorator
cls.__chz_fields__ = fields
if "__init__" in cls.__dict__:
raise ValueError("Cannot define __init__ on a chz class. " + _INIT_ALTERNATIVES)
if "__post_init__" in cls.__dict__:
raise ValueError("Cannot define __post_init__ on a chz class. " + _INIT_ALTERNATIVES)
cls.__init__ = _synthesise_init(fields.values(), sys.modules[user_module].__dict__)
cls.__init__.__qualname__ = f"{cls.__qualname__}.__init__"
cls.__chz_validate__ = __chz_validate__
cls.__chz_init_property__ = __chz_init_property__
if "__setattr__" in cls.__dict__:
raise ValueError("Cannot define __setattr__ on a chz class")
cls.__setattr__ = __setattr__
if "__delattr__" in cls.__dict__:
raise ValueError("Cannot define __delattr__ on a chz class")
cls.__delattr__ = __delattr__
if "__repr__" not in cls.__dict__:
cls.__repr__ = __repr__
if "__eq__" not in cls.__dict__:
cls.__eq__ = __eq__
if "__hash__" not in cls.__dict__:
cls.__hash__ = __hash__
if "_repr_pretty_" not in cls.__dict__:
# Special-cased by IPython
cls._repr_pretty_ = _repr_pretty_
if "__chz_pretty__" not in cls.__dict__:
cls.__chz_pretty__ = __chz_pretty__
if version is not None:
import json
# Hash all the fields and check the version matches
expected_version = version.split("-")[0]
key = [f.versioning_key() for f in sorted(fields.values(), key=lambda f: f.x_name)]
key_bytes = json.dumps(key, separators=(",", ":")).encode()
actual_version = hashlib.sha1(key_bytes).hexdigest()[:8]
if actual_version != expected_version:
raise ValueError(f"Version {version!r} does not match {actual_version!r}")
if typecheck is not None:
import chz.validators as chzval
if typecheck:
chzval._ensure_chz_validators(cls)
if chzval._decorator_typecheck not in cls.__chz_validators__:
cls.__chz_validators__.append(chzval._decorator_typecheck)
else:
if chzval._decorator_typecheck in getattr(cls, "__chz_validators__", []):
raise ValueError("Cannot disable typecheck; all validators are inherited")
return cls
# ==============================
# is_chz
# ==============================
def is_chz(c: object) -> bool:
"""Check if an object is a chz object."""
return hasattr(c, "__chz_fields__")
# ==============================
# __chz_fields__
# ==============================
def chz_fields(c: object) -> dict[str, Field]:
return c.__chz_fields__ # type: ignore[attr-defined]
# ==============================
# replace
# ==============================
def replace(obj: _T, /, **changes) -> _T:
"""Return a new object replacing specified fields with new values.
Example:
```
@chz.chz
class Foo:
a: int
b: str
foo = Foo(a=1, b="hello")
assert chz.replace(foo, a=101) == Foo(a=101, b="hello")
```
This just constructs a new object, so for example, the generated `__init__` gets run and
validation will work exactly as if you manually constructed the new object.
"""
if not hasattr(obj, "__chz_fields__"):
raise ValueError(f"{obj} is not a chz object")
for field in obj.__chz_fields__.values():
if field.logical_name not in changes:
changes[field.logical_name] = getattr(obj, field.x_name)
return obj.__class__(**changes)
# ==============================
# asdict
# ==============================
def asdict(
obj: object,
shallow: bool = False,
include_type: bool = False,
exclude: Collection[str] | None = None,
) -> dict[str, Any]:
"""Recursively convert a chz object to a dict.
This works similarly to dataclasses.asdict. Note no computed properties will be included
in the output.
See also: beta_to_blueprint_values
Args:
- shallow: if True, only take shallow copies of inner values. Otherwise,
deep copies are made.
- include_type: If True, include the type of the object in the output dict for each
chz object. Useful for debugging and identity.
- exclude: Iterable of field names to omit from the resulting dict at the top level.
"""
exclude_set = set(exclude) if exclude is not None else None
def inner(x: Any, current_exclude: Collection[str] | None = None):
if hasattr(x, "__chz_fields__"):
result = {
k: inner(getattr(x, k))
for k in x.__chz_fields__
if not current_exclude or k not in current_exclude
}
if include_type:
result["__chz_type__"] = type_repr(type(x))
return result
if isinstance(x, dict):
return {k: inner(v) for k, v in x.items()}
if isinstance(x, list):
return [inner(x) for x in x]
if isinstance(x, tuple):
return tuple(inner(x) for x in x)
if shallow:
return x
else:
return copy.deepcopy(x)
if not hasattr(obj, "__chz_fields__"):
raise RuntimeError(f"{obj} is not a chz object")
result = inner(obj, exclude_set)
assert type(result) is dict
return result
def traverse(obj: Any, obj_path: str = "") -> Iterable[tuple[str, Any]]:
"""Traverses the chz object and yields (path, value) pairs for all sub attributes recursively."""
assert is_chz(obj)
yield obj_path, obj
for f in obj.__chz_fields__.values():
value = getattr(obj, f.logical_name)
field_path = f"{obj_path}.{f.logical_name}" if obj_path else f.logical_name
yield field_path, value
if is_chz(value):
yield from traverse(value, field_path)
if isinstance(value, Mapping):
for k, v in value.items():
if is_chz(v):
yield from traverse(v, f"{field_path}.{k}")
else:
yield f"{field_path}.{k}", v
elif isinstance(value, (list, tuple)):
for i, v in enumerate(value):
if is_chz(v):
yield from traverse(v, f"{field_path}.{i}")
else:
yield f"{field_path}.{i}", v
# ==============================
# beta_to_blueprint_values
# ==============================
def beta_to_blueprint_values(obj, skip_defaults: bool = False) -> Any:
"""Return a dict which can be used to recreate the same object via blueprint.
Args:
- obj: The object to convert to blueprint values.
- skip_defaults: If True, fields whose values are equal to their default values will be
omitted from the output dict. If False (default), all fields are included.
Example:
```
@chz.chz
class Foo:
a: int
b: str
foo = Foo(a=1, b="hello")
assert chz.Blueprint(Foo).apply(chz.beta_to_blueprint_values(foo)).make() == foo
```
See also: asdict
"""
blueprint_values = {}
def join_arg_path(parent: str, child: str) -> str:
if not parent:
return child
if child.startswith("."):
return parent + child
return parent + "." + child
def inner(obj: Any, path: str):
if hasattr(obj, "__chz_fields__"):
for field_name, field_info in obj.__chz_fields__.items():
value = getattr(obj, field_info.x_name)
if skip_defaults and field_info._default == value:
continue
param_path = join_arg_path(path, field_name)
if field_info.meta_factory is not None and (
type(value)
is not typing.get_origin(field_info.meta_factory.unspecified_factory())
):
# Try to detect when we have used polymorphic construction
blueprint_values[param_path] = type(value)
inner(value, param_path)
elif (
isinstance(obj, dict)
and all(isinstance(k, str) for k in obj.keys())
and any(is_chz(v) for v in obj.values())
):
for k, v in obj.items():
param_path = join_arg_path(path, k)
blueprint_values[param_path] = type(v) # may be overridden if not needed
inner(v, param_path)
elif isinstance(obj, (list, tuple)) and any(is_chz(v) for v in obj):
for i, v in enumerate(obj):
param_path = join_arg_path(path, str(i))
blueprint_values[param_path] = type(v) # may be overridden if not needed
inner(v, param_path)
else:
blueprint_values[path] = obj
inner(obj, "")
return blueprint_values
# ==============================
# init_property
# ==============================
if TYPE_CHECKING:
init_property = functools.cached_property
else:
class init_property:
# Simplified and pickleable version of Python 3.8's cached_property
# It's important that this remains a non-data descriptor
def __init__(self, func: Callable[..., Any]) -> None:
self.func = func
self.name = None
def __set_name__(self, owner, name):
self.name = name
# Basically just validation
func_name = self.func.__name__
if (
name != func_name
and func_name != "<lambda>"
# TODO: remove figure out why mini needs name mangling
and not func_name.endswith("__register_chz_has_state")
):
raise ValueError("Are you doing something weird with init_property?")
def __get__(self, obj: Any, cls: Any) -> Any:
if obj is None:
return self
ret = self.func(obj)
if self.name is not None:
obj.__dict__[self.name] = ret
return ret
================================================
FILE: chz/factories.py
================================================
import ast
import collections
import functools
import importlib
import re
import sys
import types
import typing
from typing import Any, Callable
import typing_extensions
from chz.tiepin import (
CastError,
InstantiableType,
TypeForm,
_simplistic_try_cast,
eval_in_context,
is_instantiable_type,
is_subtype,
is_subtype_instance,
is_union_type,
type_repr,
)
from chz.util import MISSING, MISSING_TYPE
class MetaFromString(Exception): ...
class MetaFactory:
"""
A metafactory represents a set of possible factories, where a factory is a callable that can
give us a value of a given type.
This is the heart of polymorphic construction in chz. The idea is that when instantiating
Blueprints, you should be able to not only specify the arguments to whatever is being
constructed, but also specify what the thing to be constructed is!
In other words, when constructing a value, chz lets you specify the factory to produce it,
in addition to the arguments to pass to that factory.
In other other words, many tools will let you construct an X by specifying `...` to feed to
`X(...)`. But chz lets you construct an X by specifying both callee and arguments in `...(...)`
This concept is a little tricky, but it's fairly intuitive when you actually use it.
Consider looking at the docstring for `subclass` for a concrete example.
"""
def __init__(self) -> None:
# Set by chz.Field
self.field_annotation: TypeForm | MISSING_TYPE = MISSING
self.field_module: types.ModuleType | str | MISSING_TYPE = MISSING
def unspecified_factory(self) -> Callable[..., Any] | None:
"""The default callable to use to get a value of the expected type.
If this returns None, there is no default. In order to construct a value of the expected
type, the user must explicitly specify a factory.
"""
raise NotImplementedError
def from_string(self, factory: str) -> Callable[..., Any]:
"""The callable that best corresponds to `factory`."""
raise NotImplementedError
def perform_cast(self, value: str):
# TODO: maybe make this default to:
# return _simplistic_try_cast(value, default_target)
raise NotImplementedError
class subclass(MetaFactory):
"""
ATTN: this is soft deprecated, since `chz.factories.standard` is powerful enough to effectively
do a superset of this.
Read the docstring for MetaFactory first.
```
@chz.chz
class Experiment:
model: Model
```
In the above example, we want to construct a value for the model for our experiment.
How should we go about making a model?
The meta_factory we provide is what is meant to answer this question. And in this case, the
answer we want is: we should make a model by attempting to instantiate `Model` or some subclass
of `Model`.
This is a common enough answer that chz in fact defaults to it. That is, here chz will
set the meta_factory to be `subclass(base_cls=Model, default_cls=Model)` for our model field.
See the logic in chz.Field.
Given `model=Transformer model.n_layers=16 model.d_model=1024`
chz will construct `Transformer(n_layers=16, d_model=1024`
That is, if the user specifies a factory for the model field, e.g. model="Transformer", then
the logic in `subclass.from_string` will attempt to find a subclass of `Model` (the `base_cls`)
named `Transformer` and instantiate it.
Given `model.n_layers=16 model.d_model=1024`
chz will construct `Model(n_layers=Y, d_model=Z)`
That is, if the user doesn't specify a factory (maybe they only specify subcomponents, like
`model.n_layers=16`), then we will default to trying to instantiate `Model` (the `default_cls`).
"""
def __init__(
self,
base_cls: InstantiableType | MISSING_TYPE = MISSING,
default_cls: InstantiableType | MISSING_TYPE = MISSING,
) -> None:
super().__init__()
self._base_cls = base_cls
self._default_cls = default_cls
def __repr__(self) -> str:
return f"subclass(base_cls={self.base_cls!r}, default_cls={self.default_cls!r})"
@property
def base_cls(self) -> InstantiableType:
if isinstance(self._base_cls, MISSING_TYPE):
assert not isinstance(self.field_annotation, MISSING_TYPE)
if not isinstance(self.field_annotation, InstantiableType):
raise RuntimeError(
f"Must explicitly specify base_cls since {self.field_annotation!r} "
"is not an instantiable type"
)
return self.field_annotation
return self._base_cls
@property
def default_cls(self) -> InstantiableType:
if isinstance(self._default_cls, MISSING_TYPE):
return self.base_cls
return self._default_cls
def unspecified_factory(self) -> Callable[..., Any]:
return self.default_cls # type: ignore[return-value]
def from_string(self, factory: str) -> Callable[..., Any]:
"""
If factory=module:cls, import module and return cls.
If factory=cls, do our best to find a subclass of base_cls named cls.
"""
return _find_subclass(factory, self.base_cls)
def perform_cast(self, value: str):
try:
return _simplistic_try_cast(value, self.default_cls)
except CastError:
pass
return _simplistic_try_cast(value, self.base_cls)
class function(MetaFactory):
def __init__(
self,
unspecified: Callable[..., Any] | None = None,
*,
default_module: str | types.ModuleType | None | MISSING_TYPE = MISSING,
) -> None:
"""
ATTN: this is soft deprecated, since `chz.factories.standard` is powerful enough to effectively
do a superset of this.
Read the docstring for `MetaFactory` and `subclass` first.
If you specify `function` as your meta_factory, any function can serve as a factory to
construct a value of the expected type.
```
def wikipedia_text(seed: int) -> Dataset: ...
@chz.chz
class Experiment:
dataset: Dataset = field(meta_factory=function())
```
In the above example, we want to construct a value of type `Dataset` for our experiment.
The way we want to do this is by calling some function that returns a `Dataset`.
Given `dataset=wikipedia_text dataset.seed=217`
chz will construct `wikipedia_text(seed=217)`.
If you use a fully qualified name like `function=module:fn` it's obvious where to find the
function. Otherwise, chz looks for an appropriately named function in the module
`default_module` (which defaults to the module in which the chz class was defined).
If you love `wikipedia_text` and you don't wish to explicitly specify
`dataset=wikipedia_text` every time, set the `unspecified` argument to be `wikipedia_text`.
This way, chz will default to trying to call `wikipedia_text` to instantiate a value of type
`Dataset`, instead of erroring because it doesn't know what factory to use to produce a
Dataset.
"""
super().__init__()
self.unspecified = unspecified
self._default_module = default_module
def __repr__(self) -> str:
return f"function(unspecified={self.unspecified!r}, default_module={self.default_module!r})"
@property
def default_module(self) -> types.ModuleType | str | None:
if isinstance(self._default_module, MISSING_TYPE):
assert not isinstance(self.field_module, MISSING_TYPE)
return self.field_module
return self._default_module
def unspecified_factory(self) -> Callable[..., Any] | None:
return self.unspecified
def from_string(self, factory: str) -> Callable[..., Any]:
"""
If factory=module:fn, import module and return fn.
If factory=fn, look in the default module for a function named fn.
"""
if ":" not in factory:
if self.default_module is None:
raise MetaFromString(
f"No module specified in {factory!r} and no default module specified"
)
if isinstance(self.default_module, str):
module = importlib.import_module(self.default_module)
else:
module = self.default_module
var = factory
else:
module_name, var = factory.split(":", 1)
if module_name != "lambda" and not module_name.startswith("lambda "):
module = _module_from_name(module_name)
else:
import ast
if isinstance(self.default_module, str):
eval_ctx = importlib.import_module(self.default_module)
elif self.default_module is not None:
eval_ctx = self.default_module
else:
eval_ctx = None
try:
# TODO: add docs for this branch
if isinstance(ast.parse(factory).body[0].value, ast.Lambda): # type: ignore[attr-defined]
return eval_in_context(factory, eval_ctx)
except Exception as e:
raise MetaFromString(
f"Could not interpret {factory!r} as a function: {e}"
) from None
raise AssertionError
return _module_getattr(module, var)
def perform_cast(self, value: str):
assert not isinstance(self.field_annotation, MISSING_TYPE)
return _simplistic_try_cast(value, self.field_annotation)
def _module_from_name(name: str) -> types.ModuleType:
try:
return importlib.import_module(name)
except ImportError as e:
raise MetaFromString(
f"Could not import module {name!r} ({type(e).__name__}: {e})"
) from None
def _module_getattr(mod: types.ModuleType, attr: str) -> Any:
try:
for a in attr.split("."):
mod = getattr(mod, a)
return mod
except AttributeError as e:
raise MetaFromString(str(e)) from None
def _find_subclass(spec: str, superclass: TypeForm):
module_name = None
if ":" in spec:
module_name, var = spec.split(":", 1)
else:
var = spec
match = re.fullmatch(r"(?P<base>[^\s\[\]]+)(\[(?P<generic>.+)\])?", var)
if match is None:
raise MetaFromString(f"Failed to parse '{spec}' as a class name")
base = match.group("base")
generic = match.group("generic")
if module_name is None and not base.isidentifier():
if "." in base:
# This effectively adds some basic support for module.symbol, not just module:symbol
module_name, base = base.rsplit(".", 1)
if not base.isidentifier():
raise MetaFromString(
f"No subclass of {type_repr(superclass)} named {base!r} (invalid identifier)"
)
if module_name is not None:
module = _module_from_name(module_name)
# TODO: think about this type ignore
value = _maybe_generic(
_module_getattr(module, base),
generic,
template=superclass, # type: ignore[arg-type]
)
if is_subtype(value, superclass):
return value
raise MetaFromString(
f"Expected a subtype of {type_repr(superclass)}, got {type_repr(value)}"
)
superclass_class_origin = getattr(superclass, "__origin__", superclass)
if superclass_class_origin in {object, typing.Any, typing_extensions.Any}:
try:
return _maybe_generic(
_module_getattr(_module_from_name("__main__"), base),
generic,
template=superclass, # type: ignore[arg-type]
)
except MetaFromString:
pass
try:
return _maybe_generic(
_module_getattr(_module_from_name("builtins"), base),
generic,
template=superclass, # type: ignore[arg-type]
)
except MetaFromString:
pass
raise MetaFromString(
f"Could not find {spec!r}, try a fully qualified name e.g. module_name:{spec}"
) from None
if not is_instantiable_type(superclass_class_origin):
raise MetaFromString(f"Could not find subclasses of {type_repr(superclass)}")
assert superclass_class_origin is not type
visited_subclasses = set()
all_subclasses = collections.deque(superclass_class_origin.__subclasses__())
all_subclasses.appendleft(superclass)
candidates = []
while all_subclasses:
cls = all_subclasses.popleft()
if cls in visited_subclasses:
continue
visited_subclasses.add(cls)
if cls.__name__ == base:
assert module_name is None
candidates.append(_maybe_generic(cls, generic, template=superclass)) # type: ignore[arg-type]
cls_origin = getattr(cls, "__origin__", cls)
assert cls_origin is not type
all_subclasses.extend(cls_origin.__subclasses__())
if len(candidates) == 0:
raise MetaFromString(f"No subclass of {type_repr(superclass)} named {base!r}")
if len(candidates) > 1:
raise MetaFromString(
f"Multiple subclasses of {type_repr(superclass)} named {base!r}: "
f"{', '.join(type_repr(c) for c in candidates)}"
)
return candidates[0]
def _maybe_generic(
cls: type, generic: str | None, template: InstantiableType
) -> Callable[..., Any]:
if generic is None:
return cls
assert isinstance(generic, str)
generic_args_str = generic.split(",")
args: list[object] = []
for i, arg_str in enumerate(generic_args_str):
arg_str = arg_str.strip()
if ":" in arg_str:
module_name, arg = arg_str.split(":", 1)
args.append(_module_getattr(_module_from_name(module_name), arg))
elif arg_str == "...":
args.append(...)
else:
# TODO: note this assumes covariance, also give a better error
superclass = template.__args__[i] # type: ignore[union-attr]
args.append(_find_subclass(arg_str, superclass))
origin: Any = getattr(cls, "__origin__", cls)
return origin[*args]
def _return_prospective(obj: Any, annotation: TypeForm, factory: str) -> Any:
if annotation not in {
object,
typing.Any,
typing_extensions.Any,
} and not isinstance(annotation, typing.TypeVar):
if is_subtype_instance(obj, annotation):
# Allow things to be instances!
# In some sense, this is just working around deficiencies in casting...
return lambda: obj
elif not callable(obj):
assert is_subtype_instance(obj, annotation)
# Also allow things to be instances if we would just error on the next line
return lambda: obj
if not callable(obj):
raise MetaFromString(f"Expected {obj} from {factory!r} to be callable")
if isinstance(obj, type) and not is_subtype(obj, annotation):
extra = ""
if getattr(annotation, "__module__", None) == "__main__":
if any(
hasattr(sys.modules["__main__"], (witness := parent).__name__)
for parent in obj.__mro__
):
extra = f" (there may be confusion between {type_repr(witness)} and __main__:{witness.__name__})"
raise MetaFromString(
f"Expected {type_repr(obj)} from {factory!r} to be a subtype of {type_repr(annotation)}{extra}"
)
return obj
def get_unspecified_from_annotation(annotation: TypeForm) -> Callable[..., Any] | None:
if typing.get_origin(annotation) is type:
base_type = typing.get_args(annotation)[0]
if is_union_type(base_type):
# No unspecified for type[SpecialForm] e.g. type[int | str]
# TODO: annotated
return None
return type[base_type] # type: ignore[return-value]
if is_union_type(annotation):
type_args = typing.get_args(annotation)
if type_args and len(type_args) == 2 and type(None) in type_args:
unwrapped_optional = [t for t in type_args if t is not type(None)][0]
if callable(unwrapped_optional):
return unwrapped_optional
return None
if is_instantiable_type(annotation):
return annotation # type: ignore[return-value]
if annotation is None:
return lambda: None
# Probably a special form
return None
class standard(MetaFactory):
def __init__(
self,
*,
annotation: TypeForm | MISSING_TYPE = MISSING,
unspecified: Callable[..., Any] | None = None,
default_module: str | types.ModuleType | None | MISSING_TYPE = MISSING,
) -> None:
super().__init__()
self._annotation = annotation
self.original_unspecified = unspecified
self._default_module = default_module
def __repr__(self) -> str:
return f"standard(annotation={self.annotation!r}, unspecified={self.original_unspecified!r}, default_module={self.default_module!r})"
@property
def annotation(self) -> TypeForm:
if isinstance(self._annotation, MISSING_TYPE):
assert not isinstance(self.field_annotation, MISSING_TYPE)
return self.field_annotation
return self._annotation
@property
def default_module(self) -> types.ModuleType | str | None:
if isinstance(self._default_module, MISSING_TYPE):
if isinstance(self.field_module, MISSING_TYPE):
# TODO: maybe make this assert and make artificial use cases pass a value explicitly
return None
return self.field_module
if isinstance(self._default_module, str):
return _module_from_name(self._default_module)
return self._default_module
@functools.cached_property
def computed_unspecified(self) -> Callable[..., Any] | None:
return (
get_unspecified_from_annotation(self.annotation)
if self.original_unspecified is None
else self.original_unspecified
)
def unspecified_factory(self) -> Callable[..., Any] | None:
if (
self.computed_unspecified is not None
and typing.get_origin(self.computed_unspecified) is type
and typing.get_args(self.computed_unspecified)
):
base_type = typing.get_args(self.computed_unspecified)[0]
# TODO: remove special handling here and elsewhere by moving logic to collect_params
return lambda: base_type
return self.computed_unspecified
def from_string(self, factory: str) -> Callable[..., Any]:
if ":" in factory:
module_name, var = factory.split(":", 1)
# fun lambda case
# TODO: add docs for fun lambda case
if module_name == "lambda" or module_name.startswith("lambda "):
default_module = self.default_module
if isinstance(default_module, MISSING_TYPE) or default_module is None:
eval_ctx = None
else:
eval_ctx = default_module
try:
if isinstance(ast.parse(factory).body[0].value, ast.Lambda): # type: ignore[attr-defined]
return eval_in_context(factory, eval_ctx)
except Exception as e:
raise MetaFromString(
f"Could not interpret {factory!r} as a function: {e}"
) from None
raise AssertionError
# we've just got something explicitly specified
module = _module_from_name(module_name)
match = re.fullmatch(r"(?P<base>[^\s\[\]]+)(\[(?P<generic>.+)\])?", var)
if match is None:
raise MetaFromString(f"Failed to parse {factory!r} as a class name")
base = match.group("base")
generic = match.group("generic")
# TODO: think about this type ignore
typ = _maybe_generic(_module_getattr(module, base), generic, template=self.annotation) # type: ignore[arg-type]
return _return_prospective(typ, self.annotation, factory=factory)
try:
if self.annotation in {object, typing.Any, typing_extensions.Any}:
return _find_subclass(factory, self.annotation)
if typing.get_origin(self.annotation) is type:
base_type = typing.get_args(self.annotation)[0]
assert isinstance(base_type, type)
typ = _find_subclass(factory, base_type)
return lambda: typ
if is_union_type(self.annotation):
if self.original_unspecified is not None:
try:
if is_instantiable_type(self.original_unspecified):
return _find_subclass(factory, self.original_unspecified)
except MetaFromString:
pass
for t in typing.get_args(self.annotation):
try:
if is_instantiable_type(t):
return _find_subclass(factory, t)
except MetaFromString:
pass
if type(None) in typing.get_args(self.annotation) and factory == "None":
return lambda: None
raise MetaFromString(f"Could not produce a union instance from {factory!r}")
if is_instantiable_type(self.annotation):
return _find_subclass(factory, self.annotation)
if self.annotation is None and factory == "None":
return lambda: None
except MetaFromString as e:
try:
default_module = self.default_module
if isinstance(default_module, str):
default_module = _module_from_name(default_module)
if default_module is not None:
obj = _module_getattr(default_module, factory)
return _return_prospective(obj, self.annotation, factory=factory)
except MetaFromString:
pass
raise e
# Probably a special form
raise MetaFromString(
f"Could not produce a {type_repr(self.annotation)} instance from {factory!r}"
)
def perform_cast(self, value: str):
if self.original_unspecified is not None:
try:
return _simplistic_try_cast(value, self.original_unspecified)
except CastError:
pass
return _simplistic_try_cast(value, self.annotation)
================================================
FILE: chz/field.py
================================================
from __future__ import annotations
import functools
import sys
from typing import Any, Callable
import chz
from chz.mungers import Munger, default_munger
from chz.tiepin import TypeForm
from chz.util import MISSING, MISSING_TYPE
_FieldValidator = Callable[[Any, str], None]
def field(
*,
# default related
default: Any | MISSING_TYPE = MISSING,
default_factory: Callable[[], Any] | MISSING_TYPE = MISSING,
# munger related
munger: Munger | Callable[[Any, Any], Any] | None = None,
x_type: TypeForm | MISSING_TYPE = MISSING,
converter: Callable[[Any], Any] | None = None,
# blueprint related
meta_factory: chz.factories.MetaFactory | None | MISSING_TYPE = MISSING,
blueprint_unspecified: Callable[..., Any] | MISSING_TYPE = MISSING,
blueprint_cast: Callable[[str], object] | None = None,
# misc
validator: _FieldValidator | (list[_FieldValidator] | None) = None,
repr: bool | Callable[[Any], str] = True,
doc: str = "",
metadata: dict[str, Any] | None = None,
) -> Any:
"""Customise a field in a chz class.
Args:
default: The default value for the field (if any).
default_factory:
A function that returns the default value for the field.
Useful for mutable types, for instance, `default_factory=list`.
This does not interact at all with parametrisation. Perhaps a better name would be
lazy_default (but unfortunately, this is not supported by PEP 681, so static type
checkers would lose the ability to understand the class).
munger: Lets you adjust the value of a field. Essentially works the same as
an init_property.
x_type: Useful in combination with mungers. This specifies the type before munging that
will be used for parsing and type checking.
converter: Synonym for munger that works better with static type checkers. It accepts
a munger object or a callable that will be called as fn(value, chzself=chzself).
meta_factory:
Represents the set of possible callables that can give us a value of a given type.
blueprint_unspecified:
Used to construct the meta_factory, if meta_factory is unspecified. This is the
default callable `Blueprint` may attempt to call to get a value of the expected type.
See the documentation in chz.factories for more information.
In particular, the following two are equivalent:
```
x: Base = field(blueprint_unspecified=Sub)
x: Base = field(meta_factory=chz.factories.subclass(Base, default_cls=Sub))
```
blueprint_cast: A function that takes a str and returns an object. On failure to cast,
it should raise `CastError`. Used to achieve custom parsing behaviour from the command
line. Takes priority over the `__chz_cast__` dunder method (if present on the
target type).
validator: A function or list of functions that validate the field.
Field validators take two arguments: the instance of the class
and the name of the field.
repr: Whether to include the field in the `__repr__` of the class. This can also be a
callable to customise the repr of the field.
doc: The docstring for the field. Used in `--help`.
metadata: Arbitrary user-defined metadata to attach to the field.
Useful when extending `chz`.
"""
return Field(
name="",
raw_type="",
default=default,
default_factory=default_factory,
munger=munger,
raw_x_type=x_type,
converter=converter,
meta_factory=meta_factory,
blueprint_unspecified=blueprint_unspecified,
blueprint_cast=blueprint_cast,
validator=validator,
repr=repr,
doc=doc,
metadata=metadata,
)
class Field:
def __init__(
self,
*,
name: str,
raw_type: TypeForm | str,
default: Any = MISSING,
default_factory: Callable[[], Any] | MISSING_TYPE = MISSING,
munger: Munger | Callable[[Any, Any], Any] | None = None,
raw_x_type: TypeForm | MISSING_TYPE = MISSING,
converter: Callable[[Any], Any] | None = None,
meta_factory: chz.factories.MetaFactory | None | MISSING_TYPE = MISSING,
blueprint_unspecified: Callable[..., Any] | MISSING_TYPE = MISSING,
blueprint_cast: Callable[[str], object] | None = None,
validator: _FieldValidator | (list[_FieldValidator] | None) = None,
repr: bool | Callable[[Any], str] = True,
doc: str = "",
metadata: dict[str, Any] | None = None,
):
if default.__class__.__hash__ is None:
raise ValueError(
f"Mutable default {type(default)} for field "
f"{name} is not allowed: use default_factory"
)
if (
meta_factory is not MISSING
and meta_factory is not None
and not isinstance(meta_factory, chz.factories.MetaFactory)
):
raise TypeError(f"meta_factory must be a MetaFactory, not {type(meta_factory)}")
if blueprint_unspecified is not MISSING:
if not callable(blueprint_unspecified):
raise TypeError(
f"blueprint_unspecified must be callable, not {type(blueprint_unspecified)}"
)
if meta_factory is not MISSING:
raise ValueError("Cannot specify both meta_factory and blueprint_unspecified")
if default_factory is not MISSING:
if not callable(default_factory):
raise TypeError(f"default_factory must be callable, not {type(default_factory)}")
if isinstance(default_factory, chz.factories.MetaFactory):
raise TypeError(
"default_factory must be a callable that returns a value, "
"not a MetaFactory. Note that default_factory must be callable without any "
"arguments and does not interact with parametrisation."
)
if converter is not None:
if munger is not None:
raise ValueError("Cannot specify both converter and munger")
if not callable(converter):
raise TypeError(f"converter must be callable, not {type(converter)}")
if isinstance(converter, Munger):
munger = converter
else:
# Note: when the munger arg is a function, it is called as munger(chzself, value),
# but converters must be defined with the value as the only positional parameter,
# and so they are called as converter(value, chzself=chzself).
# TODO: change the signature of functions passed to the `munger` argument to be
# compatible with `converter`?
c = converter
munger = lambda s, v: c(v, chzself=s) # type: ignore
if munger is not None and not callable(munger):
raise TypeError(f"munger must be callable, not {type(munger)}")
if validator is None:
validator = []
elif not isinstance(validator, list):
validator = [validator]
self._name = name
self._raw_type = raw_type
self._raw_x_type = raw_x_type
self._default = default
self._default_factory = default_factory
self._meta_factory = meta_factory
self._blueprint_unspecified = blueprint_unspecified
self._munger = munger
self._validator: list[_FieldValidator] = validator
self._blueprint_cast = blueprint_cast
self._repr = repr
self._doc = doc
self._metadata = metadata
# We used to pass the actual globals around, but cloudpickle did not like that
# when it tried to pickle chz classes by value in __main__
# Note that this means that if we're using postponed annotations or quoted annotations
# in __main__ that self.type will likely fail if this is ever pickled and unpickled
self._user_module: str = ""
@property
def logical_name(self) -> str:
for magic_prefix in ("隐", "_X_"):
if self._name.startswith(magic_prefix):
raise RuntimeError(f"Magic prefix {magic_prefix} no longer supported, use X_")
if self._name.startswith("X_"):
return self._name.removeprefix("X_")
return self._name
@property
def x_name(self) -> str:
return "X_" + self.logical_name
@functools.cached_property
def final_type(self) -> TypeForm:
if not self._name:
raise RuntimeError(
"Something has gone horribly awry; are you using a chz.Field in a dataclass?"
)
# Delay the eval until after the class
if isinstance(self._raw_type, str):
# TODO: handle forward ref
assert self._user_module
if self._user_module not in sys.modules:
raise RuntimeError(
f"Could not find module {self._user_module}; possibly a pickling issue?"
)
user_globals = sys.modules[self._user_module].__dict__
return eval(self._raw_type, user_globals)
return self._raw_type
@functools.cached_property
def x_type(self) -> TypeForm:
if isinstance(self._raw_x_type, MISSING_TYPE):
return self.final_type
return self._raw_x_type
@property
def meta_factory(self) -> chz.factories.MetaFactory | None:
if self._meta_factory is None:
return None
if isinstance(self._meta_factory, MISSING_TYPE):
if isinstance(self._blueprint_unspecified, MISSING_TYPE):
unspec = None
else:
unspec = self._blueprint_unspecified
import chz.factories
ret = chz.factories.standard(
annotation=self.x_type, unspecified=unspec, default_module=self._user_module
)
ret.field_annotation = self.x_type
ret.field_module = self._user_module
return ret
self._meta_factory.field_annotation = self.x_type
self._meta_factory.field_module = self._user_module
return self._meta_factory
def get_munger(self) -> Callable[[Any], None] | None:
if self._munger is None:
return None
if isinstance(self._munger, Munger):
m = self._munger
else:
assert callable(self._munger)
m = default_munger(self._munger)
# Must return a new callable every time
return lambda chzself: m(getattr(chzself, self.x_name), chzself=chzself, field=self)
@property
def metadata(self) -> dict[str, Any] | None:
return self._metadata
def __repr__(self):
return f"Field(name={self._name!r}, type={self.final_type!r}, ...)"
def versioning_key(self) -> tuple[str, ...]:
from chz.tiepin import approx_type_hash
raw_type_key = approx_type_hash(self._raw_type)
if self._default is MISSING:
default_key = ""
elif self._default.__repr__ is not object.__repr__:
default_key = repr(self._default)
else:
default_key = self._default_factory.__class__.__name__
if isinstance(self._default_factory, MISSING_TYPE):
default_factory_key = ""
else:
# TODO: support lambdas
default_factory_key = (
self._default_factory.__module__ + "." + self._default_factory.__name__
)
return (self._name, raw_type_key, default_key, default_factory_key)
================================================
FILE: chz/mungers.py
================================================
from __future__ import annotations
from typing import TYPE_CHECKING, Any, Callable, Mapping, TypeVar, overload
if TYPE_CHECKING:
from frozendict import frozendict
from chz.field import Field
_T = TypeVar("_T")
_K = TypeVar("_K")
_V = TypeVar("_V")
class Munger:
"""Marker class for mungers"""
def __call__(self, value: Any, *, chzself: Any = None, field: Field | None = None) -> Any:
raise NotImplementedError
class if_none(Munger):
"""If None, munge the field to the result of an arbitrary function of the chz object."""
def __init__(self, replacement: Callable[[Any], Any]):
self.replacement = replacement
def __call__(self, value: _T | None, *, chzself: Any = None, field: Field | None = None) -> _T:
if value is not None:
return value
return self.replacement(chzself)
class attr_if_none(Munger):
"""If None, munge the field to another attribute of the chz object."""
def __init__(self, replacement_attr: str):
self.replacement_attr = replacement_attr
def __call__(self, value: _T | None, *, chzself: Any = None, field: Field | None = None) -> _T:
if value is not None:
return value
return getattr(chzself, self.replacement_attr)
class default_munger(Munger):
def __init__(self, fn: Callable[[Any, Any], Any]):
self.fn = fn
def __call__(self, value: Any, *, chzself: Any = None, field: Field | None = None) -> Any:
# Note: when the munger arg is a function, it is called as munger(chzself, value),
# and we keep that calling convention here. See also the comment in Field.__init__.
return self.fn(chzself, value)
class freeze_dict(Munger):
"""Freezes a dictionary value so the object is hashable."""
@overload
def __call__(
self, value: Mapping[_K, _V], *, chzself: Any = None, field: Field | None = None
) -> frozendict[_K, _V]: ...
@overload
def __call__(
self, value: Mapping[_K, _V] | None, *, chzself: Any = None, field: Field | None = None
) -> frozendict[_K, _V] | None: ...
def __call__(
self, value: Mapping[_K, _V] | None, *, chzself: Any = None, field: Field | None = None
) -> frozendict[_K, _V] | None:
from frozendict import frozendict
if value is not None and not isinstance(value, frozendict):
return frozendict[_K, _V](value) # pyright: ignore[reportUnknownArgumentType]
return value
================================================
FILE: chz/py.typed
================================================
================================================
FILE: chz/tiepin.py
================================================
"""
It's a fair question why this module exists, instead of using something third party.
There are two things I would have liked to farm out: 1) is_subtype_instance, 2) _simplistic_try_cast.
For is_subtype_instance, I would have liked to use `typeguard`. Unfortunately, the `typeguard`
version we were on did not support a lot of basic things. We couldn't upgrade either, because the
new version had breaking changes and more importantly created ref cycles in places that caused us
to hold on to GPU tensors for longer than we should have, causing GPU OOMs. Update: I eventually
got this fixed upstream.
For _simplistic_try_cast, despite its name, seems to work better than most things out there for our
use case. This is also nice to be able to customise for chz's purposes.
I also have another motivation, which is by writing my own Python runtime type checker, I'll
become a better maintainer of typing.py / typing_extensions.py upstream.
"""
import ast
import collections.abc
import functools
import hashlib
import importlib
import inspect
import operator
import sys
import types
import typing
import typing_extensions
def type_repr(typ) -> str:
# Similar to typing._type_repr
if isinstance(typ, (types.GenericAlias, typing._GenericAlias)):
if typ.__origin__.__module__ in {"typing", "typing_extensions", "collections.abc"}:
if typ.__origin__ is collections.abc.Callable:
return repr(typ).removeprefix("collections.abc.").removeprefix("typing.")
# Based on typing._GenericAlias.__repr__
name = typ.__origin__.__name__
if typ.__args__:
args = ", ".join([type_repr(a) for a in typ.__args__])
else:
args = "()"
return f"{name}[{args}]"
return repr(typ)
if isinstance(typ, (type, types.FunctionType)):
module = getattr(typ, "__module__", None)
name = getattr(typ, "__qualname__", None)
if name is None:
name = getattr(typ, "__name__", None)
if name is not None:
if module == "typing":
return f"{module}.{name}"
if module is not None and module != "builtins" and module != "__main__":
return f"{module}:{name}"
return name
if typ is ...:
return "..."
return repr(typ)
def _approx_type_to_bytes(t) -> bytes:
# This tries to keep the resulting value similar with and without __future__ annotations
# As a result, the conversion is approximate. For instance, `builtins.float` and
# `class float: ...` will look the same.
# If you need something more discerning, maybe just use pickle? Although note that pickle
# doesn't work on at least forward refs and non-module level typevars
origin = getattr(t, "__origin__", None)
args = getattr(t, "__args__", ())
if origin is None:
if isinstance(t, type):
# don't use t.__module__, so that we're more likely to preserve hashes
# with and without future annotations
origin_bytes = t.__name__.encode("utf-8")
elif isinstance(t, typing._SpecialForm):
origin_bytes = t._name.encode("utf-8")
elif isinstance(t, typing.TypeVar):
origin_bytes = t.__name__.encode("utf-8")
elif isinstance(t, typing.ForwardRef):
origin_bytes = t.__forward_arg__.encode("utf-8")
elif isinstance(t, str):
origin_bytes = t.encode("utf-8")
elif isinstance(t, (bytes, int, type(...), type(None))):
# enums?
origin_bytes = repr(t).encode("utf-8")
else:
raise TypeError(f"Cannot convert {t} of {type(t)} to bytes")
else:
origin_bytes = _approx_type_to_bytes(origin)
arg_bytes = (b"[" + b",".join(_approx_type_to_bytes(a) for a in args) + b"]") if args else b""
return origin_bytes + arg_bytes
def approx_type_hash(t) -> str:
return hashlib.sha1(_approx_type_to_bytes(t)).hexdigest()
def eval_in_context(annot: str, obj: object) -> typing.Any:
# Based on inspect.get_annotations
if isinstance(obj, type):
obj_globals = None
module_name = getattr(obj, "__module__", None)
if module_name:
module = sys.modules.get(module_name, None)
if module:
obj_globals = getattr(module, "__dict__", None)
obj_locals = dict(vars(obj))
unwrap = obj
elif isinstance(obj, types.ModuleType):
obj_globals = getattr(obj, "__dict__", None)
obj_locals = None
unwrap = None
elif callable(obj):
obj_globals = getattr(obj, "__globals__", None)
obj_locals = None
unwrap = obj
elif obj is None:
obj_globals = None
obj_locals = None
unwrap = None
else:
raise TypeError(f"{obj!r} is not a module, class, or callable.")
if unwrap is not None:
while True:
if hasattr(unwrap, "__wrapped__"):
unwrap = unwrap.__wrapped__
continue
if isinstance(unwrap, functools.partial):
unwrap = unwrap.func
continue
break
if hasattr(unwrap, "__globals__"):
obj_globals = unwrap.__globals__
assert isinstance(annot, str)
return eval(annot, obj_globals, obj_locals)
def maybe_eval_in_context(annot: str, obj: object) -> typing.Any:
if isinstance(annot, str):
return eval_in_context(annot, obj)
if annot is inspect.Parameter.empty:
return typing.Any
return annot
if sys.version_info >= (3, 11):
typing_Never = (
typing.NoReturn,
typing_extensions.NoReturn,
typing_extensions.Never,
typing.Never,
)
else:
typing_Never = (typing.NoReturn, typing_extensions.NoReturn, typing_extensions.Never)
TypeForm = object
InstantiableType: typing.TypeAlias = type | types.GenericAlias # | typing._GenericAlias
def is_instantiable_type(t: TypeForm) -> typing.TypeGuard[InstantiableType]:
origin = getattr(t, "__origin__", t)
return isinstance(origin, type) and origin is not type
def is_union_type(t: TypeForm) -> bool:
# This has gotten a little messy with Python 3.14
origin = getattr(t, "__origin__", t)
return origin is typing.Union or isinstance(t, types.UnionType) or t is types.UnionType
def is_typed_dict(t: TypeForm) -> bool:
return isinstance(t, (typing._TypedDictMeta, typing_extensions._TypedDictMeta))
class CastError(Exception):
pass
def _module_from_name(name: str) -> types.ModuleType:
try:
return importlib.import_module(name)
except ImportError as e:
raise CastError(f"Could not import module {name!r} ({type(e).__name__}: {e})") from None
def _module_getattr(mod: types.ModuleType, attr: str) -> typing.Any:
try:
for a in attr.split("."):
mod = getattr(mod, a)
return mod
except AttributeError:
raise CastError(f"No attribute named {attr!r} in module {mod.__name__}") from None
def _sort_for_union_preference(typs: tuple[TypeForm, ...]):
def sort_key(typ):
typ = getattr(typ, "__origin__", typ)
if typ is str:
# sort str to last, because anything can be cast to str
return 1
if typ is typing.Literal or typ is typing_extensions.Literal:
# sort literals to first, because they exact match
return -2
if typ is type(None) or typ is None:
# None exact matches as well (like all singletons)
return -1
return 0
# note this is a stable sort, so we preserve user ordering
return sorted(typs, key=sort_key)
def is_args_unpack(t: TypeForm) -> bool:
return getattr(t, "__unpacked__", False) or getattr(t, "__origin__", t) in {
typing.Unpack,
typing_extensions.Unpack,
}
def is_kwargs_unpack(t: TypeForm) -> bool:
return getattr(t, "__origin__", t) in {typing.Unpack, typing_extensions.Unpack}
def _unpackable_arg_length(t: TypeForm) -> tuple[int, bool]:
item_args = None
if getattr(t, "__unpacked__", False):
assert t.__origin__ is tuple # TODO
item_args = t.__args__
elif getattr(t, "__origin__", t) in {typing.Unpack, typing_extensions.Unpack}:
assert len(t.__args__) == 1
assert t.__args__[0].__origin__ is tuple
item_args = t.__args__[0].__args__
else:
return (1, False)
if not item_args or item_args[-1] is ...:
assert len(item_args) == 2
return (0, True)
min_length = 0
unbounded = False
for item_arg in item_args:
arg_length, arg_unbounded = _unpackable_arg_length(item_arg)
min_length += arg_length
unbounded |= arg_unbounded
return (min_length, unbounded)
def _cast_unpacked_tuples(
inst_items: list[str], args: tuple[TypeForm, ...]
) -> tuple[typing.Any, ...]:
# Cursed PEP 646 stuff
arg_lengths = [_unpackable_arg_length(arg) for arg in args]
min_length = sum(arg_length for arg_length, _ in arg_lengths)
if len(inst_items) < min_length:
raise CastError(
f"Could not cast {repr(','.join(inst_items))} to {type_repr(tuple[*args])} "
"because of length mismatch"
)
ret = []
i = 0
for arg, (arg_length, arg_unbounded) in zip(args, arg_lengths):
if is_args_unpack(arg):
if arg_unbounded:
arg_length += len(inst_items) - min_length
min_length = len(inst_items)
if getattr(arg, "__origin__", arg) in {typing.Unpack, typing_extensions.Unpack}:
assert len(arg.__args__) == 1
assert arg.__args__[0].__origin__ is tuple
arg = arg.__args__[0]
arg = arg.__args__
if len(arg) == 0:
ret.extend(inst_items[i : i + arg_length])
elif len(arg) == 2 and arg[-1] is ...:
ret.extend(
_cast_unpacked_tuples(inst_items[i : i + arg_length], (arg[0],) * arg_length)
)
else:
ret.extend(_cast_unpacked_tuples(inst_items[i : i + arg_length], arg))
else:
assert arg_length == 1
assert not arg_unbounded
ret.append(_simplistic_try_cast(inst_items[i], arg))
i += arg_length
return tuple(ret)
def _simplistic_try_cast(inst_str: str, typ: TypeForm):
origin = getattr(typ, "__origin__", typ)
if is_union_type(origin):
# sort str to last spot
args = _sort_for_union_preference(getattr(typ, "__args__", ()))
for arg in args:
try:
return _simplistic_try_cast(inst_str, arg)
except CastError:
pass
raise CastError(f"Could not cast {repr(inst_str)} to {type_repr(typ)}")
if origin is typing.Any or origin is typing_extensions.Any or origin is object:
try:
return ast.literal_eval(inst_str)
except (ValueError, SyntaxError):
pass
# Also accept some lowercase spellings
if inst_str in {"true", "false"}:
return inst_str == "true"
if inst_str in {"none", "null", "NULL"}:
return None
return inst_str
if isinstance(origin, typing.TypeVar):
if origin.__constraints__:
for constraint in origin.__constraints__:
try:
return _simplistic_try_cast(inst_str, constraint)
except CastError:
pass
raise CastError(f"Could not cast {repr(inst_str)} to {type_repr(typ)}")
if origin.__bound__:
return _simplistic_try_cast(inst_str, origin.__bound__)
return _simplistic_try_cast(inst_str, object)
if origin is typing.Literal or origin is typing_extensions.Literal:
values_by_type = {}
for arg in getattr(typ, "__args__", ()):
values_by_type.setdefault(type(arg), []).append(arg)
for literal_typ, literal_values in values_by_type.items():
try:
value = _simplistic_try_cast(inst_str, literal_typ)
if value in literal_values:
return value
except CastError:
pass
raise CastError(f"Could not cast {repr(inst_str)} to {type_repr(typ)}")
if origin is None or origin is type(None):
if inst_str == "None":
return None
raise CastError(f"Could not cast {repr(inst_str)} to {type_repr(typ)}")
if origin is bool:
if inst_str in {"t", "true", "True", "1"}:
return True
if inst_str in {"f", "false", "False", "0"}:
return False
raise CastError(f"Could not cast {repr(inst_str)} to {type_repr(typ)}")
if origin is str:
return inst_str
if origin is float:
try:
return float(inst_str)
except ValueError as e:
raise CastError(f"Could not cast {repr(inst_str)} to {type_repr(typ)}") from e
if origin is int:
try:
return int(inst_str)
except ValueError as e:
raise CastError(f"Could not cast {repr(inst_str)} to {type_repr(typ)}") from e
if origin is list or origin is collections.abc.Sequence or origin is collections.abc.Iterable:
if not inst_str:
return []
args = getattr(typ, "__args__", ())
item_type = args[0] if args else typing.Any
if inst_str[0] in {"[", "("}:
try:
value = ast.literal_eval(inst_str)
except (ValueError, SyntaxError):
raise CastError(f"Could not cast {repr(inst_str)} to {type_repr(typ)}") from None
if is_subtype_instance(value, typ):
return value
raise CastError(f"Could not cast {repr(inst_str)} to {type_repr(typ)}")
inst_items = inst_str.split(",") if inst_str else []
ret = [_simplistic_try_cast(item, item_type) for item in inst_items]
if origin is list:
return ret
return tuple(ret)
if origin is tuple:
args = getattr(typ, "__args__", ())
inst_items = inst_str.split(",") if inst_str else []
if len(args) == 0:
return tuple(inst_items)
if len(args) == 2 and args[-1] is ...:
item_type = args[0]
return tuple(_simplistic_try_cast(item, item_type) for item in inst_items)
num_unpack = sum(is_args_unpack(arg) for arg in args)
if num_unpack == 0:
# Great, normal heterogenous tuple
if len(args) != len(inst_items):
raise CastError(
f"Could not cast {repr(inst_str)} to {type_repr(typ)} because of length mismatch"
+ (
f". Homogeneous tuples should be typed as tuple[{type_repr(args[0])}, ...] not tuple[{type_repr(args[0])}]"
if len(args) == 1
else ""
)
)
return tuple(
_simplistic_try_cast(item, item_typ) for item, item_typ in zip(inst_items, args)
)
else:
# Cursed PEP 646 stuff
return _cast_unpacked_tuples(inst_items, args)
if origin is dict or origin is collections.abc.Mapping:
if not inst_str:
return {}
if inst_str[0] == "{":
try:
value = ast.literal_eval(inst_str)
except (ValueError, SyntaxError):
raise CastError(f"Could not cast {repr(inst_str)} to {type_repr(typ)}") from None
if is_subtype_instance(value, typ):
return value
raise CastError(f"Could not cast {repr(inst_str)} to {type_repr(typ)}")
if origin is collections.abc.Callable:
# TODO: also support type, callback protocols
# TODO: unify with factories.from_string
# TODO: needs module context
if ":" in inst_str:
try:
module_name, var = inst_str.split(":", 1)
module = _module_from_name(module_name)
value = _module_getattr(module, var)
if not is_subtype_instance(value, typ):
raise CastError(f"{type_repr(value)} is not a subtype of {type_repr(typ)}")
except CastError as e:
raise CastError(
f"Could not cast {repr(inst_str)} to {type_repr(typ)}. {e}"
) from None
return value
else:
raise CastError(
f"Could not cast {repr(inst_str)} to {type_repr(typ)}. Try using a fully qualified name, e.g. module_name:{inst_str}"
)
if "torch" in sys.modules:
import torch
if origin is torch.dtype:
value = getattr(torch, inst_str, None)
if value and isinstance(value, torch.dtype):
return value
raise CastError(f"Could not cast {repr(inst_str)} to {type_repr(typ)}")
if "datetime" in sys.modules:
import datetime
if origin is datetime.datetime:
try:
return datetime.datetime.fromisoformat(inst_str)
except ValueError:
raise CastError(f"Could not cast {repr(inst_str)} to {type_repr(typ)}") from None
if "enum" in sys.modules:
import enum
if isinstance(origin, type) and issubclass(origin, enum.Enum):
try:
# Look up by name
return origin[inst_str]
except KeyError:
pass
# Fallback to looking up by value
for member in origin:
try:
value = _simplistic_try_cast(inst_str, type(member.value))
except CastError:
continue
if value == member.value:
return member
raise CastError(f"Could not cast {repr(inst_str)} to {type_repr(typ)}")
if "fractions" in sys.modules:
import fractions
if origin is fractions.Fraction:
try:
return fractions.Fraction(inst_str)
except ValueError as e:
raise CastError(f"Could not cast {repr(inst_str)} to {type_repr(typ)}") from e
if "pathlib" in sys.modules:
import pathlib
if origin is pathlib.Path:
return pathlib.Path(inst_str)
if hasattr(origin, "__chz_cast__"):
return origin.__chz_cast__(inst_str)
if not isinstance(origin, type):
raise CastError(f"Unrecognised type object {type_repr(typ)}")
raise CastError(f"Could not cast {repr(inst_str)} to {type_repr(typ)}")
class _SignatureOf:
def __init__(self, fn: typing.Callable, strip_self: bool = False):
self.fn = fn
self._sig = inspect.signature(fn)
self.pos = []
self.kwonly = {}
self.varpos = None
self.varkw = None
for param in self._sig.parameters.values():
if param.kind in {param.POSITIONAL_OR_KEYWORD, param.POSITIONAL_ONLY}:
self.pos.append(param)
elif param.kind is param.KEYWORD_ONLY:
self.kwonly[param.name] = param
elif param.kind is param.VAR_POSITIONAL and param.name != "__chz_args":
self.varpos = param
elif param.kind is param.VAR_KEYWORD:
self.varkw = param
if strip_self:
if self.pos[0].name != "self":
raise ValueError(f"Cannot strip self from signature of {self.fn}")
self.pos = self.pos[1:]
self.ret = self._sig.return_annotation
if isinstance(self.ret, str):
self.ret = eval_in_context(self.ret, self.fn)
def is_subtype(left: TypeForm, right: TypeForm) -> bool:
left_origin = getattr(left, "__origin__", left)
left_args = getattr(left, "__args__", ())
right_origin = getattr(right, "__origin__", right)
right_args = getattr(right, "__args__", ())
if left_origin is typing.Any or left_origin is typing_extensions.Any:
return True
if right_origin is typing.Any or right_origin is typing_extensions.Any:
return True
if left_origin is None:
if right_origin is None or right_origin is type(None):
return True
if is_union_type(right_origin):
if is_union_type(left_origin):
possible_left_types = left_args
else:
possible_left_types = [left]
return all(
any(is_subtype(possible_left, right_arg) for right_arg in right_args)
for possible_left in possible_left_types
)
if right_origin is typing.Literal or right_origin is typing_extensions.Literal:
if left_origin is typing.Literal or left_origin is typing_extensions.Literal:
return all(left_arg in right_args for left_arg in left_args)
return False
if left_origin is typing.Literal or left_origin is typing_extensions.Literal:
return all(is_subtype_instance(left_arg, right) for left_arg in left_args)
if isinstance(left_origin, typing.TypeVar):
if left_origin == right_origin:
return True
bound = left_origin.__bound__
if bound is None:
bound = object
if is_subtype(bound, right_origin):
return True
if left_origin.__constraints__:
return any(
is_subtype(left_arg, right_origin) for left_arg in left_origin.__constraints__
)
return False
if isinstance(right_origin, typing.TypeVar):
if right_origin.__constraints__:
return any(
is_subtype(left_origin, constraint) for constraint in right_origin.__constraints__
)
if right_origin.__bound__:
return is_subtype(left_origin, right_origin.__bound__)
return True
if typing_extensions.is_protocol(left) and typing_extensions.is_protocol(right):
left_attrs = typing_extensions.get_protocol_members(left)
right_attrs = typing_extensions.get_protocol_members(right)
if not right_attrs.issubset(left_attrs):
return False
# TODO: this is incorrect
return True
if typing_extensions.is_protocol(right):
if not isinstance(left_origin, type):
return False
right_attrs = typing_extensions.get_protocol_members(right)
if not all(hasattr(left_origin, attr) for attr in right_attrs):
return False
# TODO: this is incorrect
return True
if isinstance(left, _SignatureOf) and isinstance(right, _SignatureOf):
empty = inspect.Parameter.empty
for left_param, right_param in zip(left.pos, right.pos):
if right_param.kind is right_param.POSITIONAL_OR_KEYWORD:
if right_param.name != left_param.name:
return False
if left_param.kind is left_param.POSITIONAL_ONLY:
return False
if right_param.default is not empty and left_param.default is empty:
return False
left_param_annot = maybe_eval_in_context(left_param.annotation, left.fn)
right_param_annot = maybe_eval_in_context(right_param.annotation, right.fn)
if not is_subtype(right_param_annot, left_param_annot):
return False
if len(left.pos) < len(right.pos):
# Okay if left has a *args that accepts all the extra args
if left.varpos is None:
return False
left_varpos_annot = maybe_eval_in_context(left.varpos.annotation, left.fn)
for i in range(len(left.pos), len(right.pos)):
right_param = right.pos[i]
right_param_annot = maybe_eval_in_context(right_param.annotation, right.fn)
if not is_subtype(right_param_annot, left_varpos_annot):
return False
if len(left.pos) > len(right.pos):
# Must either have a default or correspond to a required keyword-only arg
for i in range(len(right.pos), len(left.pos)):
left_param = left.pos[i]
if left_param.default is not empty:
continue
if (
left_param.name in right.kwonly
and left_param.kind is left_param.POSITIONAL_OR_KEYWORD
):
continue
return False
for name in left.kwonly.keys() & right.kwonly.keys():
right_param = right.kwonly[name]
left_param = left.kwonly[name]
if right_param.default is not empty and left_param.default is empty:
return False
left_param_annot = maybe_eval_in_context(left_param.annotation, left.fn)
right_param_annot = maybe_eval_in_context(right_param.annotation, right.fn)
if not is_subtype(right_param_annot, left_param_annot):
return False
for name in left.kwonly.keys() - right.kwonly.keys():
# Must either have a default or match a varkwarg
left_param = left.kwonly[name]
if left_param.default is not empty:
continue
if right.varkw is not None:
left_param_annot = maybe_eval_in_context(left_param.annotation, left.fn)
right_varkw_annot = maybe_eval_in_context(right.varkw.annotation, right.fn)
if is_subtype(right_varkw_annot, left_param_annot):
continue
return False
right_only_kwonly = right.kwonly.keys() - left.kwonly.keys()
if right_only_kwonly:
# Must correspond to a positional-or-keyword arg
left_pos_or_kw = {p.name: p for p in left.pos if p.kind is p.POSITIONAL_OR_KEYWORD}
for name in right_only_kwonly:
if name not in left_pos_or_kw:
return False
left_param = left_pos_or_kw[name]
if right.kwonly[name].default is not empty and left_param.default is empty:
return False
left_param_annot = maybe_eval_in_context(left_param.annotation, left.fn)
right_param_annot = maybe_eval_in_context(right.kwonly[name].annotation, right.fn)
if not is_subtype(right_param_annot, left_param_annot):
return False
if right.varkw is not None:
if left.varkw is None:
return False
right_varkw_annot = maybe_eval_in_context(right.varkw.annotation, right.fn)
left_varkw_annot = maybe_eval_in_context(left.varkw.annotation, left.fn)
if not is_subtype(right_varkw_annot, left_varkw_annot):
return False
if right.ret is not empty and left.ret is not empty:
# TODO: handle Cls.__init__ like below
if not is_subtype(left.ret, right.ret):
return False
return True
if left_origin is collections.abc.Callable and right_origin is collections.abc.Callable:
*left_params, left_ret = left_args
*right_params, right_ret = right_args
if len(left_params) != len(right_params):
return False
if not is_subtype(left_ret, right_ret):
return False
return all(
is_subtype(right_param, left_param)
for left_param, right_param in zip(left_params, right_params)
)
if is_typed_dict(left_origin) and is_typed_dict(right_origin):
if not right_origin.__required_keys__.issubset(left_origin.__required_keys__):
return False
left_hints = typing_extensions.get_type_hints(left_origin)
right_hints = typing_extensions.get_type_hints(right_origin)
for k, v in right_hints.items():
if k not in left_hints:
return False
if not is_subtype(left_hints[k], v):
# Technically this should be invariant due to mutability
return False
return True
# TODO: handle other special forms
if left_origin is right_origin and left_args == right_args:
return True
try:
if not issubclass(left_origin, right_origin):
return False
except TypeError:
return False
# see comments in is_subtype_instance
# TODO: add invariance
# TODO: think about some of this logic more carefully
if hasattr(left_origin, "__class_getitem__") and hasattr(right_origin, "__class_getitem__"):
if (
issubclass(right_origin, collections.abc.Mapping)
and typing.Generic not in left_origin.__mro__
and typing.Generic not in right_origin.__mro__
):
if left_args:
left_key, left_value = left_args
else:
left_key, left_value = typing.Any, typing.Any
if right_args:
right_key, right_value = right_args
else:
right_key, right_value = typing.Any, typing.Any
return is_subtype(left_key, right_key) and is_subtype(left_value, right_value)
if left_origin is tuple and right_origin is tuple:
if not left_args:
left_args = (typing.Any, ...)
if not right_args:
right_args = (typing.Any, ...)
if len(right_args) == 2 and right_args[1] is ...:
return all(is_subtype(left_arg, right_args[0]) for left_arg in left_args)
if len(left_args) == 2 and left_args[1] is ...:
return False
return len(left_args) == len(right_args) and all(
is_subtype(left_arg, right_arg)
for left_arg, right_arg in zip(left_args, right_args)
)
if (
issubclass(right_origin, collections.abc.Iterable)
and typing.Generic not in left_origin.__mro__
and typing.Generic not in right_origin.__mro__
):
if left_args:
(left_item,) = left_args
else:
left_item = typing.Any
if right_args:
(right_item,) = right_args
else:
right_item = typing.Any
return is_subtype(left_item, right_item)
return True
def is_subtype_instance(inst: typing.Any, typ: TypeForm) -> bool:
if typ is typing.Any or typ is typing_extensions.Any:
return True
if typ is None and inst is None:
return True
if isinstance(typ, typing.TypeVar):
if typ.__constraints__:
# types must match exactly
return any(
type(inst) is getattr(c, "__origin__", c) and is_subtype_instance(inst, c)
for c in typ.__constraints__
)
if typ.__bound__:
return is_subtype_instance(inst, typ.__bound__)
return True
if isinstance(typ, typing.NewType):
return isinstance(inst, typ.__supertype__)
origin: typing.Any
args: typing.Any
if sys.version_info >= (3, 10) and isinstance(typ, types.UnionType):
origin = typing.Union
else:
origin = getattr(typ, "__origin__", typ)
args = getattr(typ, "__args__", ())
del typ
if origin is typing.Union:
return any(is_subtype_instance(inst, t) for t in args)
if origin is typing.Literal or origin is typing_extensions.Literal:
return inst in args
if origin is typing.LiteralString:
return isinstance(inst, str)
if is_typed_dict(origin):
if not isinstance(inst, dict):
return False
for k, v in typing_extensions.get_type_hints(origin).items():
if k in inst:
if not is_subtype_instance(inst[k], v):
return False
elif k in origin.__required_keys__:
return False
return True
# Pydantic implements generics in a special way. Just delegate validation to Pydantic.
# Note that all pydantic models have __pydantic_generic_metadata__, even non-generic ones.
if hasattr(origin, "__pydantic_generic_metadata__"):
from pydantic import ValidationError
try:
origin.model_validate(inst)
return True
except ValidationError:
return False
if typing_extensions.is_protocol(origin):
if getattr(origin, "_is_runtime_protocol", False):
return isinstance(inst, origin)
if origin in type(inst).__mro__:
return True
annotations = typing_extensions.get_type_hints(origin)
for attr in sorted(typing_extensions.get_protocol_members(origin)):
if not hasattr(inst, attr):
return False
if attr in annotations:
if not is_subtype_instance(getattr(inst, attr), annotations[attr]):
return False
elif callable(getattr(origin, attr)):
if attr == "__call__" and isinstance(inst, (type, types.FunctionType)):
# inst will have a better inspect.signature than inst.__call__
inst_attr = inst
else:
inst_attr = getattr(inst, attr)
gitextract_3v4reuym/
├── .gitignore
├── CHANGELOG.md
├── LICENSE
├── README.md
├── chz/
│ ├── __init__.py
│ ├── blueprint/
│ │ ├── __init__.py
│ │ ├── _argmap.py
│ │ ├── _argv.py
│ │ ├── _blueprint.py
│ │ ├── _entrypoint.py
│ │ ├── _lazy.py
│ │ └── _wildcard.py
│ ├── data_model.py
│ ├── factories.py
│ ├── field.py
│ ├── mungers.py
│ ├── py.typed
│ ├── tiepin.py
│ ├── universal.py
│ ├── util.py
│ └── validators.py
├── docs/
│ ├── 01_quickstart.md
│ ├── 02_object_model.md
│ ├── 03_validation.md
│ ├── 04_command_line.md
│ ├── 05_blueprint.md
│ ├── 06_serialisation.md
│ ├── 21_post_init.md
│ ├── 22_field_api.md
│ ├── 91_philosophy.md
│ ├── 92_alternatives.md
│ └── 93_testimonials.md
├── pyproject.toml
└── tests/
├── test_blueprint.py
├── test_blueprint_cast.py
├── test_blueprint_errors.py
├── test_blueprint_meta_factory.py
├── test_blueprint_methods.py
├── test_blueprint_reference.py
├── test_blueprint_root_polymorphism.py
├── test_blueprint_unit.py
├── test_blueprint_variadic.py
├── test_data_model.py
├── test_factories.py
├── test_munge.py
├── test_tiepin.py
├── test_todo.py
└── test_validate.py
SYMBOL INDEX (495 symbols across 29 files)
FILE: chz/__init__.py
function _chz (line 52) | def _chz(cls=None, *, version: str | None = None, typecheck: bool | None...
function chz (line 65) | def chz(version: str = ..., typecheck: bool = ...) -> Callable[[type], t...
function chz (line 68) | def chz(cls: _TypeT, /) -> _TypeT: ...
function chz (line 70) | def chz(*a, **k):
FILE: chz/blueprint/_argmap.py
class Layer (line 15) | class Layer:
method __init__ (line 16) | def __init__(self, args: Mapping[str, Any], layer_name: str | None):
method get_kv (line 32) | def get_kv(self, exact_key: str) -> tuple[str, Any, str | None] | None:
method iter_keys (line 41) | def iter_keys(self) -> Iterator[tuple[str, bool]]:
method nest_subpath (line 45) | def nest_subpath(self, subpath: str | None) -> Layer:
method __repr__ (line 53) | def __repr__(self) -> str:
class _FoundArgument (line 58) | class _FoundArgument:
function _valid_parent (line 65) | def _valid_parent(parts: list[str], param_paths: AbstractSet[str]) -> st...
class ArgumentMap (line 73) | class ArgumentMap:
method __init__ (line 74) | def __init__(self, layers: list[Layer]) -> None:
method add_layer (line 82) | def add_layer(self, layer: Layer) -> None:
method consolidate (line 86) | def consolidate(self) -> None:
method subpaths (line 102) | def subpaths(self, path: str, strict: bool = False) -> list[str]:
method get_kv (line 174) | def get_kv(self, exact_key: str, *, ignore_wildcards: bool = False) ->...
method check_extraneous (line 195) | def check_extraneous(
method __repr__ (line 276) | def __repr__(self) -> str:
function join_arg_path (line 280) | def join_arg_path(parent: str, child: str) -> str:
FILE: chz/blueprint/_argv.py
function argv_to_blueprint_args (line 15) | def argv_to_blueprint_args(
function beta_argv_arg_to_string (line 38) | def beta_argv_arg_to_string(key: str, value: Any) -> list[str]:
function beta_blueprint_to_argv (line 71) | def beta_blueprint_to_argv(blueprint: chz.Blueprint[_T]) -> list[str]:
function _collapse_layer (line 85) | def _collapse_layer(
function _collapse_layers (line 114) | def _collapse_layers(blueprint: chz.Blueprint[_T]) -> list[tuple[str, An...
FILE: chz/blueprint/_blueprint.py
class SpecialArg (line 52) | class SpecialArg: ...
class Castable (line 55) | class Castable(SpecialArg):
method __init__ (line 58) | def __init__(self, value: str) -> None:
method __repr__ (line 61) | def __repr__(self) -> str:
method __hash__ (line 64) | def __hash__(self) -> int:
method __eq__ (line 67) | def __eq__(self, other: object) -> bool:
class Reference (line 76) | class Reference(SpecialArg):
method __init__ (line 79) | def __init__(self, ref: str) -> None:
method __repr__ (line 84) | def __repr__(self) -> str:
class Computed (line 89) | class Computed(SpecialArg):
method __repr__ (line 95) | def __repr__(self) -> str:
class _MakeResult (line 101) | class _MakeResult:
function _entrypoint_caster (line 129) | def _entrypoint_caster(o: str) -> object:
function _found_arg_desc (line 133) | def _found_arg_desc(
class Blueprint (line 189) | class Blueprint(Generic[_T_cov_def]):
method __init__ (line 190) | def __init__(
method clone (line 235) | def clone(self) -> Blueprint[_T_cov_def]:
method apply (line 239) | def apply(
method apply_from_argv (line 282) | def apply_from_argv(
method _make_lazy (line 297) | def _make_lazy(self) -> _MakeResult:
method _make_from_make_result (line 339) | def _make_from_make_result(self, r: _MakeResult) -> _T_cov_def:
method make (line 355) | def make(self) -> _T_cov_def:
method make_from_argv (line 360) | def make_from_argv(
method get_help (line 375) | def get_help(self, *, color: bool = False) -> str:
function _lambda_repr (line 486) | def _lambda_repr(fn) -> str | None:
class _Default (line 499) | class _Default:
method to_help_str (line 503) | def to_help_str(self) -> str:
method instantiate (line 515) | def instantiate(self) -> Any:
method from_field (line 521) | def from_field(cls, field: Field) -> _Default | None:
method from_inspect_param (line 527) | def from_inspect_param(cls, sigparam: inspect.Parameter) -> _Default |...
class _Param (line 534) | class _Param:
method cast (line 543) | def cast(self, value: str) -> object:
function _get_variadic_elements (line 559) | def _get_variadic_elements(obj_path: str, arg_map: ArgumentMap) -> set[s...
function _collect_params_from_chz (line 570) | def _collect_params_from_chz(
function _collect_params_from_sequence (line 590) | def _collect_params_from_sequence(
function _collect_params_from_mapping (line 659) | def _collect_params_from_mapping(
function _collect_params_from_typed_dict (line 701) | def _collect_params_from_typed_dict(
function _collect_params_from_callable (line 729) | def _collect_params_from_callable(
function _collect_params (line 869) | def _collect_params(
class _WriteOnlyMapping (line 935) | class _WriteOnlyMapping(Generic[_K, _V], Protocol):
method __setitem__ (line 936) | def __setitem__(self, __key: _K, __value: _V, /) -> None: ...
method update (line 937) | def update(self, __m: Mapping[_K, _V], /) -> None: ...
class ConstructionIssue (line 940) | class ConstructionIssue:
method __init__ (line 941) | def __init__(self, issue: str) -> None:
function _construct_factory (line 945) | def _construct_factory(
function _construct_unspecified_param (line 1002) | def _construct_unspecified_param(
function _construct_param (line 1105) | def _construct_param(
function _check_for_wildcard_matching_variadic_top_level (line 1318) | def _check_for_wildcard_matching_variadic_top_level(
FILE: chz/blueprint/_entrypoint.py
class EntrypointException (line 17) | class EntrypointException(Exception): ...
class EntrypointHelpException (line 20) | class EntrypointHelpException(EntrypointException): ...
class ExtraneousBlueprintArg (line 23) | class ExtraneousBlueprintArg(EntrypointException): ...
class InvalidBlueprintArg (line 26) | class InvalidBlueprintArg(EntrypointException): ...
class MissingBlueprintArg (line 29) | class MissingBlueprintArg(EntrypointException): ...
class ConstructionException (line 32) | class ConstructionException(EntrypointException): ...
function exit_on_entrypoint_error (line 35) | def exit_on_entrypoint_error(fn: _F) -> _F:
function entrypoint (line 54) | def entrypoint(
function nested_entrypoint (line 85) | def nested_entrypoint(
function methods_entrypoint (line 113) | def methods_entrypoint(
function dispatch_entrypoint (line 173) | def dispatch_entrypoint(
function _resolve_annotation (line 216) | def _resolve_annotation(annotation: Any, func: Any) -> Any:
function get_nested_target (line 225) | def get_nested_target(main: Callable[[_T], object]) -> type[_T]:
FILE: chz/blueprint/_lazy.py
class Evaluatable (line 11) | class Evaluatable: ...
class Value (line 14) | class Value(Evaluatable):
method __init__ (line 15) | def __init__(self, value: Any) -> None:
method __repr__ (line 18) | def __repr__(self) -> str:
class ParamRef (line 22) | class ParamRef(Evaluatable):
method __init__ (line 23) | def __init__(self, ref: str) -> None:
method __repr__ (line 26) | def __repr__(self) -> str:
class Thunk (line 30) | class Thunk(Evaluatable):
method __init__ (line 31) | def __init__(self, fn: Callable[..., Any], kwargs: dict[str, ParamRef]...
method __repr__ (line 35) | def __repr__(self) -> str:
function evaluate (line 39) | def evaluate(value_mapping: dict[str, Evaluatable]) -> Any:
function check_reference_targets (line 87) | def check_reference_targets(
FILE: chz/blueprint/_wildcard.py
function wildcard_key_to_regex_str (line 6) | def wildcard_key_to_regex_str(key: str) -> str:
function wildcard_key_to_regex (line 14) | def wildcard_key_to_regex(key: str) -> re.Pattern[str]:
function _wildcard_key_match (line 18) | def _wildcard_key_match(key: str, target_str: str) -> bool:
function wildcard_key_approx (line 46) | def wildcard_key_approx(key: str, target_str: str) -> tuple[float, str]:
FILE: chz/data_model.py
function _create_fn (line 50) | def _create_fn(
function _synthesise_field_init (line 76) | def _synthesise_field_init(f: Field, out_vars: dict[str, Any]) -> tuple[...
function _synthesise_init (line 102) | def _synthesise_init(fields: Collection[Field], user_globals: dict[str, ...
function __setattr__ (line 128) | def __setattr__(self, name, value):
function __delattr__ (line 132) | def __delattr__(self, name):
function _recursive_repr (line 136) | def _recursive_repr(user_function):
function __repr__ (line 156) | def __repr__(self) -> str:
function __eq__ (line 172) | def __eq__(self, other):
function __hash__ (line 178) | def __hash__(self) -> int:
function __chz_validate__ (line 193) | def __chz_validate__(self) -> None:
function _get_init_properties (line 213) | def _get_init_properties(cls: type) -> list[str]:
function __chz_init_property__ (line 220) | def __chz_init_property__(self) -> None:
function pretty_format (line 225) | def pretty_format(obj: Any, colored: bool = True) -> str:
function _repr_pretty_ (line 297) | def _repr_pretty_(self, p, cycle: bool) -> None:
function __chz_pretty__ (line 302) | def __chz_pretty__(self, colored: bool = True) -> str:
function _is_classvar_annotation (line 312) | def _is_classvar_annotation(annot: str | Any) -> bool:
function _is_property_like (line 322) | def _is_property_like(obj: Any) -> bool:
function chz_make_class (line 328) | def chz_make_class(cls, version: str | None, typecheck: bool | None) -> ...
function is_chz (line 521) | def is_chz(c: object) -> bool:
function chz_fields (line 531) | def chz_fields(c: object) -> dict[str, Field]:
function replace (line 540) | def replace(obj: _T, /, **changes) -> _T:
function asdict (line 571) | def asdict(
function traverse (line 623) | def traverse(obj: Any, obj_path: str = "") -> Iterable[tuple[str, Any]]:
function beta_to_blueprint_values (line 656) | def beta_to_blueprint_values(obj, skip_defaults: bool = False) -> Any:
class init_property (line 734) | class init_property:
method __init__ (line 738) | def __init__(self, func: Callable[..., Any]) -> None:
method __set_name__ (line 742) | def __set_name__(self, owner, name):
method __get__ (line 754) | def __get__(self, obj: Any, cls: Any) -> Any:
FILE: chz/factories.py
class MetaFromString (line 28) | class MetaFromString(Exception): ...
class MetaFactory (line 31) | class MetaFactory:
method __init__ (line 50) | def __init__(self) -> None:
method unspecified_factory (line 55) | def unspecified_factory(self) -> Callable[..., Any] | None:
method from_string (line 63) | def from_string(self, factory: str) -> Callable[..., Any]:
method perform_cast (line 67) | def perform_cast(self, value: str):
class subclass (line 73) | class subclass(MetaFactory):
method __init__ (line 111) | def __init__(
method __repr__ (line 120) | def __repr__(self) -> str:
method base_cls (line 124) | def base_cls(self) -> InstantiableType:
method default_cls (line 136) | def default_cls(self) -> InstantiableType:
method unspecified_factory (line 141) | def unspecified_factory(self) -> Callable[..., Any]:
method from_string (line 144) | def from_string(self, factory: str) -> Callable[..., Any]:
method perform_cast (line 151) | def perform_cast(self, value: str):
class function (line 159) | class function(MetaFactory):
method __init__ (line 160) | def __init__(
method __repr__ (line 204) | def __repr__(self) -> str:
method default_module (line 208) | def default_module(self) -> types.ModuleType | str | None:
method unspecified_factory (line 214) | def unspecified_factory(self) -> Callable[..., Any] | None:
method from_string (line 217) | def from_string(self, factory: str) -> Callable[..., Any]:
method perform_cast (line 258) | def perform_cast(self, value: str):
function _module_from_name (line 263) | def _module_from_name(name: str) -> types.ModuleType:
function _module_getattr (line 272) | def _module_getattr(mod: types.ModuleType, attr: str) -> Any:
function _find_subclass (line 281) | def _find_subclass(spec: str, superclass: TypeForm):
function _maybe_generic (line 370) | def _maybe_generic(
function _return_prospective (line 395) | def _return_prospective(obj: Any, annotation: TypeForm, factory: str) ->...
function get_unspecified_from_annotation (line 426) | def get_unspecified_from_annotation(annotation: TypeForm) -> Callable[.....
class standard (line 453) | class standard(MetaFactory):
method __init__ (line 454) | def __init__(
method __repr__ (line 466) | def __repr__(self) -> str:
method annotation (line 470) | def annotation(self) -> TypeForm:
method default_module (line 477) | def default_module(self) -> types.ModuleType | str | None:
method computed_unspecified (line 488) | def computed_unspecified(self) -> Callable[..., Any] | None:
method unspecified_factory (line 495) | def unspecified_factory(self) -> Callable[..., Any] | None:
method from_string (line 507) | def from_string(self, factory: str) -> Callable[..., Any]:
method perform_cast (line 594) | def perform_cast(self, value: str):
FILE: chz/field.py
function field (line 15) | def field(
class Field (line 105) | class Field:
method __init__ (line 106) | def __init__(
method logical_name (line 200) | def logical_name(self) -> str:
method x_name (line 209) | def x_name(self) -> str:
method final_type (line 213) | def final_type(self) -> TypeForm:
method x_type (line 231) | def x_type(self) -> TypeForm:
method meta_factory (line 237) | def meta_factory(self) -> chz.factories.MetaFactory | None:
method get_munger (line 260) | def get_munger(self) -> Callable[[Any], None] | None:
method metadata (line 274) | def metadata(self) -> dict[str, Any] | None:
method __repr__ (line 277) | def __repr__(self):
method versioning_key (line 280) | def versioning_key(self) -> tuple[str, ...]:
FILE: chz/mungers.py
class Munger (line 16) | class Munger:
method __call__ (line 19) | def __call__(self, value: Any, *, chzself: Any = None, field: Field | ...
class if_none (line 23) | class if_none(Munger):
method __init__ (line 26) | def __init__(self, replacement: Callable[[Any], Any]):
method __call__ (line 29) | def __call__(self, value: _T | None, *, chzself: Any = None, field: Fi...
class attr_if_none (line 35) | class attr_if_none(Munger):
method __init__ (line 38) | def __init__(self, replacement_attr: str):
method __call__ (line 41) | def __call__(self, value: _T | None, *, chzself: Any = None, field: Fi...
class default_munger (line 47) | class default_munger(Munger):
method __init__ (line 48) | def __init__(self, fn: Callable[[Any, Any], Any]):
method __call__ (line 51) | def __call__(self, value: Any, *, chzself: Any = None, field: Field | ...
class freeze_dict (line 57) | class freeze_dict(Munger):
method __call__ (line 61) | def __call__(
method __call__ (line 66) | def __call__(
method __call__ (line 70) | def __call__(
FILE: chz/tiepin.py
function type_repr (line 35) | def type_repr(typ) -> str:
function _approx_type_to_bytes (line 69) | def _approx_type_to_bytes(t) -> bytes:
function approx_type_hash (line 103) | def approx_type_hash(t) -> str:
function eval_in_context (line 107) | def eval_in_context(annot: str, obj: object) -> typing.Any:
function maybe_eval_in_context (line 149) | def maybe_eval_in_context(annot: str, obj: object) -> typing.Any:
function is_instantiable_type (line 172) | def is_instantiable_type(t: TypeForm) -> typing.TypeGuard[InstantiableTy...
function is_union_type (line 177) | def is_union_type(t: TypeForm) -> bool:
function is_typed_dict (line 183) | def is_typed_dict(t: TypeForm) -> bool:
class CastError (line 187) | class CastError(Exception):
function _module_from_name (line 191) | def _module_from_name(name: str) -> types.ModuleType:
function _module_getattr (line 198) | def _module_getattr(mod: types.ModuleType, attr: str) -> typing.Any:
function _sort_for_union_preference (line 207) | def _sort_for_union_preference(typs: tuple[TypeForm, ...]):
function is_args_unpack (line 225) | def is_args_unpack(t: TypeForm) -> bool:
function is_kwargs_unpack (line 232) | def is_kwargs_unpack(t: TypeForm) -> bool:
function _unpackable_arg_length (line 236) | def _unpackable_arg_length(t: TypeForm) -> tuple[int, bool]:
function _cast_unpacked_tuples (line 261) | def _cast_unpacked_tuples(
function _simplistic_try_cast (line 304) | def _simplistic_try_cast(inst_str: str, typ: TypeForm):
class _SignatureOf (line 524) | class _SignatureOf:
method __init__ (line 525) | def __init__(self, fn: typing.Callable, strip_self: bool = False):
function is_subtype (line 554) | def is_subtype(left: TypeForm, right: TypeForm) -> bool:
function is_subtype_instance (line 808) | def is_subtype_instance(inst: typing.Any, typ: TypeForm) -> bool:
function simplified_union (line 1011) | def simplified_union(types):
function _simplistic_type_of_value (line 1037) | def _simplistic_type_of_value(value: object) -> TypeForm:
FILE: chz/util.py
class MISSING_TYPE (line 1) | class MISSING_TYPE:
method __repr__ (line 2) | def __repr__(self) -> str:
FILE: chz/validators.py
class validate (line 13) | class validate:
method __init__ (line 14) | def __init__(self, fn: Callable[[Any], None]):
method __set_name__ (line 17) | def __set_name__(self, owner: Any, name: str) -> None:
function _ensure_chz_validators (line 23) | def _ensure_chz_validators(cls: Any) -> None:
function for_all_fields (line 32) | def for_all_fields(fn: Callable[[Any, str], None]) -> Callable[[Any], No...
function instancecheck (line 40) | def instancecheck(self: Any, attr: str) -> None:
function typecheck (line 48) | def typecheck(self: Any, attr: str) -> None:
function instance_of (line 65) | def instance_of(typ: type) -> Callable[[Any, str], None]:
function gt (line 76) | def gt(base) -> Callable[[Any, str], None]:
function lt (line 85) | def lt(base) -> Callable[[Any, str], None]:
function ge (line 94) | def ge(base) -> Callable[[Any, str], None]:
function le (line 103) | def le(base) -> Callable[[Any, str], None]:
function valid_regex (line 112) | def valid_regex(self: Any, attr: str) -> None:
function const_default (line 123) | def const_default(self: Any, attr: str) -> None:
function _decorator_typecheck (line 139) | def _decorator_typecheck(self: Any) -> None:
function check_field_consistency_in_tree (line 145) | def check_field_consistency_in_tree(obj: Any, fields: set[str], regex_ro...
function _find_original_definitions (line 192) | def _find_original_definitions(instance: Any) -> dict[str, tuple[Field, ...
function is_override (line 206) | def is_override(
class IsOverrideMixin (line 241) | class IsOverrideMixin:
method _check_overrides (line 264) | def _check_overrides(self) -> None:
FILE: tests/test_blueprint.py
function test_entrypoint (line 13) | def test_entrypoint():
function test_entrypoint_nested (line 45) | def test_entrypoint_nested():
function test_apply_strictness (line 62) | def test_apply_strictness():
function test_basic_function_blueprint (line 79) | def test_basic_function_blueprint():
function test_basic_class_blueprint (line 111) | def test_basic_class_blueprint():
function test_blueprint_unused (line 139) | def test_blueprint_unused():
function test_blueprint_unused_nested_default (line 228) | def test_blueprint_unused_nested_default():
function test_blueprint_missing_args (line 243) | def test_blueprint_missing_args():
function three_item_dataset (line 280) | def three_item_dataset(first: str = "a", second: str = "b", third: str =...
class Model (line 285) | class Model:
class Experiment (line 292) | class Experiment:
function test_nested_construction (line 297) | def test_nested_construction():
function test_nested_construction_with_default_value (line 316) | def test_nested_construction_with_default_value():
function test_nested_construction_with_factory_dataset (line 328) | def test_nested_construction_with_factory_dataset():
function test_nested_construction_with_wildcards (line 349) | def test_nested_construction_with_wildcards():
function test_nested_all_defaults (line 402) | def test_nested_all_defaults():
function test_nested_not_all_defaults (line 415) | def test_nested_not_all_defaults():
function test_nested_all_defaults_primitive (line 431) | def test_nested_all_defaults_primitive():
function test_nested_all_defaults_unspecified_nested (line 443) | def test_nested_all_defaults_unspecified_nested():
function test_nested_construction_with_default_factory (line 463) | def test_nested_construction_with_default_factory():
function test_help (line 482) | def test_help():
function test_logical_name_blueprint (line 540) | def test_logical_name_blueprint():
function test_blueprint_unpack_kwargs (line 560) | def test_blueprint_unpack_kwargs():
function test_blueprint_castable_but_subpaths (line 580) | def test_blueprint_castable_but_subpaths():
function test_blueprint_value_but_subpaths (line 602) | def test_blueprint_value_but_subpaths():
function test_blueprint_apply_subpath (line 619) | def test_blueprint_apply_subpath():
function test_blueprint_enum_all_defaults (line 648) | def test_blueprint_enum_all_defaults():
function test_blueprint_functools_partial (line 670) | def test_blueprint_functools_partial():
function test_blueprint_unspecified_functools_partial (line 707) | def test_blueprint_unspecified_functools_partial():
function test_blueprint_positional_only (line 739) | def test_blueprint_positional_only():
function test_blueprint_args_kwargs (line 756) | def test_blueprint_args_kwargs():
FILE: tests/test_blueprint_cast.py
function test_castable (line 11) | def test_castable():
function test_castable_object_str (line 69) | def test_castable_object_str():
function test_meta_factory_cast_unspecified (line 78) | def test_meta_factory_cast_unspecified():
function test_chz_cast_dunder (line 106) | def test_chz_cast_dunder():
function test_cast_per_field (line 149) | def test_cast_per_field():
FILE: tests/test_blueprint_errors.py
function test_target_bad_signature (line 7) | def test_target_bad_signature():
function test_target_just_plain_old_bad (line 16) | def test_target_just_plain_old_bad():
function test_target_no_params_extraneous (line 21) | def test_target_no_params_extraneous():
function test_nested_target_default_values (line 30) | def test_nested_target_default_values():
function test_blueprint_extraneous_valid_parent (line 49) | def test_blueprint_extraneous_valid_parent():
FILE: tests/test_blueprint_meta_factory.py
class A (line 10) | class A:
class B (line 14) | class B(A):
class C (line 18) | class C(B):
function test_meta_factory_subclass (line 22) | def test_meta_factory_subclass():
function test_meta_factory_subclass_limited (line 42) | def test_meta_factory_subclass_limited():
function test_meta_factory_default_subclass (line 59) | def test_meta_factory_default_subclass():
function test_meta_factory_blueprint_unspecified (line 142) | def test_meta_factory_blueprint_unspecified():
function test_meta_factory_blueprint_unspecified_more (line 179) | def test_meta_factory_blueprint_unspecified_more():
function test_meta_factory_blueprint_unspecified_all_default_help (line 219) | def test_meta_factory_blueprint_unspecified_all_default_help():
function test_meta_factory_blueprint_unspecified_optional (line 240) | def test_meta_factory_blueprint_unspecified_optional():
function test_meta_factory_subclass_generic (line 260) | def test_meta_factory_subclass_generic():
function test_meta_factory_optional (line 301) | def test_meta_factory_optional():
function test_meta_factory_union (line 318) | def test_meta_factory_union():
function test_meta_factory_non_chz (line 335) | def test_meta_factory_non_chz():
function test_meta_factory_function_lambda (line 360) | def test_meta_factory_function_lambda():
function test_meta_factory_type_subclass (line 390) | def test_meta_factory_type_subclass():
function test_meta_factory_function_union (line 405) | def test_meta_factory_function_union():
function test_meta_factory_none (line 430) | def test_meta_factory_none():
FILE: tests/test_blueprint_methods.py
class Run1 (line 12) | class Run1:
method launch (line 15) | def launch(self, cluster: str):
method history (line 19) | def history(self):
class RunDefault (line 24) | class RunDefault:
method launch (line 25) | def launch(self, cluster: str):
function test_methods_entrypoint (line 29) | def test_methods_entrypoint():
function test_methods_entrypoint_help (line 50) | def test_methods_entrypoint_help():
class RunAltSelfParam (line 93) | class RunAltSelfParam:
method launch (line 96) | def launch(run, cluster: str):
function test_methods_entrypoint_self (line 100) | def test_methods_entrypoint_self():
class RunDefaultChild (line 113) | class RunDefaultChild(RunDefault): ...
function test_methods_entrypoint_polymorphic (line 116) | def test_methods_entrypoint_polymorphic():
function test_methods_entrypoint_transform (line 122) | def test_methods_entrypoint_transform():
FILE: tests/test_blueprint_reference.py
function test_blueprint_reference (line 7) | def test_blueprint_reference():
function test_blueprint_reference_multiple_invalid (line 34) | def test_blueprint_reference_multiple_invalid():
function test_blueprint_reference_nested (line 52) | def test_blueprint_reference_nested():
function test_blueprint_reference_wildcard (line 71) | def test_blueprint_reference_wildcard():
function test_blueprint_reference_wildcard_default (line 93) | def test_blueprint_reference_wildcard_default():
function test_blueprint_reference_wildcard_default_no_default (line 107) | def test_blueprint_reference_wildcard_default_no_default():
function test_blueprint_reference_wildcard_default_constructable (line 123) | def test_blueprint_reference_wildcard_default_constructable():
function test_blueprint_reference_cycle (line 154) | def test_blueprint_reference_cycle():
FILE: tests/test_blueprint_root_polymorphism.py
function test_root_polymorphism (line 6) | def test_root_polymorphism():
FILE: tests/test_blueprint_unit.py
function test_beta_argv_arg_to_string (line 8) | def test_beta_argv_arg_to_string():
function test_wildcard_key_to_regex (line 41) | def test_wildcard_key_to_regex():
function test_wildcard_key_match (line 52) | def test_wildcard_key_match():
function test_join_arg_path (line 86) | def test_join_arg_path():
function test_arg_map (line 95) | def test_arg_map():
function test_arg_map_wildcard (line 126) | def test_arg_map_wildcard():
function test_layer (line 188) | def test_layer():
function test_collapse_layers (line 198) | def test_collapse_layers():
function test_collapse_blueprint_to_argv (line 245) | def test_collapse_blueprint_to_argv():
function test_apply_from_argv (line 265) | def test_apply_from_argv():
function test_apply_with_types (line 278) | def test_apply_with_types():
function test_castable_eq (line 291) | def test_castable_eq():
FILE: tests/test_blueprint_variadic.py
function test_variadic_list (line 15) | def test_variadic_list():
function test_variadic_wildcard (line 47) | def test_variadic_wildcard():
function test_variadic_tuple (line 84) | def test_variadic_tuple():
function test_variadic_dict (line 129) | def test_variadic_dict():
function test_variadic_collections_type (line 143) | def test_variadic_collections_type():
function test_variadic_dict_non_int_or_str_key (line 158) | def test_variadic_dict_non_int_or_str_key():
function test_variadic_dict_unannotated (line 175) | def test_variadic_dict_unannotated():
function test_variadic_typed_dict (line 185) | def test_variadic_typed_dict():
function test_variadic_typed_dict_not_required (line 217) | def test_variadic_typed_dict_not_required():
function test_variadic_default (line 329) | def test_variadic_default():
function test_variadic_default_wildcard_error (line 343) | def test_variadic_default_wildcard_error():
function test_variadic_default_wildcard_error_using_types_from_default (line 371) | def test_variadic_default_wildcard_error_using_types_from_default():
function test_polymorphic_variadic_generic (line 413) | def test_polymorphic_variadic_generic():
FILE: tests/test_data_model.py
function test_basic (line 13) | def test_basic():
function _test_construct_helper (line 64) | def _test_construct_helper(X):
function test_construct_without_future_annotations (line 89) | def test_construct_without_future_annotations():
function test_construct_with_future_annotations (line 97) | def test_construct_with_future_annotations():
function test_inheritance (line 105) | def test_inheritance():
function test_immutability (line 150) | def test_immutability():
function test_no_post_init (line 204) | def test_no_post_init():
function test_no_annotation (line 215) | def test_no_annotation():
function test_asdict (line 225) | def test_asdict():
function test_asdict_computed_properties (line 252) | def test_asdict_computed_properties():
function test_asdict_include_type (line 283) | def test_asdict_include_type():
class Outer (line 302) | class Outer:
class Config (line 304) | class Config:
function test_asdict_include_type_nested_class (line 308) | def test_asdict_include_type_nested_class():
function test_asdict_exclude (line 313) | def test_asdict_exclude():
function test_replace (line 329) | def test_replace():
function test_repr (line 386) | def test_repr():
function test_eq (line 412) | def test_eq():
function test_hash (line 426) | def test_hash():
function test_blueprint_values (line 499) | def test_blueprint_values():
function test_blueprint_values_polymorphic (line 605) | def test_blueprint_values_polymorphic():
function test_blueprint_values_variadic (line 684) | def test_blueprint_values_variadic():
function test_blueprint_values_skip_defaults (line 740) | def test_blueprint_values_skip_defaults():
function test_blueprint_values_unspecified_sequence (line 784) | def test_blueprint_values_unspecified_sequence():
function test_duplicate_fields (line 807) | def test_duplicate_fields():
function test_no_type_annotation_on_field (line 817) | def test_no_type_annotation_on_field():
function test_logical_name (line 825) | def test_logical_name():
function test_init_property (line 846) | def test_init_property():
function test_init_property_top_level (line 895) | def test_init_property_top_level():
function test_default_init_property (line 917) | def test_default_init_property():
function test_init_property_x_field (line 928) | def test_init_property_x_field():
function test_conflicting_superclass_no_fields_in_base (line 943) | def test_conflicting_superclass_no_fields_in_base():
function test_conflicting_superclass_field_in_base (line 1016) | def test_conflicting_superclass_field_in_base():
function test_conflicting_superclass_x_field_in_base (line 1059) | def test_conflicting_superclass_x_field_in_base():
function test_field_clobbering_in_same_class (line 1101) | def test_field_clobbering_in_same_class():
function test_dataclass_errors (line 1131) | def test_dataclass_errors():
function test_cloudpickle_main (line 1142) | def test_cloudpickle_main():
function test_protocol (line 1181) | def test_protocol():
function test_abc (line 1199) | def test_abc():
function test_pretty_format (line 1217) | def test_pretty_format():
function test_metadata (line 1277) | def test_metadata():
function test_traverse (line 1285) | def test_traverse():
function test_int_dict_keys (line 1316) | def test_int_dict_keys():
FILE: tests/test_factories.py
class A (line 14) | class A: ...
class B (line 17) | class B(A): ...
class C (line 23) | class C(B): ...
class X (line 26) | class X: ...
function foo (line 29) | def foo():
function test_standard_subclass (line 39) | def test_standard_subclass():
function test_standard_subclass_unspecified (line 65) | def test_standard_subclass_unspecified():
function test_standard_subclass_module (line 82) | def test_standard_subclass_module():
function test_standard_subclass_object_any (line 115) | def test_standard_subclass_object_any():
function test_standard_type_subclass (line 146) | def test_standard_type_subclass():
function test_standard_type_subclass_unspecified (line 163) | def test_standard_type_subclass_unspecified():
function test_standard_type_subclass_module (line 180) | def test_standard_type_subclass_module():
function test_standard_union (line 191) | def test_standard_union():
function test_standard_union_unspecified (line 210) | def test_standard_union_unspecified():
function test_standard_union_optional (line 229) | def test_standard_union_optional():
function test_standard_union_module (line 246) | def test_standard_union_module():
function test_standard_union_type (line 262) | def test_standard_union_type():
function test_standard_type_generic (line 278) | def test_standard_type_generic():
function test_standard_lambda (line 285) | def test_standard_lambda():
function test_standard_none (line 291) | def test_standard_none():
function test_standard_special_forms (line 297) | def test_standard_special_forms():
function test_standard_subclass_duplicate (line 307) | def test_standard_subclass_duplicate():
FILE: tests/test_munge.py
function test_munger (line 9) | def test_munger():
function test_munger_call_count (line 25) | def test_munger_call_count():
function test_munger_conflict (line 47) | def test_munger_conflict():
function test_munge_recursive (line 61) | def test_munge_recursive():
function test_munger_combinators (line 73) | def test_munger_combinators():
function test_munger_x_type (line 97) | def test_munger_x_type():
function test_munger_freeze_dict (line 130) | def test_munger_freeze_dict():
function test_converter (line 144) | def test_converter():
function test_converter_and_munger (line 156) | def test_converter_and_munger():
function test_converter_fn (line 167) | def test_converter_fn():
function if_none_fn (line 180) | def if_none_fn(default: _T) -> Callable[[_T | None], _T]:
function test_converter_fn_typed (line 187) | def test_converter_fn_typed():
function test_converter_freeze_dict (line 197) | def test_converter_freeze_dict():
FILE: tests/test_tiepin.py
function test_type_repr (line 26) | def test_type_repr():
function test_is_subtype_instance_basic (line 45) | def test_is_subtype_instance_basic():
function test_is_subtype_instance_user_defined_generic (line 71) | def test_is_subtype_instance_user_defined_generic():
function test_is_subtype_instance_user_defined_generic_abc (line 85) | def test_is_subtype_instance_user_defined_generic_abc():
function test_is_subtype_instance_duck_type (line 117) | def test_is_subtype_instance_duck_type():
function test_is_subtype_instance_any (line 135) | def test_is_subtype_instance_any():
function test_is_subtype_instance_list_abc (line 151) | def test_is_subtype_instance_list_abc():
function test_is_subtype_instance_iterable (line 186) | def test_is_subtype_instance_iterable():
function test_is_subtype_instance_tuple (line 232) | def test_is_subtype_instance_tuple():
function test_is_subtype_instance_mapping (line 253) | def test_is_subtype_instance_mapping():
function test_is_subtype_instance_typed_dict (line 280) | def test_is_subtype_instance_typed_dict():
function test_is_subtype_instance_typed_dict_required (line 305) | def test_is_subtype_instance_typed_dict_required():
function test_is_subtype_instance_named_tuple (line 347) | def test_is_subtype_instance_named_tuple():
function test_is_subtype_instance_type_var (line 360) | def test_is_subtype_instance_type_var():
function test_is_subtype_instance_union (line 388) | def test_is_subtype_instance_union():
function test_is_subtype_instance_callable (line 409) | def test_is_subtype_instance_callable() -> None:
function test_is_subtype_instance_callable_protocol (line 506) | def test_is_subtype_instance_callable_protocol():
function test_is_subtype_instance_protocol_chz_callable (line 626) | def test_is_subtype_instance_protocol_chz_callable():
function test_is_subtype_instance_protocol_attr (line 646) | def test_is_subtype_instance_protocol_attr():
function test_is_subtype_instance_runtime_protocol (line 676) | def test_is_subtype_instance_runtime_protocol():
function test_is_subtype_instance_literal (line 689) | def test_is_subtype_instance_literal():
function test_is_subtype_instance_type (line 703) | def test_is_subtype_instance_type():
function test_is_subtype_instance_enum (line 726) | def test_is_subtype_instance_enum():
function test_is_subtype_instance_new_type (line 742) | def test_is_subtype_instance_new_type():
function test_is_subtype_instance_literal_string (line 749) | def test_is_subtype_instance_literal_string():
function test_is_subtype_instance_explicit_protocol_lsp_violation (line 754) | def test_is_subtype_instance_explicit_protocol_lsp_violation():
function test_is_subtype_instance_pydantic (line 768) | def test_is_subtype_instance_pydantic() -> None:
function test_is_subtype_instance_pydantic_utils (line 780) | def test_is_subtype_instance_pydantic_utils() -> None:
function test_is_subtype (line 821) | def test_is_subtype():
function test_is_subtype_protocol (line 884) | def test_is_subtype_protocol():
function test_is_subtype_typed_dict (line 911) | def test_is_subtype_typed_dict():
function test_is_subtype_typevar (line 946) | def test_is_subtype_typevar() -> None:
function test_no_return (line 974) | def test_no_return():
function test_try_cast_object_any (line 993) | def test_try_cast_object_any():
function test_try_cast_tuple (line 1005) | def test_try_cast_tuple():
function test_try_cast_list (line 1019) | def test_try_cast_list():
function test_try_cast_sequence_iterable (line 1035) | def test_try_cast_sequence_iterable():
function test_try_cast_dict (line 1063) | def test_try_cast_dict():
function test_try_cast_callable (line 1077) | def test_try_cast_callable():
function test_try_cast_tuple_unpack (line 1103) | def test_try_cast_tuple_unpack():
function test_try_cast_union_overlap (line 1130) | def test_try_cast_union_overlap():
function test_try_cast_enum (line 1150) | def test_try_cast_enum():
function test_try_cast_fractions (line 1166) | def test_try_cast_fractions():
function test_try_cast_pathlib (line 1172) | def test_try_cast_pathlib():
function test_try_cast_typevar (line 1176) | def test_try_cast_typevar():
function test_approx_type_hash (line 1189) | def test_approx_type_hash():
function test_simplistic_type_of_value (line 1221) | def test_simplistic_type_of_value():
FILE: tests/test_todo.py
function test_version (line 8) | def test_version():
FILE: tests/test_validate.py
function test_validate_readme (line 12) | def test_validate_readme():
function test_validate (line 29) | def test_validate():
function test_validate_replace (line 63) | def test_validate_replace():
function test_for_all_fields (line 74) | def test_for_all_fields():
function test_validate_inheritance_field_level (line 93) | def test_validate_inheritance_field_level():
function test_validate_init_property (line 120) | def test_validate_init_property():
function test_validate_init_property_order (line 147) | def test_validate_init_property_order():
function test_validate_munger (line 160) | def test_validate_munger():
function test_validate_ge_le (line 178) | def test_validate_ge_le() -> None:
function test_validate_inheritance_class_level (line 196) | def test_validate_inheritance_class_level():
function test_validate_decorator_option (line 243) | def test_validate_decorator_option():
function test_validate_mixins (line 275) | def test_validate_mixins():
function test_validate_valid_regex (line 305) | def test_validate_valid_regex():
function test_validate_literal (line 317) | def test_validate_literal():
function test_validate_const_default (line 330) | def test_validate_const_default():
function test_validate_field_consistency (line 347) | def test_validate_field_consistency():
function test_is_override_catches_non_overriding (line 422) | def test_is_override_catches_non_overriding() -> None:
function test_is_override_catches_bad_types (line 438) | def test_is_override_catches_bad_types() -> None:
function test_is_override_mixin_catches_bad_types (line 495) | def test_is_override_mixin_catches_bad_types() -> None:
function test_is_override_catches_bad_generic_default_factory (line 542) | def test_is_override_catches_bad_generic_default_factory() -> None:
function test_is_override_works_with_default_factory (line 573) | def test_is_override_works_with_default_factory() -> None:
function test_is_override_mixin_catches_bad_types_in_subclasses (line 598) | def test_is_override_mixin_catches_bad_types_in_subclasses() -> None:
function test_is_override_mixin_works_on_field_default (line 627) | def test_is_override_mixin_works_on_field_default() -> None:
function test_is_override_mixin_works_with_x_fields (line 667) | def test_is_override_mixin_works_with_x_fields() -> None:
Condensed preview — 48 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (464K chars).
[
{
"path": ".gitignore",
"chars": 103,
"preview": "__pycache__/\n*.py[cod]\n\n.DS_Store\n\n.env\n.venv\nenv/\nvenv/\n\nbuild/\ndist/\n*.egg-info/\n\n.tox/\n.mypy_cache/\n"
},
{
"path": "CHANGELOG.md",
"chars": 10784,
"preview": "# Changelog\n\n## November 2025\n\n- fix most tests on Python 3.14\n- support cast to `datetime.datetime`\n- improve `is_subty"
},
{
"path": "LICENSE",
"chars": 1063,
"preview": "MIT License\n\nCopyright (c) 2024 OpenAI\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof "
},
{
"path": "README.md",
"chars": 1086,
"preview": "# 🪤 chz\n\n*(pronounced \"चीज़\")*\n\n`chz` helps you manage configuration, particularly from the command line.\n\n`chz` is avail"
},
{
"path": "chz/__init__.py",
"chars": 1650,
"preview": "from typing import TYPE_CHECKING, Callable, TypeVar, overload\n\nfrom . import blueprint, factories, mungers, tiepin, vali"
},
{
"path": "chz/blueprint/__init__.py",
"chars": 1314,
"preview": "from chz.blueprint._argv import argv_to_blueprint_args as argv_to_blueprint_args\nfrom chz.blueprint._argv import beta_ar"
},
{
"path": "chz/blueprint/_argmap.py",
"chars": 11892,
"preview": "from __future__ import annotations\n\nimport bisect\nimport re\nfrom dataclasses import dataclass\nfrom typing import TYPE_CH"
},
{
"path": "chz/blueprint/_argv.py",
"chars": 4738,
"preview": "from __future__ import annotations\n\nimport itertools\nimport types\nfrom typing import Any, TypeVar\n\nimport chz.blueprint\n"
},
{
"path": "chz/blueprint/_blueprint.py",
"chars": 52361,
"preview": "from __future__ import annotations\n\nimport ast\nimport collections.abc\nimport dataclasses\nimport functools\nimport inspect"
},
{
"path": "chz/blueprint/_entrypoint.py",
"chars": 7185,
"preview": "from __future__ import annotations\n\nimport functools\nimport inspect\nimport io\nimport os\nimport sys\nfrom typing import An"
},
{
"path": "chz/blueprint/_lazy.py",
"chars": 4336,
"preview": "import collections\nfrom typing import AbstractSet, Any, Callable, TypeVar\n\nfrom chz.blueprint._entrypoint import Invalid"
},
{
"path": "chz/blueprint/_wildcard.py",
"chars": 3301,
"preview": "import re\n\n_FUZZY_SIMILARITY = 0.6\n\n\ndef wildcard_key_to_regex_str(key: str) -> str:\n if key.endswith(\"...\"):\n "
},
{
"path": "chz/data_model.py",
"chars": 27074,
"preview": "\"\"\"\n\nThis is the core implementation of the chz class. It's based off of the implementation of\ndataclasses, but is somew"
},
{
"path": "chz/factories.py",
"chars": 23080,
"preview": "import ast\nimport collections\nimport functools\nimport importlib\nimport re\nimport sys\nimport types\nimport typing\nfrom typ"
},
{
"path": "chz/field.py",
"chars": 11855,
"preview": "from __future__ import annotations\n\nimport functools\nimport sys\nfrom typing import Any, Callable\n\nimport chz\nfrom chz.mu"
},
{
"path": "chz/mungers.py",
"chars": 2486,
"preview": "from __future__ import annotations\n\nfrom typing import TYPE_CHECKING, Any, Callable, Mapping, TypeVar, overload\n\nif TYPE"
},
{
"path": "chz/py.typed",
"chars": 0,
"preview": ""
},
{
"path": "chz/tiepin.py",
"chars": 40311,
"preview": "\"\"\"\n\nIt's a fair question why this module exists, instead of using something third party.\n\nThere are two things I would "
},
{
"path": "chz/universal.py",
"chars": 70,
"preview": "if __name__ == \"__main__\":\n import chz\n\n chz.entrypoint(object)\n"
},
{
"path": "chz/util.py",
"chars": 103,
"preview": "class MISSING_TYPE:\n def __repr__(self) -> str:\n return \"MISSING\"\n\n\nMISSING = MISSING_TYPE()\n"
},
{
"path": "chz/validators.py",
"chars": 9154,
"preview": "from __future__ import annotations\n\nimport collections\nimport collections.abc\nimport re\nfrom typing import Any, Callable"
},
{
"path": "docs/01_quickstart.md",
"chars": 654,
"preview": "## Quick start\n\nTurn any function into a command line tool:\n\n```python\nimport chz\n\ndef main(name: str, age: int) -> None"
},
{
"path": "docs/02_object_model.md",
"chars": 3898,
"preview": "## Declarative object model\n\nIn the beginning there was `attrs`... although people may be more familiar with its strippe"
},
{
"path": "docs/03_validation.md",
"chars": 3717,
"preview": "## Validation\n\n`chz` supports validation in a manner similar to `attrs`, but slightly nicer for class-level\nvalidation. "
},
{
"path": "docs/04_command_line.md",
"chars": 10413,
"preview": "## Command line parsing\n\nType aware CLIs are really great.\nThese let you focus on writing code, with types, and you get "
},
{
"path": "docs/05_blueprint.md",
"chars": 7293,
"preview": "## Blueprints and partial application\n\n`chz` has a `Blueprint` mechanism that powers the command line functionality. `ch"
},
{
"path": "docs/06_serialisation.md",
"chars": 1887,
"preview": "## Serialisation and deserialisation\n\n`chz` will one day have a great story for versioned serialisation and deserialisat"
},
{
"path": "docs/21_post_init.md",
"chars": 6315,
"preview": "## No `__post_init__`; details and examples\n\nThere are a couple reasons why `chz` does not have a `__post_init__` equiva"
},
{
"path": "docs/22_field_api.md",
"chars": 2691,
"preview": "## `chz.field`\n\n`chz.field` takes the following parameters:\n\n#### `default`\n\nLike with `dataclasses`, the default value "
},
{
"path": "docs/91_philosophy.md",
"chars": 3452,
"preview": "# Philosophy\n\nThere are a few different ideas in `chz` and not all of them are equally valuable or well designed.\n\nIn pa"
},
{
"path": "docs/92_alternatives.md",
"chars": 1800,
"preview": "## Alternatives\n\nThe most common question I get when someone first sees `chz` is \"...but have you heard about X?\"\n\nHere "
},
{
"path": "docs/93_testimonials.md",
"chars": 988,
"preview": "## Testimonials\n\nUnsolicited feedback from users of `chz`. To be honest, I'm surprised people like it this much:\n\n> “pre"
},
{
"path": "pyproject.toml",
"chars": 1567,
"preview": "[project]\nname = \"chz\"\nversion = \"0.4.0\"\ndescription = \"chz is a library for managing configuration\"\nreadme = \"README.md"
},
{
"path": "tests/test_blueprint.py",
"chars": 21662,
"preview": "import pytest\n\nimport chz\nfrom chz.blueprint import (\n Castable,\n ConstructionException,\n ExtraneousBlueprintAr"
},
{
"path": "tests/test_blueprint_cast.py",
"chars": 4982,
"preview": "# Note test_blueprint_meta_factory.py also contains tests relevant to casting\nfrom typing import Literal\n\nimport pytest\n"
},
{
"path": "tests/test_blueprint_errors.py",
"chars": 1832,
"preview": "import pytest\n\nimport chz\nfrom chz.blueprint import ConstructionException, ExtraneousBlueprintArg\n\n\ndef test_target_bad_"
},
{
"path": "tests/test_blueprint_meta_factory.py",
"chars": 12518,
"preview": "import typing\nfrom typing import Optional\n\nimport pytest\n\nimport chz\nfrom chz.blueprint import Castable, InvalidBlueprin"
},
{
"path": "tests/test_blueprint_methods.py",
"chars": 3714,
"preview": "import re\nimport textwrap\nfrom unittest.mock import patch\n\nimport pytest\n\nimport chz\nfrom chz.blueprint import Entrypoin"
},
{
"path": "tests/test_blueprint_reference.py",
"chars": 3921,
"preview": "import pytest\n\nimport chz\nfrom chz.blueprint import InvalidBlueprintArg, MissingBlueprintArg, Reference\n\n\ndef test_bluep"
},
{
"path": "tests/test_blueprint_root_polymorphism.py",
"chars": 1833,
"preview": "import re\n\nimport chz\n\n\ndef test_root_polymorphism():\n @chz.chz\n class X:\n a: int\n b: str = \"str\"\n\n "
},
{
"path": "tests/test_blueprint_unit.py",
"chars": 10060,
"preview": "import pytest\n\nfrom chz.blueprint import Blueprint, Castable, beta_argv_arg_to_string, beta_blueprint_to_argv\nfrom chz.b"
},
{
"path": "tests/test_blueprint_variadic.py",
"chars": 13380,
"preview": "import typing\n\nimport pytest\n\nimport chz\nfrom chz.blueprint import (\n Castable,\n ConstructionException,\n Extran"
},
{
"path": "tests/test_data_model.py",
"chars": 30802,
"preview": "# ruff: noqa: F811\nimport dataclasses\nimport functools\nimport json\nimport re\nimport typing\n\nimport pytest\n\nimport chz\n\n\n"
},
{
"path": "tests/test_factories.py",
"chars": 9842,
"preview": "\"\"\"\n\nWatch out for some of the extra parentheses in these tests.\n\n\"\"\"\n\nimport typing\n\nimport pytest\n\nfrom chz.factories "
},
{
"path": "tests/test_munge.py",
"chars": 4574,
"preview": "from typing import Any, Callable, TypedDict, TypeVar\n\nimport pytest\n\nimport chz\nfrom chz.mungers import attr_if_none, if"
},
{
"path": "tests/test_tiepin.py",
"chars": 44221,
"preview": "# ruff: noqa: UP006\n# ruff: noqa: UP007\n# ruff: noqa: UP045\nimport collections.abc\nimport enum\nimport fractions\nimport p"
},
{
"path": "tests/test_todo.py",
"chars": 401,
"preview": "import pytest\n\nimport chz\n\n# TODO: test inheritance, setattr, repr\n\n\ndef test_version():\n @chz.chz(version=\"b4d37d6e\""
},
{
"path": "tests/test_validate.py",
"chars": 19078,
"preview": "import math\nimport re\nfrom typing import Generic, TypeVar\n\nimport pytest\n\nimport chz\n\nT = TypeVar(\"T\")\n\n\ndef test_valida"
}
]
About this extraction
This page contains the full source code of the openai/chz GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 48 files (431.1 KB), approximately 112.7k tokens, and a symbol index with 495 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.